Small Datasets Research Articles

DsAMP and dsAMPGAN: Deep Learning Networks for Antimicrobial Peptides Recognition and Generation.

Antibiotic resistance is a growing public health challenge. Antimicrobial peptides (AMPs) effectively target microorganisms through non-specific mechanisms, limiting their ability to develop resistance. Therefore, the prediction and design of new AMPs is crucial. Recently, deep learning has spurred interest in computational approaches to peptide drug discovery. This study presents a novel deep learning framework for AMP classification, function prediction, and generation. We developed discoverAMP (dsAMP), a robust AMP predictor using CNN Attention BiLSTM and transfer learning, which outperforms existing classifiers. In addition, dsAMPGAN, a Generative Adversarial Network (GAN)-based model, generates new AMP candidates. Our results demonstrate the superior performance of dsAMP in terms of sensitivity, specificity, Matthew correlation coefficient, accuracy, precision, F1 score, and area under the ROC curve, achieving >95% classification accuracy with transfer learning on a small dataset. Furthermore, dsAMPGAN successfully synthesizes AMPs similar to natural ones, as confirmed by comparisons of physical and chemical properties. This model serves as a reliable tool for the identification of novel AMPs in clinical settings and supports the development of AMPs to effectively combat antibiotic resistance.

Defective Pennywort Leaf Detection Using Machine Vision and Mask R-CNN Model

Demand and market value for pennywort largely depend on the quality of the leaves, which can be affected by various ambient environment or fertigation variables during cultivation. Although early detection of defects in pennywort leaves would enable growers to take quick action, conventional manual detection is laborious and time consuming as well as subjective. Therefore, the objective of this study was to develop an automatic leaf defect detection algorithm for pennywort plants grown under controlled environment conditions, using machine vision and deep learning techniques. Leaf images were captured from pennywort plants grown in an ebb-and-flow hydroponic system under fluorescent light conditions in a controlled plant factory environment. Physically or biologically damaged leaves (e.g., curled, creased, discolored, misshapen, or brown spotted) were classified as defective leaves. Images were annotated using an online tool, and Mask R-CNN models were implemented with the integrated attention mechanisms, convolutional block attention module (CBAM) and coordinate attention (CA) and compared for improved image feature extraction. Transfer learning was employed to train the model with a smaller dataset, effectively reducing processing time. The improved models demonstrated significant advancements in accuracy and precision, with the CA-augmented model achieving the highest metrics, including a mean average precision (mAP) of 0.931 and an accuracy of 0.937. These enhancements enabled more precise localization and classification of leaf defects, outperforming the baseline Mask R-CNN model in complex visual recognition tasks. The final model was robust, effectively distinguishing defective leaves in challenging scenarios, making it highly suitable for applications in precision agriculture. Future research can build on this modeling framework, exploring additional variables to identify specific leaf abnormalities at earlier growth stages, which is crucial for production quality assurance.

Efficient self-attention based joint optimization for lithology and petrophysical parameter estimation in the Athabasca Oil Sands

Accurately identifying lithology and petrophysical parameters, such as porosity and water saturation, are essential in reservoir characterization. Manual interpretation of well-log data, the conventional approach, is not only labor-intensive but also susceptible to human errors. To address these challenges of lithology identification and petrophysical parameter estimation in the Athabasca Oil Sands area, this study introduces an AutoRegressive Vision Transformer (ARViT) model for lithology and petrophysical parameter prediction. The effectiveness of ARViT lies in its self-attention mechanism and its ability to handle data sequentially, allowing the model to capture important spatial dependencies within the well-log data. This mechanism enables the model to identify subtle spatial and temporal relationships among various geophysical measurements. The model is also interpretable and can serve as an assistive tool for geoscientists, enabling faster interpretation while reducing human bias. The interpretable nature of the model should assist geoscientists in conducting faster quality checks of the predictions, ensuring that errors are not propagated to subsequent stages. This study adopts a multitask learning approach, jointly optimizing the model's performance across multiple tasks simultaneously. To evaluate the effectiveness of the ARViT model, we conducted series of experiments and comparisions, testing it against traditional artificial neural networks (ANN), Long Short-Term Memory (LSTM), and Vision Transformer (ViT) models. To showcase the versatility of ARViT, we apply Low-Rank Adaptation (LoRA) to a different smaller dataset, showing its potential to adapt to different geological contexts. LoRA not only helps in model adaptability but also helps to reduce the number of trainable parameters. Our findings demonstrate that ARViT outperforms ANN, LSTM, and ViT in estimating lithological and petrophysical parameters. While lithology prediction has been a well-explored field, ARViT's unique blend of features, including its self-attention mechanism, autoregression, and multitask approach along with efficient fine tuning using LoRA, sets it apart as a valuable tool for the complex task of lithology prediction and petrophysical parameter estimation.

Pose‐to‐Motion: Cross‐Domain Motion Retargeting with Pose Prior

AbstractCreating plausible motions for a diverse range of characters is a long‐standing goal in computer graphics. Current learning‐based motion synthesis methods rely on large‐scale motion datasets, which are often difficult if not impossible to acquire. On the other hand, pose data is more accessible, since static posed characters are easier to create and can even be extracted from images using recent advancements in computer vision. In this paper, we tap into this alternative data source and introduce a neural motion synthesis approach through retargeting, which generates plausible motion of various characters that only have pose data by transferring motion from one single existing motion capture dataset of another drastically different characters. Our experiments show that our method effectively combines the motion features of the source character with the pose features of the target character, and performs robustly with small or noisy pose data sets, ranging from a few artist‐created poses to noisy poses estimated directly from images. Additionally, a conducted user study indicated that a majority of participants found our retargeted motion to be more enjoyable to watch, more lifelike in appearance, and exhibiting fewer artifacts. Our code and dataset can be accessed here.

Innovative application of transfer learning on small-scale datasets: Analysis and optimization of catalyst ink for the low-iridium membrane electrode assemblies of proton exchange membrane water electrolysis

The scarcity of Ir poses a significant barrier to the widespread adoption of proton exchange membrane water electrolysis (PEMWE), necessitating minimization of Ir loading in membrane electrode assemblies (MEAs). This study introduced a transfer learning strategy to optimize catalyst ink formulations for performance enhancement of low-Ir MEAs. We leveraged a pre-trained model from previous research and fine-tuned it on a small dataset of 12 MEAs’ ink preparation data, which significantly reduced labor, time, and costs associated with extensive experimentation. The developed model demonstrated high accuracy with an R2 of 0.99507. The parameters of the ink formulation were assessed using the Shapley Additive exPlanations method and optimized through the Harris Hawk Optimization algorithm. This optimization yielded an electrolysis voltage of 1.898 V at 3.0 A cm−2 for MEAs with 0.15 mg cm−2 Ir loading. These performance improvements are attributed to enhanced catalyst-ionomer particle dispersion and optimized rheological properties of the ink, which lead to more uniform distribution of Ir in the catalytic layers. These results demonstrate the feasibility and efficacy of transfer learning techniques to develop high-performance models from limited datasets, offering a promising pathway for accelerated materials discovery and optimization in the field of PEMWE.

Segmentation of Glacier Area Using U-Net through Landsat Satellite Imagery for Quantification of Glacier Recession and Its Impact on Marine Systems

Glaciers have experienced a global trend of recession within the past century. Quantification of glacier variations using satellite imagery has been of great interest due to the importance of glaciers as freshwater resources and as indicators of climate change. Spatiotemporal glacier dynamics must be monitored to quantify glacier variations. The potential methods to quantify spatiotemporal glacier dynamics with increasing complexity levels include detecting the terminus location, measuring the length of the glacier from the accumulation zone to the terminus, quantifying the glacier surface area, and measuring glacier volume. Although some deep learning methods designed purposefully for glacier boundary segmentation have achieved acceptable results, these models are often localized to the region where their training data were acquired and further rely on the training sets that were often curated manually to highlight glacial regions. Due to the very large number of glaciers, it is practically impossible to perform a worldwide study of glacier dynamics using manual methods. As a result, an automated or semi-automated method is highly desirable. The current study has built upon our previous works moving towards identification methods of the 2D glacier profile for glacier area segmentation. In this study, a deep learning method is proposed for segmentation of temporal Landsat images to quantify the glacial region within the Mount Cook/Aoraki massif located in the Southern Alps/Kā Tiritiri o te Moana of New Zealand/Aotearoa. Segmented glacial regions can be further utilized to determine the relationship of their variations due to climate change. This model has demonstrated promising performance while trained on a relatively small dataset. The permanent ice and snow class was accurately segmented at a 92% rate by the proposed model.

Automatic Acne Severity Grading with a Small and Imbalanced Data Set of Low-Resolution Images.

Developing automatic acne vulgaris grading systems based on machine learning is an expensive endeavor in terms of data acquisition. A machine learning practitioner will need to gather high-resolution pictures from a considerable number of different patients, with a well-balanced distribution between acne severity grades and potentially very tedious labeling. We developed a deep learning model to grade acne severity with respect to the Investigator's Global Assessment (IGA) scale that can be trained on low-resolution images, with pictures from a small number of different patients, a strongly imbalanced severity grade distribution and minimal labeling. A total of 1374 triplets of images (frontal and lateral views) from 391 different patients suffering from acne labeled with the IGA severity grade by an expert dermatologist were used to train and validate a deep learning model that predicts the IGA severity grade. On the test set we obtained 66.67% accuracy with an equivalent performance for all grades despite the highly imbalanced severity grade distribution of our database. Importantly, we obtained performance on par with more tedious methods in terms of data acquisition which have the same simple labeling as ours but require either a more balanced severity grade distribution or large numbers of high-resolution images. Our deep learning model demonstrated promising accuracy despite the limited data set on which it was trained, indicating its potential for further development both as an assistance tool for medical practitioners and as a way to provide patients with an immediately available and standardized acne grading tool. chinadrugtrials.org.cn identifier CTR20211314.

A double-layered fully automated insomnia identification model employing synthetic data generation using MCSA and CTGAN with single-channel EEG signals.

Insomnia was diagnosed by analyzing sleep stages obtained during polysomnography (PSG) recording. The state-of-the-art insomnia detection models that used physiological signals in PSG were successful in classification. However, the sleep stages of unbalanced data in small-time intervals were fed for classification in previous studies. This can be avoided by analyzing the insomnia detection structure in different frequency bands with artificially generated data from the existing one at the preprocessing and post-processing stages. Hence, the paper proposes a double-layered augmentation model using Modified Conventional Signal Augmentation (MCSA) and a Conditional Tabular Generative Adversarial Network (CTGAN) to generate synthetic signals from raw EEG and synthetic data from extracted features, respectively, in creating training data. The presented work is independent of sleep stage scoring and provides double-layered data protection with the utility of augmentation methods. It is ideally suited for real-time detection using a single-channel EEG provides better mobility and comfort while recording. The work analyzes each augmentation layer's performance individually, and better accuracy was observed when merging both. It also evaluates the augmentation performance in various frequency bands, which are decomposed using discrete wavelet transform, and observed that the alpha band contributes more to detection. The classification is performed using Decision Tree (DT), Ensembled Bagged Decision Tree (EBDT), Gradient Boosting (GB), Random Forest (RF), and Stacking classifier (SC), attaining the highest classification accuracy of 94% using RF with a greater Area Under Curve (AUC) value of 0.97 compared to the existing works and is best suited for small datasets.

Corrosion prediction model for long-distance pipelines based on NLFE-NGO-ELM

Abstract To improve the accuracy of buried pipeline corrosion rate prediction, we developed a Northern Goshawk Optimization-based Extreme Learning Machine (NLFE-NGO-ELM) model enhanced by Non-linear Feature Expansion. Feature selection was used to expand the small dataset and reduce the dimensionality of factors affecting pipeline corrosion. The Northern Goshawk algorithm was used, and 106 sets of pipeline external corrosion detection data were simulated in MATLAB as a case study. The ELM optimizer was analyzed using Grey Wolf Optimization (GWO-ELM), Northern Goshawk Optimization (NGO-ELM), and BP optimization as comparison models. Using NLFE-NGO-ELM significantly improved the model's prediction accuracy and speed. The NGO-ELM model was applied to predict the corrosion rates of 21 pipelines, achieving an absolute error of only 2.40% and an R value of 0.97255, surpassing the predictive performance of NGO-BP and GWO-ELM models. These results indicate that the current model exhibits superior learning performance and stronger fitting capabilities.

Motion control of autonomous underwater vehicle based on physics-informed offline reinforcement learning

Online reinforcement learning (RL) methods for autonomous underwater vehicles (AUV) are time-consuming and unsafe due to the need for real-world interaction. Offline RL methods can improve efficiency and safety by training with dynamic models, but an accurate model for AUV is difficult to obtain due to its highly nonlinear dynamics. These limit the application of RL methods in AUV control. To solve this issue, we propose physics-informed model-based conservative offline policy optimization (PICOPO). It offers the advantages of small dataset, strong generalizability and high safety by combining the physics-informed dynamic modelling method and the offline RL technique. First, the PICOPO constructs a physics-informed model based on a small offline dataset to serve as the digital twins (DT) of the actual AUV. This DT can forecast the long-term motion states of AUV with high-precision. The RL-based controller is then trained offline within this DT, eliminating the need for real-world interaction and allowing direct deployment to the AUV without fine-tuning. In this paper, simulations and field tests are carried out to evaluate the proposed method. Our results demonstrate that PICOPO achieves accurate motion control with just 2000 samples and enables zero-shot sim-to-real transfer, showcasing strong generalizability across various motion control tasks.

Automated Google Scholar Crawling with a Web-Based Tool for Publication Data Management

Here’s the revised abstract in English: The rapid growth of academic publications requires efficient tools for publication data extraction and management, especially from widely used platforms like Google Scholar. To address this need, an automated web-based tool was developed, designed to simplify the processes of data crawling, extraction, and publication data management, allowing researchers to handle large volumes of academic publications more effectively. The tool supports both simple and detailed crawling modes, enabling users to input multiple Google Scholar URLs and neatly organize the extracted data into CSV files. For multiple URLs, the data is compiled into a ZIP file containing separate CSV files for each source, ensuring organized and accessible publication data management. The tool was tested with various dataset sizes. When processing 41 entries, the simple mode completed extraction in 9.054 seconds, while the detailed mode took 71.898 seconds. For smaller datasets of 5 entries, the simple mode executed in 3.283 seconds, while the detailed mode required 11.908 seconds. These results indicate that the tool is efficient and performs well with both small and large datasets. The differences in execution time between the simple and detailed modes offer users flexibility in balancing speed and depth of data extraction according to their research needs. This web-based tool not only automates the data extraction process from Google Scholar but also enhances the organization and accessibility of publication data, making it an asset for researchers and institutions in managing publication data.

Flight Arrival Scheduling via Large Language Model

The flight arrival scheduling problem is one of the critical tasks in air traffic operations, aiming to ensure that the flight arrive in the correct sequence safely. Existing methods primarily focus on the terminal area and often overlook the presence of training flight at the airport. Due to the limited generalization of traditional methods and varying control practices at different airports, training flight at airports still rely on manual control for arrival sorting. To effectively address these issues, we propose a novel method for slot allocation that leverages the strong reasoning capabilities and generalization potential of large language models (LLMs). Our method conceptualizes the dynamic scheduling problem for training flight as a language modeling problem, a perspective not previously explored. Specifically, we represent the allocator’s inputs and outputs as language tokens, utilizing LLMs to generate conflict-free results based on a language description of requested landing information and assigned training flight information. Additionally, we employ a reset strategy to create a small dataset for scenario-specific samples, enabling LLMs to quickly learn allocation schemes from the dataset. We demonstrated the capability of LLMs in addressing time conflicts by evaluating metrics such as answer accuracy, conflict rate, and total delay time (without the wrong answer). These findings underscore the feasibility of employing LLMs in the field of air traffic control.

Explainable deep learning for glaucoma detection: Efficient dense net with quantum-based Dwarf Mongoose optimization

Glaucoma, a progressive eye disease, leads to irreversible vision loss due to optic nerve damage. This study aims to develop a more explainable and interpretable deep learning-based model for the detection and classification of glaucoma, addressing the current limitations in the field. The proposed approach utilizes a modified DenseNet-201 architecture, incorporating an additional transition layer for enhanced glaucoma classification. A pre-trained Efficient DenseNet is applied, and a reweighted cross-entropy loss function is used to handle the class imbalance in the training data. Quantum-based Dwarf Mongoose Optimization (QDMOA) is employed to fine-tune the model’s parameters, minimizing over fitting and improving generalization on small datasets. Experimental results demonstrate that the proposed method effectively detects and classifies two types of glaucoma. The use of dense connections with regularization significantly reduces over fitting, and the reweighted loss function improves model performance on imbalanced data. The developed deep learning model offers improved accuracy in glaucoma detection and classification while addressing the need for more interpretable and reliable models in medical applications. This research provides a practical tool for automated glaucoma screening, which could support ophthalmologists in early diagnosis and treatment, reducing the risk of irreversible vision loss.

A novel highly efficient alternating direction method of multipliers for large-scale trimmed concave SVM

Support vector machine (SVM) is widely utilized for classification in diverse fields thanks to its superior performance. However, when tackling large-scale SVM problems, it encounters high computational complexity. To address this issue, we develop a novel trimmed concave loss SVM model called Ltri-SVM, which achieves sparsity and robustness simultaneously. The new optimality theory for the nonconvex and nonsmooth Ltri-SVM is developed by our new constructed proximal stationary point. Based on which, a new fast alternating direction method of multipliers (ADMM) with working set and low computational complexity is established for addressing Ltri-SVM. Based on our new established optimality theory, our proposed algorithm will divides the training dataset into two distinct categories: the non-working and the working sets. In every training iteration, the algorithm modifies the parameters associated with the working set, while ensuring that the parameters belonging to the non-working set remain unchanged. This algorithm enables the updating of parameters on smaller datasets, which in turn decreases computational complexity and improves the efficiency of runtime. Further, we prove that our algorithm is global convergence. Numerical experiments have confirmed the excellent performance of our algorithm in terms of accuracy in classification, number of support vector and computational speed, surpassing nine other leading solvers. For instance, when solving the real dataset with over 107 samples, our algorithm is able to complete the task in just 18.92 s, which is much faster than other solvers that take at least 650.4 s.

Fault diagnosis of planetary gearboxes under small and imbalanced samples based on enhanced Siamese network with improved downsampling module

Due to the limitations of the operating conditions and equipment status, in practice, the normal data for planetary gearboxes are often sufficient, but the fault data are scarce and difficult to obtain, causing the fault diagnosis to face not only an imbalance problem but also a small sample problem. It leads to serious misdiagnosis and poor generalizability. To overcome above challenges, an enhanced Siamese network with improved downsampling module is proposed to reduce the negative impact of small and imbalanced datasets on fault diagnosis. Firstly, the Siamese network is combined with data enhancement to adaptively increase the fault training data with different small sample degrees. It can make the number of training data of each class get rid of small sample, while avoiding the excessive use of scarce fault data causing additional overfitting, so as to improve the generalization of the model. Among them, the data enhancement adopts the extreme noise expansion method proposed in this work, which can quickly generate samples that meet the requirements and help to improve the performance of the model in noisy environments. Secondly, adding the improved downsampling module to the Siamese network. The improved downsampling module introduces density-based spatial clustering of applications with noise (DBSCAN) and distribution ratio calculation, which can consider the distribution range and density of data when selecting. It can not only balance the training data, but also avoid serious information loss, and then reduce the probability of misdiagnosis. Finally, using six imbalanced planetary gearbox datasets with different small sample degrees for verification. The experimental results show that the proposed method can efficiently increase and balance the training data, which greatly improves the accuracy of fault diagnosis under the condition of small and imbalanced samples, especially when the small sample degree is very serious, the accuracy is improved by 57.76% on average.

MS-CGAN: Fusion of conditional generative adversarial networks and multi-scale spatio-temporal features for lithology identification

Lithology identification constitutes a crucial undertaking in formation evaluation and reservoir characterization. However, the need for improved precision arises in conventional lithology identification models due to the difficulties presented by unequal distributions of small-sample logging data. An effective combination of domain expertise and data-driven models to predict lithology is essential due to the intricate and nonlinear connection between logging parameters and lithology, combined with the distinct characteristics of the oilfield environments. In this paper, we proposed a multi-scale conditional generative adversarial network(MS-CGAN) method, which combines conditional generative adversarial networks with multi-scale spatio-temporal features to address data imbalance issues and enhance the accuracy of lithology classification. Our approach, tested on two small datasets from the Hugoton and Panoma fields, USA, and the Daqing production wells, China, stands out as the optimal choice compared to other models. Comprehensive evaluation results indicate promising practical applications and potential benefits of the new model in enhancing lithology identification using limited data.

CTCTime: A New Model for Unidimensional Time Series Classification

One-dimensional time series classification has always been an important research direction, playing an irreplaceable role in various fields. With the rapid development of deep learning, most traditional time series classification methods have gradually been replaced by neural network-based classification methods. At present, in the field of time series classification, models that perform well are mostly based on Convolutional Neural Networks (CNNs), while models with the Transformer architecture, which have shown outstanding performance in natural language processing and computer vision, do not stand out. In order to change this situation, We have investigated the feasibility of training models based on a single Transformer architecture directly on small datasets without additional data processing. Furthermore, we propose a new model called CTCTime, which combines the Transformer architecture with CNNs to address one-dimensional time series classification problems. We compared CTCTime with 13 traditional algorithms on 44 datasets from the UCR archive and with 7 advanced methods on 85 datasets. The UCR datasets are classic datasets for one-dimensional time series classification tasks. The experimental results demonstrate its feasibility, accuracy, and scalability.

Response Matching for Generating Materials and Molecules.

Diffusion models have recently emerged as powerful tools for the generation of new molecular and material structures. The key insight is that the noise in these models is related to the response of the atoms to displacement, and the denoising step is thus analogous to the geometry relaxation of atomistic systems starting from a random structure. Building on this, we present a generative method called Response Matching (RM), which leverages the fact that each stable material or molecule exists at the minimum of its potential energy surface. Any perturbation induces a response in energy and stress, driving the structure back to equilibrium. Matching this response is closely related to score matching in diffusion models. Another important aspect of state-of-the-art diffusion models is the incorporation of physical symmetries such as translation, rotation, and periodicity. RM employs a machine learning interatomic potential and random structure search as the denoising model, inherently respecting these symmetries and exploiting the locality of atomic interactions. RM handles both molecules and bulk materials under the same framework. Its efficiency and generalization are demonstrated on three systems: a small organic molecular data set, stable crystals from the Materials Project, and one-shot learning on a single diamond configuration.

Improved Small Object Detection Algorithm CRL-YOLOv5.

Detecting small objects in images poses significant challenges due to their limited pixel representation and the difficulty in extracting sufficient features, often leading to missed or false detections. To address these challenges and enhance detection accuracy, this paper presents an improved small object detection algorithm, CRL-YOLOv5. The proposed approach integrates the Convolutional Block Attention Module (CBAM) attention mechanism into the C3 module of the backbone network, which enhances the localization accuracy of small objects. Additionally, the Receptive Field Block (RFB) module is introduced to expand the model's receptive field, thereby fully leveraging contextual information. Furthermore, the network architecture is restructured to include an additional detection layer specifically for small objects, allowing for deeper feature extraction from shallow layers. When tested on the VisDrone2019 small object dataset, CRL-YOLOv5 achieved an mAP50 of 39.2%, representing a 5.4% improvement over the original YOLOv5, effectively boosting the detection precision for small objects in images.

Coding latent concepts: a human and LLM-coordinated content analysis procedure

ABSTRACT Measuring complex and latent concepts at a large-scale poses significant challenges for communication researchers. While computational and crowdsourced methods offer solutions, they often require high professional thresholds or incur significant costs. The recent advent of large language models has revolutionized content analysis. This paper employs a human and LLM-coordinated analysis procedure to measure complex and latent concepts in 1,000 public comments, exemplified by the multi-dimension concept of “deliberativeness.” We showcase the collaboration between humans and LLMs in completing complex coding tasks by designing and refining a codebook for human use and corresponding prompts for LLMs. Surprisingly, we find that fine-tuned GPT-3.5-turbo-1106 with smaller datasets can surpass GPT-4o-2024-05-13’s performance and match manual content analyses. This paper provides communication researchers with an efficient and cost-effective reference for measuring latent concepts.