Model Training Research Articles

ML-Based Pain Recognition Model Using Mixup Data Augmentation

Machine learning (ML) has revolutionized healthcare by enhancing diagnostic capabilities because of its ability to analyze large datasets and detect minor patterns often overlooked by humans. This is beneficial, especially in pain recognition, where patient communication may be limited. However, ML models often face challenges such as memorization and sensitivity to adversarial examples. Regularization techniques like mixup, which trains models on convex combinations of data pairs, address these issues by enhancing model generalization. While mixup has proven effective in image, speech, and text datasets, its application to time-series signals like electrodermal activity (EDA) is less explored. This research uses ML for pain recognition with EDA signals from the BioVid Heat Pain Database to distinguish pain by applying mixup regularization to manually extracted EDA features and using a support vector machine (SVM) for classification. The results show that this approach achieves an average accuracy of 75.87% using leave-one-subject-out cross-validation (LOSOCV) compared to 74.61% without mixup. This demonstrates mixup’s efficacy in improving ML model accuracy for pain recognition from EDA signals. This study highlights the potential of mixup in ML as a promising approach to enhance pain assessment in healthcare.

Cross-species alignment along the chronological axis reveals evolutionary effect on structural development of the human brain.

Disentangling the evolution mysteries of the human brain has always been an imperative endeavor in neuroscience. Although many previous comparative studies revealed genetic, brain structural and connectivity distinctness between human and other nonhuman primates, the brain evolutional mechanism is still largely unclear. Here, we proposed to embed the brain anatomy of human and macaque in the developmental chronological axis to construct cross-species predictive model to quantitatively characterize brain evolution using two large public human and macaque datasets. We observed that applying the trained models within-species could well predict the chronological age. Interestingly, we found the model trained in macaque showed a higher accuracy in predicting the chronological age of human than the model trained in human in predicting the chronological age of macaque. The cross-application of the trained model introduced an individual brain cross-species age gap index to quantify the cross-species discrepancy along the temporal axis of brain development and was found to be associated with the behavioral performance in visual acuity test and picture vocabulary test in human. Taken together, our study situated the cross-species brain development along the chronological axis, which highlighted the disproportionately anatomical development in human brain to extend our understanding of the potential evolutionary effects.

Short-Term Energy Generation Forecasts at a Wind Farm—A Multi-Variant Comparison of the Effectiveness and Performance of Various Gradient-Boosted Decision Tree Models

High-quality short-term forecasts of wind farm generation are crucial for the dynamically developing renewable energy generation sector. This article addresses the selection of appropriate gradient-boosted decision tree models (GBDT) for forecasting wind farm energy generation with a 10-min time horizon. In most forecasting studies, authors utilize a single gradient-boosted decision tree model and compare its performance with other machine learning (ML) techniques and sometimes with a naive baseline model. This paper proposes a comprehensive comparison of all gradient-boosted decision tree models (GBDTs, eXtreme Gradient Boosting (XGBoost), Light Gradient-Boosting Machine (LightGBM), and Categorical Boosting (CatBoost)) used for forecasting. The objective is to evaluate each model in terms of forecasting accuracy for wind farm energy generation (forecasting error) and computational time during model training. Computational time is a critical factor due to the necessity of testing numerous models with varying hyperparameters to identify the optimal settings that minimize forecasting error. Forecast quality using default hyperparameters is used here as a reference. The research also seeks to determine the most effective sets of input variables for the predictive models. The article concludes with findings and recommendations regarding the preferred GBDT models. Among the four tested models, the oldest GBDT model demonstrated a significantly longer training time, which should be considered a major drawback of this implementation of gradient-boosted decision trees. In terms of model quality testing, the lowest nRMSE error was achieved by the oldest model—GBDT in its tuned version (with the best hyperparameter values obtained from exploring 40,000 combinations).

Referential choice in spoken and written stories: A comparative study based on the corpus “Funny life stories”

The study is concerned with referential choice in spoken and written discourse in the Russian language. I consider referential choice to consist in a threefold opposition between full noun phrases, pronouns, and zero noun phrases. The study is based on discourses each of which was presented by its narrator twice, namely in the spoken and written forms. In each story, all the noun phrases were identified and described according to 29 parameters. I trained logistic regression models and decision trees on the collected samples and analyzed factor importance diagrams built on the basis of the decision trees. The interpretation of the models and diagrams shows that some factors have different impact on referential choice in spoken and written discourses, for instance, grammatical role, semantic hyperrole and sloppy identity between the anaphor and the antecedent. Besides, the models also demonstrate that the sets of significant factors for the two samples are not identical: in particular, the referent’s animacy and the anaphor’s semantic hyperrole are present solely in the decision tree for written discourse.

Identifying crowdfunding storytellers who deliver successful projects: a machine learning approach

Crowdfunding plays a key role in financial technology to provide individuals and enterprises with funding opportunities to establish start-ups and/or new business ventures. It is mainly used to link projects’ creators and backers, collect money and plan fundraising projects via social networks. This paper proposes a machine learning-enabled approach to analyse Kickstarter numerical and textual data and predict the successful funding and delivery of crowdfunding projects. It offers crowdfunding stakeholders benefits including creator credibility assessment, project risk reduction, and backer confidence enhancement. This research proposes a data preprocessing approach to prepare the dataset and extract the relevant features for the predictions. Besides, it trains and compares five numerical machine learning classification models and three text-mining methods to find the best-fitted numerical and textual analysis approaches. According to the results, the proposed SVM model outperforms the numerical benchmarks in terms of Accuracy, Precision, Recall, F1 score, and model Training latency. Moreover, BERT gives the best results if the dataset is complex, while Word2vec works better with simple features in textual analysis.

Neuro-Fuzzy Logic for Automatic Animation Scene Generation in Movie Arts in Digital Media Technology

Animation scene generation (ASG) is the best digital media tool for lifelike scenes, particularly for movies. Traditional animation methods are laborious, computationally intensive, and scalable. Thus, this work addresses animation production issues using NFL-ASG. Combining fuzzy logic with a convolution neural network may create more realistic animated situations with less human interaction and better learning. Convolutional model training uses animation scenarios’ complicated motion patterns, character interactions, and ambient factors. Deep learning and fuzzy logic might change animation by boosting production techniques and releasing digital media technological creativity. After testing the system on the Moana Island scene dataset, it achieved a perception analysis success rate of 0.981% and a minimal processing complexity of (n logn).

Integrating AI and Deep Learning for Efficient Drug Discovery and Target Identification

Artificial intelligence (AI) shows great potential in medical diagnosis and drug analysis. Through deep learning technology, AI can automatically extract features from large amounts of complex medical data, significantly improving diagnostic accuracy and drug development efficiency. Especially in drug target discovery and antiviral peptide classification, AI technology can accelerate data processing and prediction, helping researchers identify potential therapeutic molecules more quickly and optimize the drug development process. This study proposes and validates a Deep learn-based model, deep-Avpiden, to improve the classification and discovery efficiency of antiviral peptides (AVPs). By using sequential convolutional networks (TCNs), the model outperforms traditional recurrent neural networks (RNNs) and long short-term memory networks (LSTMs) in capturing long-term dependencies and parallel computing capabilities. In terms of dataset, we used AVPs and non-AVPS samples from multiple databases, totaling 5,414 cleaned and de-weighted peptide sequences for model training after data preprocessing and embedding. The Deep-AVPiden model has been shown to outperform existing advanced classifiers in experiments, and its effectiveness has been verified by accuracy, precision, recall rate, and area under the ROC curve (OC-ROC). In addition, to accommodate computing resource constraints, we propose an optimized version of Deep-AvPIDen (DS), which utilizes Deep separation convolution technology to significantly reduce computing resource consumption. Through the online application platform, researchers can efficiently classify antiviral proteins and discover new AVPs. Future research could further optimize the model's computational efficiency, handle larger data sets, and expand its potential for biomedical problems such as drug combination prediction and new drug discovery.

A Deep-Learning P-Wave Arrival Picker for Laboratory Acoustic Emissions: Model Training and Its Performance

A Deep-Learning P-Wave Arrival Picker for Laboratory Acoustic Emissions: Model Training and Its Performance

Classification and prediction of heart disease using Machine Learning models: A promising approach for medical diagnosis

Background: heart disease is one of the leading causes of death worldwide, claiming 17.9 million lives. They are a major public health problem that affects people regardless of age or gender. Objective: This work aims to classify and predict heart disease using Machine Learning (ML) models such as Random Forest (RF), K-Nearest Neighbors (KNN), Support Vector Machine (SVM), Naive Bayes (NB), Decision Tree (DT) and Logistic Regression (LR). Methods: We worked with the Cleveland dataset from Kaggle, consisting of 303 patient records and 14 attributes. This research was conducted in different stages, including model understanding, dataset analysis and cleaning, ML model training, and model performance evaluation. Results: The results showed that the RF and KNN models achieved the highest levels of performance and accuracy with 88.52%, surpassing the other models such as SVM, NB, and LR which obtained 86.89% accuracy, and DT with 78.69%. Conclusions: In conclusion, the RF and KNN models stand out over the other models for this type of prediction task.

SS3DNet-AF: A Single-Stage, Single-View 3D Reconstruction Network with Attention-Based Fusion

Learning object shapes from a single image is challenging due to variations in scene content, geometric structures, and environmental factors, which create significant disparities between 2D image features and their corresponding 3D representations, hindering the effective training of deep learning models. Existing learning-based approaches can be divided into two-stage and single-stage methods, each with limitations. Two-stage methods often rely on generating intermediate proposals by searching for similar structures across the entire dataset, a process that is computationally expensive due to the large search space and high-dimensional feature-matching requirements, further limiting flexibility to predefined object categories. In contrast, single-stage methods directly reconstruct 3D shapes from images without intermediate steps, but they struggle to capture complex object geometries due to high feature loss between image features and 3D shapes and limit their ability to represent intricate details. To address these challenges, this paper introduces SS3DNet-AF, a single-stage, single-view 3D reconstruction network with an attention-based fusion (AF) mechanism to enhance focus on relevant image features, effectively capturing geometric details and generalizing across diverse object categories. The proposed method is quantitatively evaluated using the ShapeNet dataset, demonstrating its effectiveness in achieving accurate 3D reconstructions while overcoming the computational challenges associated with traditional approaches.

Improving Transferability of Physical Adversarial Attacks on Object Detectors Through Multi-Model Optimization

Physical adversarial attacks face significant challenges in achieving transferability across different object detection models, especially in real-world conditions. This is primarily due to variations in model architectures, training data, and detection strategies, which can make adversarial examples highly model-specific. This study introduces a multi-model adversarial training approach to improve the transferability of adversarial textures across diverse detection models, including one-stage, two-stage, and transformer-based architectures. Using the Truck Adversarial Camouflage Optimization (TACO) framework and a novel combination of YOLOv8n, YOLOv5m, and YOLOv3 models for optimization, our approach achieves an AP@0.5 detection score of 0.0972—over 50% lower than textures trained on single models alone. This result highlights the importance of multi-model training in enhancing attack effectiveness across object detectors, contributing to improved adversarial effectiveness.

Enhancing karate skill performance through virtual visuals and artificial intelligence techniques

In this paper, we explore the integration of virtual visuals and artificial intelligence (AI) techniques to enhance Karate skill performance, specifically focusing on circular strike skills. Traditional Karate training often lacks the diversity of sparring partners and the precise, real-time feedback necessary for optimal skill development. We propose a Mixed Interpretation Diagram (MID) as a display to present performance features, facilitating visual imagery and best practice simulations. By employing virtual sketches and intelligent sense rules, trainees can simulate optimal actions and enhance critical Karate skills. Additionally, we consider the use of Konashi intelligent bands and Reformer indoor screens to monitor muscular fatigue, providing early warnings for recovery tasks. The adoption of augmented reality (AR) and virtual reality (VR) technologies in sports training offers a new vision for enhancing human skill performance. Our v-sports system integrates state-of-the-art AI, human motion analysis, and virtual visual generation techniques, enabling personalized coaching for Karate practitioners. This paper addresses two key research problems: creating meaningful and intuitive body-movement-based presentations of self-image to optimize training, and generating efficient, professional models for guiding without a coach. We present a timeline and percentage breakdown for the development of an AI-assisted Karate training program. This program includes research on circular strike skills, virtual vision setup, AI model training, performance metrics implementation, and comprehensive system testing. The use of VR and AI in Karate training can significantly improve technique accuracy, movement efficiency, training engagement, and personalized feedback, ultimately enhancing overall performance and reducing injury risks. Our findings suggest that integrating VR and AI technologies into Karate training provides a more engaging, effective, and accessible training experience. Future research should focus on improving the accuracy and affordability of these technologies, integrating virtual and real-world training, and ensuring user safety.

An Integrated Modeling Framework for Automated Product Design, Topology Optimization, and Mechanical Simulation

The present study introduces an integrated software approach that provides an automated product design toolkit for customized products like knives, incorporating topology optimization (TO) and numerical simulations in order to streamline engineering workflows during the product development procedure. The modeling framework combines state-of-the-art technologies into a single platform, enabling the design and the optimization of mechanical structures with minimal human intervention. In particular, the proposed solution leverages artificial intelligence (AI), shape optimization methods, and computational tools in order to iteratively optimize material utilization as well as the design of products based on certain criteria. By embedding simulation within the design optimization loop, the developed software module ensures that performance constraints are respected throughout the design process. The case studies are concentrated in designing knives, demonstrating the platform’s ability to reduce design time, enhance product performance and provide rapid iterations of structurally optimized geometries. Finally, it should be noted that this research showcases the potential of integrated modeling technologies towards the transformation of traditional design paradigms, in this way contributing to faster, more reliable and efficient product development in various engineering industries through the training and deployment of AI models in these scientific fields.

DINOV2-FCS: a model for fruit leaf disease classification and severity prediction

IntroductionThe assessment of the severity of fruit disease is crucial for the optimization of fruit production. By quantifying the percentage of leaf disease, an effective approach to determining the severity of the disease is available. However, the current prediction of disease degree by machine learning methods still faces challenges, including suboptimal accuracy and limited generalizability.MethodsIn light of the growing application of large model technology across a range of fields, this study draws upon the DINOV2 visual large vision model backbone network to construct the DINOV2-Fruit Leaf Classification and Segmentation Model (DINOV2-FCS), a model designed for the classification and severity prediction of diverse fruit leaf diseases. DINOV2-FCS employs the DINOv2-B (distilled) backbone feature extraction network to enhance the extraction of features from fruit disease leaf images. In fruit leaf disease classification, for the problem that leaf spots of different diseases have great similarity, we have proposed Class-Patch Feature Fusion Module (C-PFFM), which integrates the local detailed feature information of the spots and the global feature information of the class markers. For the problem that the model ignores the fine spots in the segmentation process, we propose Explicit Feature Fusion Architecture (EFFA) and Alterable Kernel Atrous Spatial Pyramid Pooling (AKASPP), which improve the segmentation effect of the model.ResultsTo verify the accuracy and generalizability of the model, two sets of experiments were conducted. First, the labeled leaf disease dataset of five fruits was randomly divided. The trained model exhibited an accuracy of 99.67% in disease classification, an mIoU of 90.29%, and an accuracy of 95.68% in disease severity classification. In the generalizability experiment, four disease data sets were used for training and one for testing. The mIoU of the trained model reached 83.95%, and the accuracy of disease severity grading was 95.24%.DiscussionThe results demonstrate that the model exhibits superior performance compared to other state-of-the-art models and that the model has strong generalization capabilities. This study provides a new method for leaf disease classification and leaf disease severity prediction for a variety of fruits. Code is available at https://github.com/BaiChunhui2001/DINOV2-FCS.

Multi-source domain generalization tool wear prediction based on wide convolution weighted antagonism

Abstract While traditional deep learning models achieve high accuracy in predicting tool wear under consistent working conditions, actual production processes frequently involve varying conditions due to different processing methods. The wear data of different working conditions have a large difference in distribution, so that the wear signal of milling cutter trained in one working condition can only predict the wear value of the working condition, which will cause a large waste of material and manpower for actual production. Therefore, a domain generalization method utilizing wide deep convolutional neural networks and weighted antagonistic multi-source domain generalization is introduced for tool wear prediction, and multi-source domain wide depth convolutional weighted adversarial domain generalization model (WDCNNWAL) model is constructed. After filtering and de-noising the original data, it is input into WDCNN for automatic feature learning, and the resulting multi-source feature signals are input into the prediction model and discriminator respectively to obtain corresponding loss values. Furthermore, Wasserstein distance is used to measure the probability distribution distance of multi-source feature signals as a weighted value combined with the discriminator loss function, and the final loss value is obtained by adding together. Finally, the tool wear prediction model is set up using a backpropagation neural network. Various milling wear experimental data along with the NASA tool wear public dataset were utilized to assess the predictive performance of the trained model. The model achieves a high level of generalization in tool wear prediction, with average root mean squared error (RMSE) of 0.0599 and 0.2075, as well as average R 2 of 0.9037 and 0.9196 on the NASA and self-collecting datasets respectively. To validate its exceptional precision in generalization performance, comparative experiments were conducted with other data-driven methods resulting in over a 50% reduction in average RMSE and more than a 40% increase in average R 2.

Stgcn-pad: a spatial-temporal graph convolutional network for detecting abnormal pedestrian motion patterns at grade crossings

This paper presents a Spatial-Temporal Graph Convolutional Network-based Pedestrians’ behaviors Anomaly Detection system (STGCN-PAD) for grade crossings. The behaviors of pedestrians are represented in a structured manner by skeleton trajectories that are generated using a pose estimation model. The ST-GCN components are sequentially applied to capture the spatial dependencies between skeleton key points within a single video frame and the temporal relationships for each of them. Based on these features, the system reconstructs input trajectories with a constant sliding window size, and the reconstruction error is used to distinguish abnormal behaviors from those normal. To accelerate the processing of extracted multi-dimensional feature maps, an MLP-Mixer model-based reconstruction network is developed as an alternative to the traditional convolution neural network. Only trajectories of normal walking behavior are included for model training. Anomalies, such as lingering and squatting activities, can be identified as outliers by observing the magnitude of reconstruction errors. The case studies demonstrate the salient feasibility and efficiency of the proposed system, which achieves at least comparable performance (approximately 88% in the AUC evaluation metric) with several state-of-the-art approaches while using the MLP-Mixer model accelerates model inference by 10× relative to our previous effort (Song et al. in Appl Intell 53:21676–21691, 2023).

Fundus Vascular Segmentation Based on Data Enhancement and Invariant Feature Extraction

Deep neural networks have emerged as the predominant method for medical image segmentation, owing to their robust feature learning capabilities, enabling accurate automatic segmentation of target structures within intricate medical images. Fundus images, being crucial in diagnosing ophthalmic diseases, underscore the importance of effective segmentation techniques. However, fundus vascular images pose challenges due to their high complexity and subtle individual differences, necessitating improvement in existing segmentation methodologies for enhanced disease classification accuracy. This paper introduces a fundus blood vessel segmentation model, employing data augmentation and invariant feature extraction, to systematically tackle the core challenges in medical image processing, particularly in fundus blood vessel segmentation. These challenges include limited source domain samples and inadequate model domain generalization. The model adopts a dual-dimensional strategy. Firstly, it delves into data augmentation technology to enhance the diversity and representativeness of samples within the finite source domain. This is achieved through an image enhancement module based on Fourier transform, mitigating the impact of data scarcity on model training effectiveness. Secondly, the research focuses on interdomain invariant feature extraction, aiming to extract feature representations that consistently characterize fundus blood vessel structure and pathology across different data distributions, thereby enhancing model generalization performance in unfamiliar domains. Specifically, the paper designs a fundus image enhancement module based on Fourier transform in the data augmentation dimension. In the feature extraction dimension, it proposes a normalization module based on uncertainty theory, departing from traditional normalization methods. Experimental results demonstrate the efficacy of the proposed method, showcasing superior generalization performance compared to existing techniques in retinal blood vessel and OD/OC segmentation tasks. Experience validates the model-agnostic nature of the learned strategy, indicating its potential for seamless transferability to other models, thereby offering robust support for advancing medical image segmentation research and applications.

SAITI-DCGAN: Self-Attention Based Deep Convolutional Generative Adversarial Networks for Data Augmentation of Infrared Thermal Images

Defect detection plays a crucial role in industrial production, and the implementation of this technology has significant implications for improving both product quality and processing efficiency. However, the limited availability of defect samples for training deep-learning-based object detection models within industrial processes poses challenges for model training. In this paper, we propose a novel deep convolutional generative adversarial network with self-attention mechanism for the data augmentation of infrared thermal images for the application of aluminum foil sealing. To further expand its applicability, the proposed method is designed not only to address the specific needs of aluminum foil sealing but also to serve as a robust framework that can be adapted to a wide range of industrial defect detection tasks. To be specific, the proposed approach integrates a self-attention module into the generator, adopts spectral normalization in both the generator and discriminator, and introduces a two time-scale update rule to coordinate the training process of these components. The experimental results validated the superiority of the proposed approach in terms of the synthesized image quality and diversity. The results show that our approach can capture intricate details and distinctive features of defect images of aluminum foil sealing. Furthermore, ablation experiments demonstrated that the combination of self-attention, spectral normalization, and two time-scale update rules significantly enhanced the quality of image generation, while achieving a balance between stability and training efficiency. This innovative framework marks a notable technical breakthrough in the field of industrial defect detection and image synthesis, offering broad application prospects.

Retrieval of Three-Dimensional Wave Surfaces from X-Band Marine Radar Images Utilizing Enhanced Pix2Pix Model

In this study, we propose a novel method for retrieving the three-dimensional (3D) wave surface from sea clutter using both simulated and measured data. First, the linear wave superposition model and modulation principle are employed to generate simulated datasets comprising 3D wave surfaces and corresponding sea clutter. Subsequently, we develop a Pix2Pix model enhanced with a self-attention mechanism and a multiscale discriminator to effectively capture the nonlinear relationship between the simulated 3D wave surfaces and sea clutter. The model’s performance is evaluated through error analysis, comparisons of wave number spectra, and differences in wave surface reconstructions using a dedicated test set. Finally, the trained model is applied to reconstruct wave surfaces from sea clutter data collected aboard a ship, with results benchmarked against those derived from the Schrödinger equation. The findings demonstrate that the proposed model excels in preserving high-frequency image details while ensuring precise alignment between reconstructed images. Furthermore, it achieves superior retrieval accuracy compared to traditional approaches, highlighting its potential for advancing wave surface retrieval techniques.

Federated learning merges with v2x privacy protection

Vehicle-to-Everything (V2X) communication systems are poised to revolutionize road safety, traffic efficiency, and driver experience through real-time data exchange among vehicles and infrastructure. However, ensuring the privacy and security of sensitive vehicular data remains a critical challenge. Federated Learning (FL) emerges as a promising paradigm to address these concerns by enabling collaborative model training without centralized data aggregation. This paper explores the integration of FL techniques into V2X environments, where vehicles act as decentralized participants in model training processes. We discuss the unique challenges and opportunities presented by V2X scenarios, such as intermittent connectivity and diverse data distributions across vehicles. Through a detailed review of existing FL methodologies and their adaptation to V2X settings, this paper proposes practical solutions for privacy-preserving model aggregation and efficient learning across dynamic vehicular networks. Case studies and simulation results illustrate the feasibility and benefits of FL in enhancing V2X applications while safeguarding user privacy. This study contributes to the growing field of secure and collaborative machine learning in connected vehicle environments, paving the way for safer and smarter transportation systems.