Reduce Training Time Research Articles

A Method for Reducing Training Time of ML-Based Cascade Scheme for Large-Volume Data Analysis.

We live in the era of large data analysis, where processing vast datasets has become essential for uncovering valuable insights across various domains of our lives. Machine learning (ML) algorithms offer powerful tools for processing and analyzing this abundance of information. However, the considerable time and computational resources needed for training ML models pose significant challenges, especially within cascade schemes, due to the iterative nature of training algorithms, the complexity of feature extraction and transformation processes, and the large sizes of the datasets involved. This paper proposes a modification to the existing ML-based cascade scheme for analyzing large biomedical datasets by incorporating principal component analysis (PCA) at each level of the cascade. We selected the number of principal components to replace the initial inputs so that it ensured 95% variance retention. Furthermore, we enhanced the training and application algorithms and demonstrated the effectiveness of the modified cascade scheme through comparative analysis, which showcased a significant reduction in training time while improving the generalization properties of the method and the accuracy of the large data analysis. The improved enhanced generalization properties of the scheme stemmed from the reduction in nonsignificant independent attributes in the dataset, which further enhanced its performance in intelligent large data analysis.

A Recurrent Stochastic Configuration Network for Dynamic System Modeling and Its Application

In order to model nonlinear dynamic systems quickly and accurately, a recurrent stochastic configuration network (RSCN) with universal approximation ability is proposed in this paper. The method consists of network parameter learning and network structure determination. The network parameter learning strategy consists of input weights and the biases of hidden nodes generated randomly through a supervision mechanism, while the output weights of hidden nodes are determined by solving the least-squares problem. The network structure is determined incrementally by the training error of a single-layer recurrent neural network. The effectiveness of RSCN is tested and evaluated by using three nonlinear dynamic functions and historical data from a municipal solid waste incineration (MSWI) process. The experimental results show that the proposed method can improve modeling accuracy and reduce training time, which is valuable for applications in the field of industrial process modeling.

RoBERTa, ResNeXt and BiLSTM with self-attention: The ultimate trio for customer sentiment analysis

Sentiment analysis of customer feedback is a pivotal component of Natural Language Processing (NLP), enabling businesses to gauge consumer emotions towards their products and services. The inherent variability of language introduces substantial challenges in the analysis of unstructured customer data. Sentiment analysis techniques generally fall into two categories: traditional methods, which involve the extraction of hand-crafted features and the application of machine learning algorithms, and end-to-end deep learning approaches that process raw data, transforming them layer by layer into high-level textual features. Each method presents its own balance between simplicity and speed versus analytical power and flexibility, and they all grapple with the need for extensive labeled data and computational resources. To address these challenges, this paper presents an innovative hybrid model that amalgamates RoBERTa, ResNeXt, BiLSTM, and self-attention mechanisms. This integration capitalizes on the collective strengths of these models to surmount the limitations inherent in each. The proposed model proves to be a powerful and efficient tool for sentiment analysis, demonstrating proficiency across various data types and tasks. We undertake a comprehensive evaluation of the proposed model’s accuracy and training efficiency using four benchmark datasets. Our investigation also explores the impact of different similarity measures and convolutional neural network architectures on the model’s performance. The results affirm that our model not only achieves high accuracy but also significantly reduces training time compared to RoBERTa’s fine-tuning. Furthermore, the model exhibits exceptional domain adaptability, particularly when fine-tuned on the IMDb dataset following initial training on the Yelp Review Full dataset. The practicality of the proposed model is underscored by its reduced computational complexity and its adeptness at navigating semantic and syntactic nuances.

Opportunities in Pulmonary and Critical Care Medicine for Dual-trained Medicine and Pediatric Physicians.

Combined internal medicine-pediatrics (med-peds) training has been recognized as a unique, Accreditation Council for Graduate Medical Education-accredited residency pathway since 1967, resulting in more than 10,000 graduates. Med-peds graduates have the option of pursuing combined med-peds fellowships with a 1-year reduction in training time compared with pursuing such fellowships separately. The typical med-peds resident spends 8-9 months in the intensive care unit during residency, with additional rotations in pulmonary medicine; not surprisingly, residents are increasingly inquiring about combined med-peds fellowship training within the fields of pulmonary and critical care medicine. In this review, we outline both the advantages and challenges of such training and present various pathways and considerations to achieve board certification.

Using virtual reality for occupational safety and health in a UK tunnelling megaproject

Recent advancements into key technological developments (KTDs) are leading to a paradigm shift in the way that occupational safety and health (OSH) is managed in the architecture, engineering, construction and operation sector. This paper reports on a pilot case study where building information modelling and virtual reality were used for OSH training purposes related to the operation of a tunnel-boring machine. The aim of the research was to identify the benefits, barriers and challenges associated with the implementation of these KTDs in a joint venture organisation delivering part of a complex megaproject in Central London, UK, the Thames Tideway Tunnel project (thereafter, ‘Tideway’). The reported benefits included better safety in design and improved hazard perception. Tideway estimates that the project could potentially see a 20% reduction in training time and a 10% reduction in accidents. The key lessons learned include the need to keep costs down, the necessity for system portability and ease of access. The results of the study will assist organisations that are interested in the adoption of these KTDs and provide valuable insights to the wider construction industry. This will in turn enable companies to improve how OSH is managed and promote the uptake of KTDs for OSH.

Parallel ensemble of a randomization-based online sequential neural network for classification problems using a frequency criterion

Randomization-based neural networks have gained wide acceptance in the scientific community owing to the simplicity of their algorithm and generalization capabilities. Random vector functional link (RVFL) networks and their variants are a class of randomization-based neural networks. RVFL networks have shown promising results in classification, regression, and clustering problems. For real-world applications, learning algorithms that can train with new samples over previous results are necessary because of to the constant generation of problems related to large-scale datasets. Various online sequential algorithms, commonly involving an initial learning phase followed by a sequential learning phase, have been proposed to address this issue. This paper presents a training algorithm based on multiple online sequential random vector functional link (OS-RVFL) networks for large-scale databases using a shared memory architecture. The training dataset is distributed among p OS-RVFL networks, which are trained in parallel using p threads. Subsequently, the test dataset samples are classified using each trained OS-RVFL network. Finally, a frequency criterion is applied to the results obtained from each OS-RVFL network to determine the final classification. Additionally, an equation was derived to reasonably predict the total training time of the proposed algorithm based on the learning time in the initial phase and the time scaling factor compared to the sequential learning phase. The results demonstrate a drastic reduction in training time because of data distribution and an improvement in accuracy because of the adoption of the frequency criterion.

C-KGE: Curriculum learning-based Knowledge Graph Embedding

Knowledge graph embedding (KGE) aims to embed entities and relations in knowledge graphs (KGs) into a continuous, low-dimensional vector space. It has been shown as an effective tool for integrating knowledge graphs to improve various intelligent applications, such as question answering and information extraction. However, previous KGE models ignore the hidden natural order of knowledge learning on learning the embeddings of entities and relations, leaving room for improvement in their performance. Inspired by the easy-to-hard pattern used in human knowledge learning, this paper proposes a Curriculum learning-based KGE (C-KGE) model, which learns the embeddings of entities and relations from “basic knowledge” to “domain knowledge”. Specifically, a seed set representing the basic knowledge and several knowledge subsets are identified from KG. Then, entity overlap is employed to score the learning difficulty of each subset. Finally, C-KGE trains the entities and relations in each subset according to the learning difficulty score of each subset. C-KGE leverages trained embeddings of the seed set as prior knowledge and learns knowledge subsets iteratively to transfer knowledge between the seed set and subsets, smoothing the learning process of knowledge facts. Experimental results on real-world datasets demonstrate that the proposed model achieves improved embedding performances as well as reducing training time. Our codes and data will be released later.

Genetic Cache: A Machine Learning Approach to Designing DRAM Cache Controllers in HBM Systems

DRAM memory controller plays a critical role in maximizing the performance of High Bandwidth Memory by efficiently managing data transfers between the CPU and the memory modules. Thus, they are suitable for low-power data-intensive applications. However, the complexity of DRAM command scheduling combined with tag management overheads in the cache makes the design of in-package cache controllers significantly challenging. Traditional memory controllers often face challenges with static, inflexible access scheduling designed for general applications, leading to suboptimal performance in dynamic environments. On the other hand, while some advanced controllers employing reinforcement learning offer adaptability to workload fluctuations, they tend to introduce hardware complexity and incur longer training latencies. Our approach aims to design low-power and efficient DRAM cache controllers using a machine learning model that dynamically adjusts to workload changes with optimized hardware efficiency and reduced training time. Therefore, we propose a machine learning approach to design low-power and efficient DRAM cache controllers that will leverage the trained model to produce optimal cache command schedules at runtime. The model considers several conditions for each request queue, and chooses the best response among the options. The simulation results show the superiority of our proposed design over the previous algorithms in a set of twelve data-intensive applications; our model is able to improve the performance up to 40% in some cases, and an average of 15% in performance and 10% in power consumption.

A contextual-based approach for sarcasm detection

Sarcasm is a perplexing form of human expression that presents distinct challenges in understanding. The problem of sarcasm detection has centered around analyzing individual utterances in isolation which may not provide a comprehensive understanding of the speaker’s sarcastic intent. Our work addresses this problem by exploring and understanding the specific contextual cues that contribute to sarcasm. In this paper, we propose an enhanced approach for sarcasm detection using contextual features. Our methodology involves employing pre-trained transformer models, RoBERTa and DistilBERT, and fine-tuning them on two datasets: the News Headlines and the Mustard datasets. Incorporating contextual information, the proposed approach yielded the best performance, achieving an impressive F1 score of 99% on News Headlines dataset and 90% on Mustard dataset. Moreover, we experimented summarizing the context into a concise short sentence. This enhancement reduced training time by 35.5% of the original time. We further validated the model trained on the News headlines dataset against the Reddit dataset, which resulted in 49% F1 score without context data. However, with the inclusion of context data, the F1 score surged to 75%. Proposed approach enhances the understanding of sarcasm in different contextual settings, enabling more accurate sentiment analysis and better decision-making in various applications.

LMBiS-Net: A lightweight bidirectional skip connection based multipath CNN for retinal blood vessel segmentation

Blinding eye diseases are often related to changes in retinal structure, which can be detected by analysing retinal blood vessels in fundus images. However, existing techniques struggle to accurately segment these delicate vessels. Although deep learning has shown promise in medical image segmentation, its reliance on specific operations can limit its ability to capture crucial details such as the edges of the vessel. This paper introduces LMBiS-Net, a lightweight convolutional neural network designed for the segmentation of retinal vessels. LMBiS-Net achieves exceptional performance with a remarkably low number of learnable parameters (only 0.172 million). The network used multipath feature extraction blocks and incorporates bidirectional skip connections for the information flow between the encoder and decoder. In addition, we have optimised the efficiency of the model by carefully selecting the number of filters to avoid filter overlap. This optimisation significantly reduces training time and improves computational efficiency. To assess LMBiS-Net’s robustness and ability to generalise to unseen data, we conducted comprehensive evaluations on four publicly available datasets: DRIVE, STARE, CHASE_DB1, and HRF The proposed LMBiS-Net achieves significant performance metrics in various datasets. It obtains sensitivity values of 83.60%, 84.37%, 86.05%, and 83.48%, specificity values of 98.83%, 98.77%, 98.96%, and 98.77%, accuracy (acc) scores of 97.08%, 97.69%, 97.75%, and 96.90%, and AUC values of 98.80%, 98.82%, 98.71%, and 88.77% on the DRIVE, STARE, CHEASE_DB, and HRF datasets, respectively. In addition, it records F1 scores of 83.43%, 84.44%, 83.54%, and 78.73% on the same datasets. Our evaluations demonstrate that LMBiS-Net achieves high segmentation accuracy (acc) while exhibiting both robustness and generalisability across various retinal image datasets. This combination of qualities makes LMBiS-Net a promising tool for various clinical applications.

Wheat Leaf Disease Detection: A Lightweight Approach with Shallow CNN Based Feature Refinement

Improving agricultural productivity is essential due to rapid population growth, making early detection of crop diseases crucial. Although deep learning shows promise in smart agriculture, practical applications for identifying wheat diseases in complex backgrounds are limited. In this paper, we propose CropNet, a hybrid method that utilizes Red, Green, and Blue (RGB) imaging and a transfer learning approach combined with shallow convolutional neural networks (CNN) for further feature refinement. To develop our customized model, we conducted an extensive search for the optimal deep learning architecture. Our approach involves freezing the pre-trained model for feature extraction and adding a custom trainable CNN layer. Unlike traditional transfer learning, which typically uses trainable dense layers, our method integrates a trainable CNN, deepening the architecture. We argue that pre-trained features in transfer learning are better suited for a custom shallow CNN followed by a fully connected layer, rather than being fed directly into fully connected layers. We tested various architectures for pre-trained models including EfficientNetB0 and B2, DenseNet, ResNet50, MobileNetV2, MobileNetV3-Small, and Inceptionv3. Our approach combines the strengths of pre-trained models with the flexibility of custom architecture design, offering efficiency, effective feature extraction, customization options, reduced overfitting, and differential learning rates. It distinguishes itself from classical transfer learning techniques, which typically fine-tune the entire pre-trained network. Our aim is to provide a lightweight model suitable for resource-constrained environments, capable of delivering outstanding results. CropNet achieved 99.80% accuracy in wheat disease detection with reduced training time and computational cost. This efficient performance makes CropNet promising for practical implementation in resource-constrained agricultural settings, benefiting farmers and enhancing production.

Ensuring Food Safety among Food and Agricultural Nonprofit Organizations: A Review of Literature about Challenges and Opportunities

Food and agricultural nonprofit organizations (FANOs) are a critical part of the food supply chain, requiring evaluation of their food safety assurance within the production, preparation, and distribution of food. This article provides an overview of literature and policy surrounding food safety in nonprofit organizations, particularly FANOs, and proposes opportunities and challenges in FANO research. It addresses the prevalent risks of foodborne illnesses associated with FANOs with poor food safety practices. Three specific problem areas that need to be better addressed in FANO research were revealed. First, poor food safety practices in FANOs can lead to food loss and waste. Second, inadequate food safety behaviors and practices among volunteers are a unique issue that FANOs face due to high volunteer turnover and reduced training time. And third, there is a large disparity between who is regulating and enforcing food safety in FANOs. Potential research questions and methodological limitations of conducting research on each problem area are proposed.

Suppression of negative transfer in motor imagery brain-computer interface based on mutual information and Pearson correlation coefficient.

The focus of this paper is on the main challenges in brain-computer interface transfer learning: how to address data characteristic length and the source domain sample selection problems caused by individual differences. To overcome the negative migration that results from feature length, we propose a migration algorithm based on mutual information transfer (MIT), which selects effective features by calculating the entropy value of the probability distribution and conditional distribution, thereby reducing negative migration and improving learning efficiency. Source domain participants who differ too much from the target domain distribution can affect the overall classification performance. On the basis of MIT, we propose the Pearson correlation coefficient source domain automatic selection algorithm (PDAS algorithm). The PDAS algorithm can automatically select the appropriate source domain participants according to the target domain distribution, which reduces the negative migration of participant data among the source domain participants, improves experimental accuracy, and greatly reduces training time. The two proposed algorithms were tested offline and online on two public datasets, and the results were compared with those from existing advanced algorithms. The experimental results showed that the MIT algorithm and the MIT + PDAS algorithm had obvious advantages.

Addressing imbalance in graph datasets: Introducing GATE-GNN with graph ensemble weight attention and transfer learning for enhanced node classification

Significant challenges arise when Graph Neural Networks (GNNs) try to deal with uneven data. Specifically in signed and weighted graph structures. This makes classification tasks less effective. Within the GNN context, researchers have found traditional solutions like resampling, reweighting, and synthetic sample generation to be inadequate. GATE-GNN, a novel architecture designed specifically for imbalanced datasets, overcomes these limitations. GATE-GNN integrates an ensemble of network modules that harness the spatial features of graph networks and effectively utilise embedding information from earlier layers. This unique approach not only bolsters generalisation by reducing volatility. It also refines the optimisation algorithm, resulting in more accurate and stable classification outcomes. We rigorously tested the effectiveness of GATE-GNN on four widely recognised datasets: Cora, NELL, Citeseer, and PubMed. We performed a comparative analysis against established methods such as Graph Convolutional Networks (GCN), Graph Sample and Aggregate (GraphSAGE), Propagation Multilayer Perceptron PMLP), Imbalanced Node Sampling GNN (INS-GNN), GNN-Curriculum Learning (GNN-CL) and Graph Attention Networks (GAT). Empirical results demonstrate that GATE-GNN significantly outperforms these existing models, achieving an average improvement in classification accuracy of approximately 5%–10% over the previous best results. Additionally, GATE-GNN presents a marked reduction in training time. This underscores its efficiency and suitability for practical applications in imbalanced graph data scenarios. Implementation of the proposed GATE-GNN can be accessed here https://github.com/afofanah/GATE-GNN.

A lightweight complex-domain acoustic feature extraction method for rotating machinery fault detection

Abstract In the field of fault diagnosis for factory machinery systems, the development of deep learning methods has been hindered by the challenge of acquiring fault data, highlighting the need to extract noise robust features from limited labeled data. In this paper, a light and efficient complex-domain acoustic feature extraction method (CPFCN) is proposed for fault diagnosis in rotating machinery, which consists of a principal frequency filter (PFF) and stacked convolution network (SCN). The PFF filters out non-principal frequency noise to focus on the predominant frequency. The SCN is designed to effectively extract the amplitude and phase features, which can fully leverage the complex-domain information within the acoustic data. The experimental results show that the proposed CPFCN have 33% increasing in accuracy while 87% reduction in training time and 41% reduction in feature extraction time. Additionally, the proposed framework has improved the accuracy by 59% on the dataset with noise compared to the best-performing method in the experimental study, achieving stronger noise robustness in the case of limited samples.

Image Analysis Method of Substation Equipment Status Based on Cross‐Modal Learning

In response to increasing power supply needs, maintaining stable substations is vital for reliable electricity. Traditional manual equipment inspections in these substations are inefficient and risky, often leading to hazards and delayed detection of faults. Therefore, there's a growing shift towards using intelligent image recognition technology in video surveillance systems for safer and more efficient inspections. This paper focuses on enhancing the level of intelligent inspection in substations using artificial intelligence‐based visual recognition technology. It introduces a novel small‐sample classification algorithm based on the CLIP architecture. This method uses cross‐modal equipment status information as additional training samples, optimizing the loss function together with image samples, and devises hand‐crafted strategies for text sample inputs to distinguish between equipment and states. The experimental results show that with only 16 training samples per category for 21 types of electrical equipment states, our method achieved a maximum accuracy of 93.38%. This represents a 2.98% higher accuracy than the PPLCNet trained on the full dataset and an 8.63% higher accuracy than the PPLCNet trained with an equal number of samples, with significantly reduced training time. © 2024 Institute of Electrical Engineers of Japan and Wiley Periodicals LLC.

Swarm optimized differential evolution and probabilistic extreme learning based intrusion detection in MANET

MANETs are an attracting mechanism foSr several applications, to name a few being rescue functioning, environmental surveillance and so on due to the reason that they allow users to communicate without the utilization of persistent framework. This pliability, although creates additional security proneness. As a consequence of its advantages and growing insistence, MANETs have fascinated a lot of attentiveness from the scientific research community. In spite of that, exposed or unprotected to be more susceptible to several attacks that perpetrate destruction on their performance than any network. Conventional cryptography mechanisms cannot completely safeguard MANETs in terms of susceptibility owing to dispersed nature however these issues can be addressed by using optimization and deep learning techniques-based Intrusion Detection System (IDS). In this work, we develop a Binary Swarm Optimized Differential and Method of Moments Probabilistic Extreme Learning (BSOD-MMPEL) node behavior-based IDS for intrusion detection in MANET with minimum training time, misclassification and high precision, accuracy is proposed. Initially, network samples are collected from KDD-CUP-1999 dataset. To ensure robust and significant balance between exploration and exploitation with high probability of convergence to local sub-optima Gudermannian Activation Binary Swarm Optimized Differential Evolution-based Feature Selection is applied. Next, with the selected features, early detection of intrusion employing Method of Moments Probabilistic Extreme Learning Node Behavior-based IDS is designed. Simulations are performed to validate the result. The performance evaluation in terms of precision, accuracy, sensitivity, training time and misclassification rate show that the proposed method outperforms existing IDS in MANET. The proposed BSOD-MMPEL method is achieved to improve precision of 23 % and accuracy of 27 % and reduce training time of 62 % and misclassification rate of 68 % than existing conventional methods.

HSFE: A hierarchical spatial-temporal feature enhanced framework for traffic flow forecasting

Currently, spatio-temporal fusion strategy is a key direction in traffic flow prediction. Current work employs a higher degree of spatio-temporal self-attention in order to capture spatio-temporal dependencies. However, the features to which this approach is applied are more targeted, increasing the computational complexity of the process and making it more difficult to capture long-range dependencies. This paper proposes a new framework for predicting traffic flow that enhances spatio-temporal features at multiple levels. The framework includes a periodic embedding module that captures temporal periodicity and encodes input data into more representative feature vectors for model training. Also, a component for fusing parallel channel attention has been designed to adaptively weigh the aggregation of features from global, local, and aggregated channels. This enhances the attention given to important feature information in the model. In addition, a multilevel sequential feature fusion enhancer has been designed that ensures feature processing at different levels. Experimental results on four public transportation datasets demonstrate that the innovative approach enhances the MAE metrics by an average of 2.50%, respectively, over all metrics in the baseline models. Notably, it reduces training time by approximately 50%. This paper also discusses ablation experiments to evaluate the performance of each module.

3D reconstruction of building structures incorporating neural radiation fields and geometric constraints

Neural Radiance Fields (NeRF) techniques demonstrate potential for reconstructing complex architectural scenes in three dimensions. However, applying NeRF poses challenges, such as prolonged training processes and limited detailed characterization. To address these issues, this paper introduces NeRFusion by integrating Mipmapped Neural Radiance Fields (Mip-NeRF) and Instant Neural Graphics Primitives (iNGP). NeRFusion enhances training efficiency, rendering quality, and reduces jaggedness through multisampling, weight reduction, and loss function optimization. Geometric constraints—normal consistency, plane fitting, and vertical/horizontal constraints—improve 3D reconstruction accuracy. Using the Manhattan World Model (MWM), NeRFusion accurately extracts primary building components and dimensions. Experimental results show NeRFusion achieved a Peak Signal-to-Noise Ratio (PSNR) of 36.23 dB and completed training in 50 min. Structural Similarity Index Measure (SSIM) improved by 13.41% over Mip-NeRF and 50.00% over iNGP, with PSNR increasing by 10.45% and 25.10%. NeRFusion significantly reduces training time by a factor of 15 compared to the original NeRF.

Driver behaviour recognition based on recursive all‐pair field transform time series model

AbstractTo standardize driver behaviour and enhance transportation system safety, a dynamic driver behaviour recognition method based on the Recurrent All‐Pairs Field Transforms (RAFT) temporal model is proposed. This study involves the creation of two datasets, namely, Driver‐img and Driver‐vid, including driver behaviour images and videos across various scenarios. These datasets are subject to preprocessing using RAFT optical flow techniques to enhance the cognitive process of the network. This approach employs a two‐stage temporal model for driver behaviour recognition. In the initial stage, the MobileNet network is optimized and the GYY module is introduced, which includes residuals and global average pooling layers, thereby enhancing the network's feature extraction capabilities. In the subsequent stage, a bidirectional GRU network is constructed to learn driver behaviour video features with temporal information. Additionally, a method for compressing and padding video frames is proposed, which serves as input to the GRU network and enables intent prediction 0.2 s prior to driver actions. Model performance is assessed through accuracy, recall, and F1 score, with experimental results indicating that RAFT preprocessing enhances accuracy, reduces training time, and improves overall model stability, facilitating the recognition of driver behaviour intent.