Reduce Training Time Research Articles

Driver behaviour recognition based on recursive all‐pair field transform time series model

AbstractTo standardize driver behaviour and enhance transportation system safety, a dynamic driver behaviour recognition method based on the Recurrent All‐Pairs Field Transforms (RAFT) temporal model is proposed. This study involves the creation of two datasets, namely, Driver‐img and Driver‐vid, including driver behaviour images and videos across various scenarios. These datasets are subject to preprocessing using RAFT optical flow techniques to enhance the cognitive process of the network. This approach employs a two‐stage temporal model for driver behaviour recognition. In the initial stage, the MobileNet network is optimized and the GYY module is introduced, which includes residuals and global average pooling layers, thereby enhancing the network's feature extraction capabilities. In the subsequent stage, a bidirectional GRU network is constructed to learn driver behaviour video features with temporal information. Additionally, a method for compressing and padding video frames is proposed, which serves as input to the GRU network and enables intent prediction 0.2 s prior to driver actions. Model performance is assessed through accuracy, recall, and F1 score, with experimental results indicating that RAFT preprocessing enhances accuracy, reduces training time, and improves overall model stability, facilitating the recognition of driver behaviour intent.

Computer Simulators - Modern Tools For Training Future Officers And Sergets

The purpose of this article was to consider the prerequisites for the emergence, stages and prospects for the development of military computer games in the Republic of Kazakhstan, used in combat training of the armed forces of a number of states, as well as an attempt to conduct a comparative assessment of various military simulators using developed military computer programs. Simulators of military weapons at the present stage play an important role in preparing future reserve officers and sergeants in the Republic of Kazakhstan for real combat operations. They are computer programs or physical devices that help simulate various aspects of military combat activities. One of the main advantages of using computer simulations is the ability to create realistic combat conditions without having to risk lives and resources. Future officers and NCOs can enhance their professional skills through simulations to develop strategic and tactical skills, make real-time decisions and coordinate the actions of their units. In our opinion, the use of military weapons simulators provides an opportunity to gain excellent skills in working with military weapons, reduces training time for beginners, improves reactions and quick decision-making, and of course reduces the risk of injury during training. So, simulators of weapons and military equipment will contribute to the training of future reserve officers and sergeants, helping to develop their skills, leadership qualities and make effective decisions in combat situations. A modern approach effectively complements traditional military training.

Black Hawk 70i Helicopter Crew Simula- tion Training

Over the past several decades, fl ight simulators have played an important role in pilot training for both civilian and military aircraft. There are fi nancial considerationsbehind the production or purchase of fl ight simulators. Although the initial fi nancial contribution is high, in the long run the use of simulation systems is more cost-effective than real-world pilot training. Furthermore, the service’s limited fi nancial resources may cause it to halt or limit the amount of real-world training it receives. The cost of aviation fuel causes drastic reductions in training time for helicopter crews (police, military or civilian). Consequently, alternative methods of training personnel using aircraft have become the focus of the search for new solutions. One example being built is the Helicopter Simulator for Police Aviation. The implementation of the Police Aviation Helicopter Simulator into the police training system will enable the proper preparation of Police Aviation crews and counterterrorism units of the Police to perform their tasks profi ciently and effectively. The information presented in thisarticle is the result of analysis of the source data obtained from the design documentation of the “Helicopter Simulator for Police Aviation” No. DOB-BIO10/07/01/2019. One undeniable research limitation was the inability to provide in this study a detailed description of the simulator due to reservations related to protection of Intellectual Property Rights, which were included in the contracts for the execution and fi nancing of the project.

Distriformer: Research on a Distributed Training Rockburst Prediction Method

The precise forecasting of rockburst is fundamental for safeguarding human lives and property, upholding national energy security, and protecting social welfare. Traditional methods for predicting rockburst suffer from poor accuracy and extended model training durations. This paper proposes a distributed training rockburst prediction method called Distriformer, which uses deep learning technology combined with distributed training methods to predict rockburst. To assess the efficacy of the Distriformer rockburst model proposed herein, five datasets were used to compare the proposed method with Transformer and Informer. The experimental results indicate that, compared with Transformer, the proposed method reduces the mean absolute error by 44.4% and the root mean square error by 30.7% on average. In terms of training time, the proposed method achieves an average accelaration ratio of 1.72. The Distriformer rockburst model enhances the accuracy of rockburst prediction, reduces training time, and serves as a reference for development of subsequent real-time prediction models for extensive rockburst data.

Revolutionizing urban mapping: deep learning and data fusion strategies for accurate building footprint segmentation

In the dynamic urban landscape, understanding the distribution of buildings is paramount. Extracting and delineating building footprints from high-resolution images, captured by aerial platforms or satellites, is essential but challenging to accomplish manually, due to the abundance of high-resolution data. Automation becomes imperative, yet it introduces complexities related to handling diverse data sources and the computational demands of advanced algorithms. The innovative solution proposed in this paper addresses some intricate challenges occurring when integrating deep learning and data fusion on Earth Observed imagery. By merging RGB orthophotos with Digital Surface Models, deriving from the same aerial high-resolution surveys, an integrated consistent four-band dataset is generated. This unified approach, focused on the extraction of height information through stereoscopy utilizing a singular source, facilitates enhanced pixel-to-pixel data fusion. Employing DeepLabv3 algorithms, a state-of-the-art semantic segmentation network for multi-scale context, pixel-based segmentation on the integrated dataset was performed, excelling in capturing intricate details, particularly when enhanced by the additional height information deriving from the Digital Surface Models acquired over urban landscapes. Evaluation over a 21 km2 area in Turin, Italy, featuring diverse building frameworks, showcases how the proposed approach leads towards superior accuracy levels and building boundary refinement. Notably, the methodology discussed in the present article, significantly reduces training time compared to conventional approaches like U-Net, overcoming inherent challenges in high-resolution data automation. By establishing the effectiveness of leveraging DeepLabv3 algorithms on an integrated dataset for precise building footprint segmentation, the present contribution holds promise for applications in 3D modelling, Change detection and urban planning. An approach favouring the application of deep learning strategies on integrated high-resolution datasets can then guide decision-making processes facilitating urban management tasks.

Classification of Grapevine Varieties Using UAV Hyperspectral Imaging

Classifying grapevine varieties is crucial in precision viticulture, as it allows for accurate estimation of vineyard row growth for different varieties and ensures authenticity in the wine industry. This task can be performed with time-consuming destructive methods, including data collection and analysis in the laboratory. In contrast, unmanned aerial vehicles (UAVs) offer a markedly more efficient and less restrictive method for gathering hyperspectral data, even though they may yield data with higher levels of noise. Therefore, the first task is the processing of these data to correct and downsample large amounts of data. In addition, the hyperspectral signatures of grape varieties are very similar. In this study, we propose the use of a convolutional neural network (CNN) to classify seventeen different varieties of red and white grape cultivars. Instead of classifying individual samples, our approach involves processing samples alongside their surrounding neighborhood for enhanced accuracy. The extraction of spatial and spectral features is addressed with (1) a spatial attention layer and (2) inception blocks. The pipeline goes from data preparation to dataset elaboration, finishing with the training phase. The fitted model is evaluated in terms of response time, accuracy and data separability and is compared with other state-of-the-art CNNs for classifying hyperspectral data. Our network was proven to be much more lightweight by using a limited number of input bands (40) and a reduced number of trainable weights (560 k parameters). Hence, it reduced training time (1 h on average) over the collected hyperspectral dataset. In contrast, other state-of-the-art research requires large networks with several million parameters that require hours to be trained. Despite this, the evaluated metrics showed much better results for our network (approximately 99% overall accuracy), in comparison with previous works barely achieving 81% OA over UAV imagery. This notable OA was similarly observed over satellite data. These results demonstrate the efficiency and robustness of our proposed method across different hyperspectral data sources.

Musculoskeletal injury and medical discharge prevalence during Army basic training

Musculoskeletal injuries (MSKIs) pose the primary health and operational risk to the military trainee population particularly those within basic training, leading to several issues such as reduced training time, chronic morbidity, and inevitably increased medical discharge. The purpose of the study was to investigate the current incidence, prevalence, and nature of MSKIs across British Army basic training. In addition to identifying the prevalence of MSKI medical discharges concerning the type and nature of the injury. A retrospective analysis of the Army Recruiting and Initial Training Command MSKI and Medical Discharge database was undertaken. Injury data was collected from Primary Care Rehabilitation Facilities at British Army training units, using a standardised procedure. A multiple-choice questionnaire was used within the initial consultation with the clinician to determine the severity and progression of the injury, whilst consenting to a data privacy notice. Prevalence of MSKIs (% recruits) was articulated in relation to the total number of recruits starting each training year. Over the past decade MSKIs have declined from a peak of 47% in men and 58% in women in 2013/14, to 22% in men and 21% in women in the 2022/23 training year, with the prevalence of female MSKIs being lower than male counterparts for the first time in this population. Trauma and overuse injuries correspond for 48% and 52% of these MSKIs respectively in 2022/2023, whereas historically overuse injuries have been responsible for a greater proportion. These overuse and trauma injuries were most common in the early stages of training. Lower limb injuries were more prevalent throughout training, resembling 2021/22 data where lower limb injuries accounted for c.80% of total MSKIs among both sexes. The total number of MSKIs resulting in medical discharges was 70, constituting 1.1% of all recruits. Among these discharges, 54 were observed in men and 16 in women. The prevalence of MSKIs resulting in medical discharges was 4.5% in men and 12.4 % in women. The gap between MSKI prevalence between sexes has decreased with recent data showing prevalence being similar. Despite this, women had an increased risk of medical discharge, suggesting gender differences in the severity of MSKIs. To reduce MSKI prevalence, strategies should focus on lower limb injuries, with an emphasis on the early stages of training.

Stream label distribution learning processing via broad learning system

Label distribution learning (LDL) is designed for label ambiguity problem which widely exists in tasks like image classification and sentiment analysis. Nevertheless, most existing LDL methods adopt a batch-wise processing strategy that is not applicable to stream data issues, like video semantic segmentation and object tracking in auto-driving applications. For these scenarios where dynamic new data is generated continuously, stream LDL processing can be applied to reflect the changes in time and improve the accuracy of online prediction. This paper proposes a novel broad learning system (BLS) based online label distribution learning framework (SLD_BLS). Broad learning system was adopted as the baseline model because its incremental weight updating scheme, excellent efficiency, and outstanding performance can process streaming data in dynamically changing environments. However, there are several challenges to adapt BLS for LDL: 1) BLS's update rule cannot process LDL data; 2) BLS cannot show the mapping relationship between feature space and label space; 3) BLS neglects the inter-label relationships. To tackle these challenges, 1) a new weight update rule for LDL is designed; 2) a manifold regularization is applied to exploit the feature space manifolds; 3) an explicit label collaborating approach is introduced to positively bootstrapping the model training. Extensive experiments on 13 label distribution datasets indicated the performance of SLD_BLS outperforms 9 state-of-the-art LDL models on most datasets with significant reductions in training time across 6 metrics.

An Analysis of Radio Frequency Transfer Learning Behavior

Transfer learning (TL) techniques, which leverage prior knowledge gained from data with different distributions to achieve higher performance and reduced training time, are often used in computer vision (CV) and natural language processing (NLP), but have yet to be fully utilized in the field of radio frequency machine learning (RFML). This work systematically evaluates how the training domain and task, characterized by the transmitter (Tx)/receiver (Rx) hardware and channel environment, impact radio frequency (RF) TL performance for example automatic modulation classification (AMC) and specific emitter identification (SEI) use-cases. Through exhaustive experimentation using carefully curated synthetic and captured datasets with varying signal types, channel types, signal to noise ratios (SNRs), carrier/center frequencys (CFs), frequency offsets (FOs), and Tx and Rx devices, actionable and generalized conclusions are drawn regarding how best to use RF TL techniques for domain adaptation and sequential learning. Consistent with trends identified in other modalities, our results show that RF TL performance is highly dependent on the similarity between the source and target domains/tasks, but also on the relative difficulty of the source and target domains/tasks. Results also discuss the impacts of channel environment and hardware variations on RF TL performance and compare RF TL performance using head re-training and model fine-tuning methods.

Feature Extraction Based on Sparse Coding Approach for Hand Grasp Type Classification

The kinematics of the human hand exhibit complex and diverse characteristics unique to each individual. Various techniques such as vision-based, ultrasonic-based, and data-glove-based approaches have been employed to analyze human hand movements. However, a critical challenge remains in efficiently analyzing and classifying hand grasp types based on time-series kinematic data. In this paper, we propose a novel sparse coding feature extraction technique based on dictionary learning to address this challenge. Our method enhances model accuracy, reduces training time, and minimizes overfitting risk. We benchmarked our approach against principal component analysis (PCA) and sparse coding based on a Gaussian random dictionary. Our results demonstrate a significant improvement in classification accuracy: achieving 81.78% with our method compared to 31.43% for PCA and 77.27% for the Gaussian random dictionary. Furthermore, our technique outperforms in terms of macro-average F1-score and average area under the curve (AUC) while also significantly reducing the number of features required.

A 3D ray traced biological neural network learning model

Training large neural networks on big datasets requires significant computational resources and time. Transfer learning reduces training time by pre-training a base model on one dataset and transferring the knowledge to a new model for another dataset. However, current choices of transfer learning algorithms are limited because the transferred models always have to adhere to the dimensions of the base model and can not easily modify the neural architecture to solve other datasets. On the other hand, biological neural networks (BNNs) are adept at rearranging themselves to tackle completely different problems using transfer learning. Taking advantage of BNNs, we design a dynamic neural network that is transferable to any other network architecture and can accommodate many datasets. Our approach uses raytracing to connect neurons in a three-dimensional space, allowing the network to grow into any shape or size. In the Alcala dataset, our transfer learning algorithm trains the fastest across changing environments and input sizes. In addition, we show that our algorithm also outperformance the state of the art in EEG dataset. In the future, this network may be considered for implementation on real biological neural networks to decrease power consumption.

SPARK: A High-Efficiency Black-Box Domain Adaptation Framework for Source Privacy-Preserving Drowsiness Detection.

Developing an effective and efficient electroencephalography (EEG)-based drowsiness monitoring system is crucial for enhancing road safety and reducing the risk of accidents. For general usage, cross-subject evaluation is indispensable. Despite progress in unsupervised domain adaptation (UDA) and source-free domain adaptation (SFDA) methods, these often rely on the availability of labeled source data or white-box source models, posing potential privacy risks. This study explores a more challenging setting of UDA for EEG-based drowsiness detection, termed black-box domain adaptation (BBDA). In BBDA, adaptation in the target domain relies solely on a black-box source model, without access to the source data or parameters of the source model. Specifically, we propose a framework called Self-distillation and Pseudo-labelling for Ensemble Deep Random Vector Functional Link (edRVFL)-based Black-box Knowledge Adaptation (SPARK). SPARK employs entropy-based selection of high-confidence samples, which are then pseudo-labeled to train a student edRVFL network. Subsequently, ensemble self-distillation is performed to extract knowledge by training the edRVFL using refined labels introduced by ensemble learning. This process further improves the robustness of the student edRVFL network. The features of the edRVFL are beneficial for improving the computational efficiency of the framework, making it more suitable for tasks involving small datasets. The proposed SPARK framework is evaluated on two publicly available driver drowsiness datasets. Experimental results demonstrate its superior performance over strong baselines, while significantly reducing training time. These findings underscore the potential for practical integration of the proposed framework into drowsiness monitoring systems, thereby contributing substantially to the privacy preservation of source subjects.

Language Translator Using Deep Learning

Abstract: Digital translation (MT) is a process of translating text from one language to another using software by including both computer and language information. Initially, the MT system acquires text translation into the source language by simply associating the meaning of the words in the source language with the target language with the help of grammar. However, such methods did not produce good results due to their failure to capture the various sentence structures in the language. This process of translation is time-consuming and requires skilled craftsmen in both languages. Subsequently, integrated translation methods such as mathematical translation (SMT) and neural translation (NMT) technology have been introduced to address the challenges of legal-based approaches. MT has already shown promising results in bilingual translation. In contrast to SMT, which requires sub-components trained separately in translation, NMT uses one large neural network for training. This paper outlines how to train a repetitive neural network for rearrangement for a source to identify. The default encoder helps to reconstruct the vectors of the target language. Therefore, powerful hardware (GPU) support is required. The GPU improves system performance by reducing training time.

Path planning of mobile robot based on improved TD3 algorithm in dynamic environment

This paper proposes an improved TD3 (Twin Delayed Deep Deterministic Policy Gradient) algorithm to address the flaws of low success rate and slow training speed, when using the original TD3 algorithm in mobile robot path planning in dynamic environment. Firstly, prioritized experience replay and transfer learning are introduced to enhance the learning efficiency, where the probability of beneficial experiences being sampled in the experience pool is increased, and the pre-trained model is applied in an obstacle-free environment as the initial model for training in a dynamic environment. Secondly, dynamic delay update strategy is devised and OU noise is added to improve the success rate of path planning, where the probability of missing high-quality value estimate is reduced through changing the delay update interval dynamically, and the correlated exploration of the mobile robot inertial navigation system in the dynamic environment is temporally improved. The algorithm is tested by simulation where the Turtlebot3 robot model as a training object, the ROS melodic operating system and Gazebo simulation software as an experimental environment. Meanwhile, the result shows that the improved TD3 algorithm has a 16.6 % increase in success rate and a 23.5 % reduction in algorithm training time. A generalization experiment was designed finally, and it indicates that superior generation performance has been acquired in mobile robot path planning with continuous action spaces through the improved TD3 algorithm.

Entropy-based deep neural network training optimization for optical coherence tomography imaging

ABSTRACT This paper presents an optimization technique for the number of training epochs needed for deep learning models. The proposed method eliminates the need for separate validation data and significantly decreases training epochs. Using a four-class Optical Coherence Tomography (OCT) image dataset encompassing Choroidal Neovascularization (CNV), Diabetic Macular Edema (DME), Drusen, and Normal retina categories, we evaluated twelve architectures. These include general-purpose models (Alexnet, VGG11, VGG13, VGG16, VGG19, ResNet-18, ResNet-34, and ResNet-50) and OCT image-specific models (RetiNet, AOCT-NET, DeepOCT, and Octnet). The proposed technique reduced training epochs ranging from 4.35% to 58.27% for all architectures except Alexnet. Although the overall increase in accuracy ranges from 0.28% to 12.6%, with some architectures experiencing minor improvements, this is seen as acceptable considering the substantial reduction in training time. By achieving higher accuracy with fewer training epochs and eliminating the need for separate validation data, our methodology streamlines early stopping significantly. Statistical evaluations via Shapiro-Wilk and Kruskal-Wallis tests further affirm these results, showcasing the potential of this novel technique for efficient deep learning practices in scenarios constrained by time or computational resources.

Generalizability of deep learning in organ-at-risk segmentation: A transfer learning study in cervical brachytherapy

PurposeDeep learning can automate delineation in radiation therapy, reducing time and variability. Yet, its efficacy varies across different institutions, scanners, or settings, emphasizing the need for adaptable and robust models in clinical environments. Our study demonstrates the effectiveness of the transfer learning (TL) approach in enhancing the generalizability of deep learning models for auto-segmentation of organs-at-risk (OARs) in cervical brachytherapy. MethodsA pre-trained model was developed using 120 scans with ring and tandem applicator on a 3T magnetic resonance (MR) scanner (RT3). Four OARs were segmented and evaluated. Segmentation performance was evaluated by Volumetric Dice Similarity Coefficient (vDSC), 95 % Hausdorff Distance (HD95), surface DSC, and Added Path Length (APL). The model was fine-tuned on three out-of-distribution target groups. Pre- and post-TL outcomes, and influence of number of fine-tuning scans, were compared. A model trained with one group (Single) and a model trained with all four groups (Mixed) were evaluated on both seen and unseen data distributions. ResultsTL enhanced segmentation accuracy across target groups, matching the pre-trained model’s performance. The first five fine-tuning scans led to the most noticeable improvements, with performance plateauing with more data. TL outperformed training-from-scratch given the same training data. The Mixed model performed similarly to the Single model on RT3 scans but demonstrated superior performance on unseen data. ConclusionsTL can improve a model’s generalizability for OAR segmentation in MR-guided cervical brachytherapy, requiring less fine-tuning data and reduced training time. These results provide a foundation for developing adaptable models to accommodate clinical settings.

Evolving filter criteria for randomly initialized network pruning in image classification

Network pruning aims to enhance the performance of deep neural networks by eliminating redundant components from the model. However, existing pruning methods typically require a well-trained model and employ fixed, single pruning criteria throughout the pruning cycles. To address these limitations, we propose a novel method called Evolutionary Filter Criteria (EvoFC). This method enables the automated search for the network pruning ratio and criterion during a population-based heuristic search process. We introduce a unique encoding space that represents the chosen pruning criterion and ratio for each layer, facilitating the acquisition of optimal architecture configurations for candidate networks during iterations. Additionally, we devise a novel weight inheritance mechanism to mitigate the computational burden associated with the population-based nature of the method, resulting in a significant reduction in overall training time. We validate our method by applying it to randomly initialized networks and conducting empirical experiments on CIFAR-10/100, ILSVRC-2012 and Places365 datasets. The results demonstrate that our method effectively reduces the number of FLOPs while striking a fine balance between accuracy and computational efficiency. This underscores the practical value of our method in optimizing performance while efficiently utilizing computational resources, particularly when pruning networks starting from random initialization.

An Efficient, Fast and Accurate Online Signature Verification Using Blended Feature Vector and Deep Learning

Online signature verification is one of the biometric authentication techniques. Online signature verification reduces the human error that may occur due to physical signature verification. Many researchers have provided a method of online signature verification still there is a scope for improvement. In this paper, a novel blended feature vector (BFV) is formed by combining two feature vectors. The first of these feature vectors is formed using a raw online signature database and the other feature vector is formed of the features extracted from the grayscale images obtained from the signature database. The algorithm makes use of the probability distribution of the features to provide a feature vector that results in an enhanced verification system. Two online signature databases, namely SVC2004 and ATVS-SSig are used. BFV is used to train the proposed mixed sequence deep neural network (MS-DNN). The use of a blended feature vector in the training of MS-DNN, instead of a whole online signature database, improves the training time of deep neural network to a great extent. This reduction in training time might be useful for new user registration. The performance parameters considered are validation efficiency and equal error rate. Results are compared with the existing state of technology and the comparison shows that the use of a blended feature vector as a classification vector improves the validation efficiency (99.5%) and also there is an improvement in the verification success rate in terms of equal error rate (EER 1.5%) as compared to the existing research.

Knowledge-shareable adaptive deep dynamic programming for hierarchical generation control of distributed high-percentage renewable energy systems

The increased proportion of distributed renewable energy sources connected to systems leads to frequency regulation difficulty, such as uncoordinated algorithms, inconsistent optimization and control objectives, and long training time. This work proposes a knowledge-shareable adaptive deep dynamic programming (KSADDP) to improve the active control accuracy and generation economy of the proposed hierarchical generation control framework, which contains two-layer control structures. In the first layer, droop control is employed to optimize the outputs of controllable generators, follow the frequency changes, reduce frequency deviation, and improve the operation economy of battery energy storage systems. In the second layer, a KSADDP is proposed to mitigate local optimum and accelerate the hybrid cooperative structure optimization by sharing the parameters of neural networks. These two layers synergistically optimize control performance and reduce generation costs. The simulation results of a three-high-percentage renewable energy system show that: with two layers of cooperation and sharing training information, the KSADDP outperforms the comparison algorithms in seven evaluation indices and reduces the generation cost by 4.85%–7.41 % compared to the comparison algorithms. The proposed KSADDP cooperates the time scales between multiple layers, cooperates the objectives between multiple layers, increases the parameter-sharing process, and reduces training time.

Quantum Machine Learning for Credit Scoring

This study investigates the integration of quantum circuits with classical neural networks for enhancing credit scoring for small- and medium-sized enterprises (SMEs). We introduce a hybrid quantum–classical model, focusing on the synergy between quantum and classical rather than comparing the performance of separate quantum and classical models. Our model incorporates a quantum layer into a traditional neural network, achieving notable reductions in training time. We apply this innovative framework to a binary classification task with a proprietary real-world classical credit default dataset for SMEs in Singapore. The results indicate that our hybrid model achieves efficient training, requiring significantly fewer epochs (350) compared to its classical counterpart (3500) for a similar predictive accuracy. However, we observed a decrease in performance when expanding the model beyond 12 qubits or when adding additional quantum classifier blocks. This paper also considers practical challenges faced when deploying such models on quantum simulators and actual quantum computers. Overall, our quantum–classical hybrid model for credit scoring reveals its potential in industry, despite encountering certain scalability limitations and practical challenges.