Fast Training Research Articles

Surface quality prediction and quantitative evaluation of process parameter effects for 3D printing with transfer learning-enhanced gradient-boosting decision trees

3D printing has the potential to revolutionize industrial manufacturing through efficient and sustainable techniques. Fused Deposition Modeling (FDM) is a broadly deployed technique among various 3D printing methods. However, the surface quality of FDM is greatly influenced by multiple factors, making it challenging to unravel the relationship between printing quality and parameter settings. To break through this bottleneck, this study proposes an intelligent approach that combines Transfer Learning (TL)-based Feature Extractor (FE) and Gradient-Boosting Decision Trees (GBDT) to investigate the effects of FDM printing parameters on surface quality. Experiments are conducted in the laboratory to validate the effectiveness of the FE-GBDT, which is then compared with the exemplary Machine Learning (ML) algorithms. The results show that our proposed TL model can achieve high precision and accuracy over 0.9900, demonstrating the efficacy of FE-GBDT in deciphering the impact of FDM printing parameters on surface quality. The contribution of each parameter is evaluated and indicates that layer height could dramatically affect the surface quality with an importance score of 0.626. The results provide valuable insights for the 3D printing community, proving that the FE-GBDT approach offers improved generalization, faster training, enhanced feature extraction, addressing data scarcity, and the ability to leverage the strengths of both approaches for superior performance across various tasks.

Bayesian extreme learning machines for hydrological prediction uncertainty

In recent years, extreme learning machines (ELM) have been used to accurately predict a variety of hydrological variables (e.g., streamflow, precipitation, river water quality). Using the same model structure, ELM often obtains similar performance to multi-layer perceptron (MLP) networks without the need for an iterative learning process (backpropagation), resulting in faster training. However, despite the increasing popularity of ELM, the hydrology literature has not focused on training algorithms that can be used to generate probabilistic predictions for this method. This is an interesting research gap, as it is generally accepted that quantifying hydrological prediction uncertainty and producing probabilistic predictions (instead of point predictions or mean value predictions) is a prerequisite for reliable water resource management. Thus, for the first time, Bayesian ELM (BELM) and sparse BELM (SBELM) methods are adopted and applied for probabilistic streamflow simulation and multi-step ahead forecasting (1–3 days), using as a case study four watersheds from Mexico, Germany, Canada, and Belgium. Using deterministic and probabilistic metrics, BELM and SBELM are compared against Bayesian linear regression (BLR), MLP combined with Monte-Carlo dropout weights, and a deep learning method: long short-term memory network (LSTM) coupled with Monte-Carlo dropout weights. Adding time-lagged observations of streamflow and meteorological variables (up to 14 days), such as precipitation and potential evapotranspiration, improves the simulation accuracy up to a factor of 10. In general, both BELM and SBELM show more accurate point predictions than MLP, BLR, and LSTM. BELM and SBELM show similar performance in terms of accuracy and reliability. Although, BELM marginally outperforms SBELM by generating, on average, a narrower prediction interval width. The sparsity feature of SBELM reduces (on average) the network size by 14–83 %. BELM and SBELM produce more accurate and reliable predictions than LSTM and are up to 122 and 125 times more computationally efficient to train, respectively. The case study suggests that BELM and SBELM are promising probabilistic machine learning models for hydrological prediction that are attractive alternatives to physical (e.g., lumped conceptual) models and common deep learning models (e.g., LSTM).

A White Dwarf Search Model Based on a Deep Transfer-learning Method

White dwarfs represent the ultimate stage of evolution for over 97% of stars and play a crucial role in studies of the Milky Way’s structure and evolution. Recent years have witnessed significant progress in using deep-learning methods for identifying unique objects in large-scale data. In this paper, we present a model based on transfer learning for identifying white dwarfs. We constructed a data set using the spectra released by LAMOST DR9 and trained a convolutional neural network model. The model was then further trained using a transfer-learning approach for a binary classification model. Our final model is comprised of a seven-class classification model and a binary classification model. The testing set yielded an accuracy rate of 96.08%. Our proposed model successfully identifies 4314 of the 4479 white dwarfs published in previous papers. We applied this model to filter the 1,121,128 spectral data from the LAMOST DR9 V1 catalog. Subsequently, we obtained 6317 white dwarf candidates, of which 5014 were cross-validated and found to be known white dwarfs. We finally identified 489 new white dwarfs out of the remaining 1303 candidates, containing 377 DAs, 1 DB, 4 DZs, 1 magnetic WD, 101 DA+M binaries, and 1 DB+M binary. Our study also compared transfer-learning methods with non-transfer-learning methods, and the results show that transfer learning provides faster training speed and a higher accuracy rate. We provide the trained model and a corresponding usage program for subsequent studies.

Research on an Improved Detection Algorithm Based on YOLOv5s for Power Line Self-Exploding Insulators

In the process of inspecting the self-exploding defects of power line insulators, traditional algorithms suffer from various issues such as long detection time, insufficient accuracy, and difficulties in effective detection under complex environments. To address these problems, we introduce an advanced one-stage object detection algorithm called YOLOv5s, which offers fast training and excellent detection performance. In this paper, we applied the YOLOv5s algorithm to improve the detection precision and classification accuracy of insulator self-explosions. To further enhance the YOLOv5s algorithm, we introduced a BiFPN (Bidirectional Feature Pyramid Network) module for feature fusion. This module improved the feature fusion process by learning the importance weights of different input features, considering their contributions. To tackle the challenge of detecting small objects in the self-exploding insulator dataset, we incorporated an SPD (spatial-to-depth convolution) module that focuses on capturing features in small regions and utilizes one-step convolution layers to avoid losing fine-grained information. To address the issue of high similarity between self-exploding insulator regions and intact insulator regions, we introduced an attention mechanism that concentrates attention on the defective insulator regions to gather more information about insulator defects. Experimental results validate that all three improvement methods significantly enhance detection precision. The final model achieves improvements of 2.0% in precision, 0.9% in recall, and 1.5% in average detection accuracy. Through target detection of the test dataset, insulators with self-explosion cases can be effectively detected.

Real-Time Video Smoke Detection Based on Deep Domain Adaptation for Injection Molding Machines

Leakage with smoke is often accompanied by fire and explosion hazards. Detecting smoke helps gain time for crisis management. This study aims to address this issue by establishing a video smoke detection system, based on a convolutional neural network (CNN), with the help of smoke synthesis, auto-annotation, and an attention mechanism by fusing gray histogram image information. Additionally, the study incorporates the domain adversarial training of neural networks (DANN) to investigate the effect of domain shifts when adapting the smoke detection model from one injection molding machine to another on-site. It achieves the function of domain confusion without requiring labeling, as well as the automatic extraction of domain features and automatic adversarial training, using target domain data. Compared to deep domain confusion (DDC), naïve DANN, and the domain separation network (DSN), the proposed method achieves the highest accuracy rates of 93.17% and 91.35% in both scenarios. Furthermore, the experiment employs t-distributed stochastic neighbor embedding (t-SNE) to facilitate fast training and smoke detection between machines by leveraging domain adaption features.

Pneumonia Detection on Chest X- Ray using Deep Learning

Abstract: In recent years, Pneumonia has become a serious issue in people. The pneumonia which affects the lungs in a way by accumulating the fluid in the lungs. This pneumonia should be treated effectively and should be identified in early stages. The inaccuracies of some algorithms had created problems in diagnosing pneumonia. We came up with an idea to detect pneumonia by taking Chest X-rays and applying a novel algorithm which includes Feature maps, gradient vanishing, skip connection and faster training of the model, minimum computation. This could bring significant impact and help the doctors to identify the pneumonia on their knowledge as we will train the model with different images and we identify Pneumonia effectively.

E-Learning in Health: Investigation of Technology Acceptance and Usage of Family Physicians According to Demographic Factors

E-learning is the changing and renewing face of education in health as in many fields. Developments in technology prepare a suitable environment for the realization of various structural changes in the continuing education of health professionals. Since e-learning provides asynchronous learning opportunities to health professionals working in different geographical regions, it provides various opportunities for the career development of health professionals. One of Turkey's most significant e-learning projects in the health field was carried out by the Family Medicine Distance Education Centre (AHUZEM). The e-learning project prepared for family physicians aimed to provide much more accessible, effective, and faster training for family physicians. This study aims to reveal family physicians' adoption of e-learning applications according to demographic factors. In the study, the Unified Technology Acceptance and Use Model (UTAUT) developed by Venkatesh et al. (2003) by examining many theories and models was used as a data collection tool. In addition, the author integrated the attitude towards technology use dimension consisting of 4 questions into the model by examining the relevant literature. In addition, the research was organized according to a 5-point Likert-type scale. The data was obtained from 429 family physicians who agreed to participate in the online survey shared on social networks nationwide by convenience sampling. The results obtained by entering the data into AMOS 18.0 program were analyzed in t-test and ANOVA tests. According to the results of the analyses, a significant difference was found in the intention to use e-learning and UTAUT according to age, specialty, working time, and e-learning experience. However, no significant difference was found according to gender.

Multi-Criteria Feature Selection Based Intrusion Detection for Internet of Things Big Data.

The rapid growth of the Internet of Things (IoT) and big data has raised security concerns. Protecting IoT big data from attacks is crucial. Detecting real-time network attacks efficiently is challenging, especially in the resource-limited IoT setting. To enhance IoT security, intrusion detection systems using traffic features have emerged. However, these face difficulties due to varied traffic feature formats, hindering fast and accurate detection model training. To tackle accuracy issues caused by irrelevant features, a new model, LVW-MECO (LVW enhanced with multiple evaluation criteria), is introduced. It uses the LVW (Las Vegas Wrapper) algorithm with multiple evaluation criteria to identify pertinent features from IoT network data, boosting intrusion detection precision. Experimental results confirm its efficacy in addressing IoT security problems. LVW-MECO enhances intrusion detection performance and safeguards IoT data integrity, promoting a more secure IoT environment.

PENGENALAN SARANA PERKERETAAPIAN PADA SISWA DAN GURU SMPN 1 NGRAHO BOJONEGORO

At this time, transportation and mobility of people in Indonesia have used many modes of rail transportation. This choice is influenced by the various advantages offered by rail transportation modes such as its large and efficient transport capacity, on time and good accessibility. Railway technology has undergone very significant developments in recent times. Starting from conventional trains, electric trains and even fast trains (High-speed Train). One of the things that influence the development of railways is the application of science and technology, one of which is science and engineering. Therefore, strengthening the basis of science and introducing technology must be carried out, including at the primary and secondary education levels as an effort to prepare quality human resources for the development of railroad technology in the future. The introduction of rolling stock to students and teachers at SMPN 1 Ngraho Bojonegoro aims to enable students and teachers to have a deeper understanding of railway technology, especially the types of railway facilities in the hope that it will become one of the tools in learning science and technology. The method used in this activity is counseling with several stages such as preparation, implementation and evaluation. The core activity is to provide a presentation regarding the development of railway technology in general, followed by an understanding of railway facilities and types of rolling stock. The results of the evaluation showed that there was an increase in knowledge and understanding regarding railways and rolling stock among the participants. The average percentage increase in understanding of grades pre-test and post-test is equal to 76.5%. In addition, 80% of respondents stated that this activity was very useful.

IoT based real-time traffic monitoring system using images sensors by sparse deep learning algorithm

Intelligent traffic monitoring systems are necessary and useful tools due to the emerging technologies related to the Internet of Things (IoT) and Artificial Intelligence (AI). The integration of both technologies can facilitate better urban traffic management for fast networks, such as 5G, 5G+ and 6G environments. However, existing studies focus on complex and expensive solutions or present a latency for generating new training models with accuracy not superior to 98%. Thus, this paper proposes a vehicle identification using a sparse and soft variant of the CNN-based approach, the LightSpaN, which offers a fast training model, without using a complex solution. The effectiveness of the proposed solution is evaluated using the Simulation of Urban MObility (SUMO) tool and a real vehicle traffic implementation using IoT devices. The proposal identified the majority kind of vehicles in a short period of time, faster and more accurately than the related works. The results validated the proposed solution for a real-time traffic monitoring system, presenting an average accuracy of around 99.9% for emergency vehicles. Furthermore, a reduction of both total waiting time and total traveling time was reached by our proposal.

Statistical process control versus deep learning for power plant condition monitoring

This study compares four models for industrial condition monitoring including a principal components analysis (PCA) approach and three deep learning models, one of which is a new, lightweight version of another. We also propose a simple attention mechanism for enchancing deep learning models with better predictions and feature importance. Two datasets are used, one simulated from the Tennessee Eastman Process, the other from two feedwater pumps at a Danish combined heat and power plant. Our final results show evidence in favour of the PCA-based approach as it has detection ability comparable to the deep learning approaches as well as faster training time, fewer hyperparameters, as well as robustness to changing operating conditions. We conclude the paper by putting into perspective the importance of building up complexity incrementally with a recommendation to start modelling with simpler and well-tested models before the adoption of more advanced, less transparent models.

Interpretable single-cell transcription factor prediction based on deep learning with attention mechanism

Predicting the transcription factor binding site (TFBS) in the whole genome range is essential in exploring the rule of gene transcription control. Although many deep learning methods to predict TFBS have been proposed, predicting TFBS using single-cell ATAC-seq data and embedding attention mechanisms needs to be improved. To this end, we present IscPAM, an interpretable method based on deep learning with an attention mechanism to predict single-cell transcription factors. Our model adopts the convolution neural network to extract the data feature and optimize the pre-trained model. In particular, the model obtains faster training and prediction due to the embedded attention mechanism. For datasets, we take ATAC-seq, ChIP-seq, and DNA sequences data for the pre-trained model, and single-cell ATAC-seq data is used to predict the TF binding graph in the given cell. We verify the interpretability of the model through ablation experiments and sensitivity analysis. IscPAM can efficiently predict the combination of whole genome transcription factors in single cells and study cellular heterogeneity through chromatin accessibility of related diseases.

Automatic recognition of coronary artery disease and congestive heart failure using a multi-granularity cascaded hybrid network

Numerous researchers have developed electrocardiogram (ECG) classification systems to automatically diagnose coronary artery disease (CAD) and congestive heart failure (CHF). However, they have failed to assess the applicability of their proposed methods on inter-patient, noisy, imbalanced and small-scale data, limiting their use in real-world situations. To address these issues, a multi-granularity cascaded hybrid network (MGCH-Net) is proposed to automatically identify abnormal ECGs. MGCH-Net mainly consists of a multi-granularity cascaded task-related component analysis (TRCA)-principal component analysis (PCA) network (MGC-TPNet), a multi-granularity cascaded independent component analysis (ICA)-PCA network (MGC-IPNet), and a cascaded weighted averaging and Dempster–Shafer (CWA-DS) method. Among them, MGC-TPNet extracts multi-scale inter-lead correlation features from multi-lead ECGs, by which the information from the sagittal and horizontal planes of the heart is comprehensively utilized by maximizing the reproducibility under the same task. Then, MGC-IPNet mines multi-scale intra-lead specificity features from more-valuable-lead ECGs. Finally, CWA-DS is employed to fuse linear support vector machine (SVM)- and softmax-based decision probabilities for the above features to obtain the final results. Since the convolutional kernels of MGCH-Net can be directly calculated through TRCA, PCA, and FastICA algorithms, multiple iterative processes are not required. Hence, MGCH-Net has a faster training speed than most convolutional neural networks. In this work, accuracies of 99.92% and 97.46% were achieved in recognizing normal, CAD and CHF heartbeats in intra- and inter-patient experiments, respectively. In addition, MGCH-Net performed well on imbalanced, noisy, and small-scale data. These results show that our method outperforms the state-of-the-art methods.

PipePar: Enabling fast DNN pipeline parallel training in heterogeneous GPU clusters

Recently, pipeline parallelism for large-scale Deep Neural Network (DNN) training has been developed, which partitions the DNN model across multiple devices (e.g., GPUs) and improves the training efficiency by processing data divided into minibatches as a pipeline. However, existing model partitioning algorithms are mostly designed for homogeneous clusters with the same GPU devices and network connections (e.g., bandwidths), while heterogeneous GPU clusters are widely used in mainstream computing infrastructures. In heterogeneous environment, devices are equipped with different GPUs and network connections, and the efficiency of previous approaches is unsatisfactory due to the unbalanced load of the pipeline stages. In this paper, we propose PipePar, a model partitioning and task placement algorithm for pipeline parallel DNN training in heterogeneous GPU clusters. PipePar is based on dynamic programming with search space pruning that takes into consideration both the heterogeneity of GPUs and network bandwidth. PipePar can profile the DNN model for each type of GPU and conduct model partitioning and task placement based on given GPUs and network connections, which can optimize pipeline load balancing in heterogeneous environments and thus improve training efficiency. We design and implement a pipeline-based distributed deep learning training system in a heterogeneous GPU cluster and show through extensive experiments that PipePar outperforms the baseline approaches in the speed of large-scale DNN training.

Research on Coal and Gangue Recognition Model Based on CAM-Hardswish with EfficientNetV2

In response to the multiscale shape of coal and gangue in actual production conditions, existing coal separation methods are inefficient in recognizing coal and gangue, causing environmental pollution and other problems. Combining image data preprocessing and deep learning techniques, this paper presents an improved EfficientNetV2 network for coal and gangue recognition. To expand the dataset and prevent network overfitting, a pipeline-based data enhancement method is used on small sample datasets to simulate coal and gangue production conditions under actual working conditions. This method involves modifying the attention mechanism module in the model, employing the CAM attention mechanism module, selecting the Hardswish activation function, and updating the block structure in the network. The parallel pooling layer introduced in the CAM module can minimize information loss and extract rich feature information compared with the single pooling layer of the SE module. The Hardswish activation function is characterized by excellent numerical stability and fast computation speed. It can effectively be deployed to solve complex computation and derivation problems, compensate for the limitations of the ReLu activation function, and improve the efficiency of neural network training. We increased the training speed of the network while maintaining the accuracy of the model by selecting optimized hyperparameters for the network structure. Finally, we applied the improved model to the problem of coal and gangue recognition. The experimental results showed that the improved EfficientNetV2 coal and gangue recognition method is easy to train, has fast convergence and training speeds, and thus achieves high recognition accuracy under insufficient dataset conditions. The accuracy of coal and gangue recognition increased by 3.98% compared with the original model, reaching 98.24%. Moreover, the training speed improved, and the inference time of the improved model decreased by 6.6 ms. The effectiveness of our proposed model improvements is confirmed by these observations.

Modified Knowledge-Based Neural Networks Using Control Variates for the Fast Uncertainty Quantification of On-Chip MWCNT Interconnects

In this article, a modified knowledge-based artificial neural network (KBANN) metamodel is developed for the efficient uncertainty quantification of on-chip multiwalled carbon nanotube (MWCNT) interconnects. The proposed KBANN metamodel utilizes the notion of control variates to enable much faster training than what is possible with standard KBANNs. Importantly, techniques to calculate the optimal value of the control variates in an a priori manner without augmenting the training dataset have been developed in this article. Furthermore, techniques to exploit the control variates depending on whether one or multiple low-fidelity models of the MWCNT interconnects are available have also been developed in this article. The benefits of the proposed KBANN metamodel using control variates over standard KBANN metamodels have been validated using multiple MWCNT interconnect examples spanning multiple technology nodes.

Accelerating Graph Neural Network Training on ReRAM-Based PIM Architectures via Graph and Model Pruning

Graph Neural Networks (GNNs) are used for predictive analytics on graph-structured data, and they have become very popular in diverse real-world applications. Resistive random-access memory (ReRAM)-based PIM architectures can accelerate GNN training. However, GNN training on ReRAMbased architectures is both compute-and data-intensive in nature. In this work, we propose a framework called SlimGNN that synergistically combines both graph and model pruning to accelerate GNN training on ReRAM-based architectures. The proposed framework reduces the amount of redundant information in both the GNN model and input graph(s) to streamline the overall training process. This enables fast and energy-efficient GNN training on ReRAM-based architectures. Experimental results demonstrate that using this framework, we can accelerate GNN training by up to 4.5× while using 6.6× less energy compared to the unpruned counterparts.

Intentionally Slow Concentric Velocity Resistance Exercise and Strength Adaptations: A Meta-Analysis.

Hermes, MJ and Fry, AC. Intentionally slow concentric velocity resistance exercise and strength adaptations: a meta-analysis. J Strength Cond Res 37(8): e470-e484, 2023-Intentionally slow-velocity resistance exercise (ISVRE) is suggested by some to be equally or more effective than fast or traditional velocities for increasing muscular strength. The purpose of this meta-analysis was to assess the effect ISVRE has on strength adaptations compared with faster or traditional velocities, with subgroup analyses exploring age, sex, and training status as confounding factors on the influence of velocity on strength adaptation. Eligible studies (n = 24) were required to be chronic (multiple weeks) randomized or nonrandomized comparative studies using dynamic constant external resistance for training and testing, and pre-post strength assessments. All studies examined healthy individuals (n = 625; fast or traditional n = 306, intentionally slow n = 319). A random-effects meta-analysis indicated a significant (p ≤ 0.05) effect in favor of fast training (effect size [ES] = 0.21, 95% confidence interval [CI] = 0.02-0.41, p = 0.03). Publication bias was noted through trim and fill analysis, with an adjusted effect size estimate of 0.32 (p < 0.001). Subgroup analyses indicated no difference between trained and untrained subjects (QM = 0.01, p = 0.93), and no difference between older and younger subgroups (QM = 0.09, p = 0.77), despite younger favoring faster (ES = 0.23, p = 0.049) and older not favoring either velocity (ES = 0.16, p = 0.46). Subgroup analysis also indicated women favored faster training (ES = 0.95, p < 0.001) in comparison to men (ES = 0.08, p = 0.58). Contrary to some previous reviews, these results indicate that chronic fast or traditional velocity resistance exercise increases muscular strength to a greater degree than ISVRE training. Resistance training velocity must be considered if strength is a desired outcome.

Experimental analysis of coal gangue identification using visible-near infrared spectroscopy under different working conditions with local and global model

ABSTRACT In response to the deficiencies in visible-near infrared spectral identification of coal gangue under different working conditions, this paper employs multiple algorithms to construct local and global recognition models and conducts analysis. Simulating five different working conditions in the laboratory, including different detection angles, illumination angles, detection distances, coal powder content, and moisture content. Each working condition is further divided into four levels. Collecting spectral data of coal and gangue samples. Constructing global and local datasets, which are respectively derived from all levels and single levels under certain working conditions. Training local and global models to identify samples based on machine learning algorithms such as Lightgbm and 1DCNN. The results of the experiment indicate that there are variations in the sample spectra and recognition accuracy based on different working conditions. Furthermore, the local model exhibits lower recognition accuracy and inadequate robustness for other level samples under the same working condition. The global model is more robust and practical, as it achieves higher or equal recognition accuracy for local data compared to the local model. Among the global models, Lightgbm has the best overall performance, with an average F1-score of 99.34% and 97.9% for local and global datasets, respectively. It also has fast training time of 0.75s and recognition time of 1 ms. The Lightgbm global model can accurately identify coal and gangue in various working conditions, making it a reliable solution.

AC2AS: Activation Consistency Coupled ANN-SNN framework for fast and memory-efficient SNN training

Spiking neural networks are efficient computation models for low-power environments. Spike-based BP algorithms and ANN-to-SNN (ANN2SNN) conversions are successful techniques for SNN training. Nevertheless, the spike-base BP training is slow and requires large memory costs, while ANN2SNN needs many inference steps to obtain good performance. In this paper, we propose an Activation Consistency Coupled ANN-SNN (AC2AS) framework to train the SNN in a fast and memory-efficient way. The AC2AS consists of two components: (a) a weight-shared architecture between ANN and SNN and (b) spiking mapping units. Firstly, the architecture trains the weight-shared parameters on the ANN branch, resulting in fast training and low memory costs for SNN. Secondly, the spiking mapping units are designed to ensure that the activation values of the ANN are the spiking features. As a result, the activation consistency is guaranteed, and the classification error of the SNN can be optimized by training the ANN branch. Besides, we design an adaptive threshold adjustment (ATA) algorithm to decrease the firing of noisy spikes. Experiment results show that our AC2AS-based models perform well on the benchmark datasets (CIFAR10, CIFAR100, and Tiny-ImageNet). Moreover, the AC2AS achieves comparable accuracy under 0.625× time steps, 0.377× training time, 0.27× GPU memory costs, and 0.33× spike activities of the Spike-based BP model. The code is available at https://github.com/TJXTT/AC2ASNN.