Trained Network Model Research Articles

Hot rolled steel surface defect detection and classification using an automatic ensemble approach

This study introduces an ensemble-based Deep Neural Network (DNN) model for detecting defects on steel surfaces. The method suggested in this study classifies steel surface conditions into six possible fault categories, namely, crazing, inclusion, rolled in, pitted surface, scratches, and patches. The images undergo preprocessing and extraction of features in spatial and frequency domains using image segmentation techniques such as grey level difference method (GLDM), fast Fourier Transform (FFT), grey level co-occurrence matrix (GLCM), texture analysis and discrete wavelet transform (DWT). The ensembling of image features into a fused feature pool is carried out after the preprocessing of input images that are provided as input to a light-weight neural network model for training and testing. The performance of the model is comprehensively evaluated via an ablation study both before and after ensembling. In addition, the model capability is effectively analyzed using receiver operating characteristics (ROC) curve, confusion matrix from which classification accuracy of the model could be obtained and other parameters including precision and f1-score. It was observed that the proposed deep learning network presents phenomenally high accuracy of 99.72% for detection and classification of steel surface faults. This result was found to be superior when compared with the performance of the same neural network over each feature type individually. This study also compares the classification results of the model built based on the ensembled feature set with the results of various other classification approaches available in literature. The ensemble-based model could potentially be integrated into existing inspection systems for real-time, efficient and robust condition monitoring of steel surfaces.

Frequency-extended inverse design of transmission-type linear-to-circular polarization control metasurface based on deep learning

Due to the parameter range limitations of the training dataset, traditional inverse prediction network models can only predict structure parameters of the metasurface within a limited frequency range. When the given design targets exceed the prediction range of network models, the predicted results will not match the actual results. This paper proposes a frequency-extended inverse design method (FEIDM) based on deep learning to address the problem. The method can automatically collect the required data and train the network model based on the center working frequency of the design targets, thereby achieving accurate prediction of metasurface structural parameters and effectively reducing labor and computational costs. Taking the transmission-type linear-to-circular polarization control metasurface as an example, the unit cell of the metasurface is first established in the paper. The structural parameters and corresponding electromagnetic parameters are collected without changing the unit size of the metasurface, and an initial inverse prediction network model (IIPNM) is constructed. The research results indicate that its predictable center working frequency range is 3–5.5 GHz. Using the design concept proposed in this paper, a program is constructed, it can automatically achieve data collection, target extraction, network model training, and prediction. Four given design targets are predicted. Among them, the center working frequencies of the three design targets are outside the initial predictable range. The predicted results meet the requirements of the given target, verifying the effectiveness of the proposed scheme. Finally, a set of parameters is selected to fabricate, and the experimental results are consistent with the simulation results. The research results can provide a reference for the efficient prediction of metasurface structural parameters over a wide frequency band.

High-Voltage Cable Buffer Layer Ablation Fault Identification Based on Artificial Intelligence and Frequency Domain Impedance Spectroscopy.

In recent years, the occurrence of high-voltage cable buffer layer ablation faults has become frequent, posing a serious threat to the safe and stable operation of cables. Failure to promptly detect and address such faults may lead to cable breakdowns, impacting the normal operation of the power system. To overcome the limitations of existing methods for identifying buffer layer ablation faults in high-voltage cables, a method for identifying buffer layer ablation faults based on frequency domain impedance spectroscopy and artificial intelligence is proposed. Firstly, based on the cable distributed parameter model and frequency domain impedance spectroscopy, a mathematical model of the input impedance of a cable containing buffer layer ablation faults is derived. Through a simulation, the input impedance spectroscopy at the first end of the cables under normal conditions, buffer layer ablation, local aging, and inductive faults is performed, enabling the identification of inductive and capacitive faults through a comparative analysis. Secondly, the frequency domain amplitude spectroscopy of the buffer layer ablation and local aging faults are used as datasets and are input into a neural network model for training and validation to identify buffer layer ablation and local aging faults. Finally, using multiple evaluation metrics to assess the neural network model validates the superiority of the MLP neural network in cable fault identification models and experimentally confirms the effectiveness of the proposed method.

Fast Aerodynamic Prediction of Airfoil with Trailing Edge Flap Based on Multi-Task Deep Learning

Conventional methods for solving Navier–Stokes (NS) equations to analyze flow fields and aerodynamic forces of airfoils with trailing edge flaps (TEFs) are known for their significant time cost. This study presents a Multi-Task Swin Transformer (MT-Swin-T) deep learning framework tailored for swift prediction of velocity fields and aerodynamic coefficients of TEF-equipped airfoils. The proposed model combines a Swin Transformer (Swin-T) for flow field prediction with a multi-layer perceptron (MLP) dedicated to lift coefficient prediction. Both networks undergo gradient updates through the shared encoder component of the Swin Transformer. Such a trained network model for computational fluid dynamics simulations is both effective and robust, significantly improving the efficiency of complex aerodynamic shape design optimization and flow control. The study further investigates the impact of integrating multi-task learning loss functions, skip connections, and the network’s structural design on prediction accuracy. Additionally, the effectiveness of deep learning in improving the aerodynamic simulation efficiency of airfoils with TEF is examined. Results demonstrate that the multi-task deep learning approach provides accurate predictions for TEF airfoil flow fields and lift coefficients. The strategic combination of these tasks during network training, along with the optimal selection of loss functions, significantly enhances prediction accuracy compared with the single-task network. In a specific case study, the MT-Swin-T model demonstrated a prediction time that was 1/7214 of the time necessitated by CFD simulation.

Polymorphic Clustering and Approximate Masking Framework for Fine-Grained Insect Image Classification

Insect diversity monitoring is crucial for biological pest control in agriculture and forestry. Modern monitoring of insect species relies heavily on fine-grained image classification models. Fine-grained image classification faces challenges such as small inter-class differences and large intra-class variances, which are even more pronounced in insect scenes where insect species often exhibit significant morphological differences across multiple life stages. To address these challenges, we introduce segmentation and clustering operations into the image classification task and design a novel network model training framework for fine-grained classification of insect images using multi-modality clustering and approximate mask methods, named PCAM-Frame. In the first stage of the framework, we adopt the Polymorphic Clustering Module, and segmentation and clustering operations are employed to distinguish various morphologies of insects at different life stages, allowing the model to differentiate between samples at different life stages during training. The second stage consists of a feature extraction network, called Basenet, which can be any mainstream network that performs well in fine-grained image classification tasks, aiming to provide pre-classification confidence for the next stage. In the third stage, we apply the Approximate Masking Module to mask the common attention regions of the most likely classes and continuously adjust the convergence direction of the model during training using a Deviation Loss function. We apply PCAM-Frame with multiple classification networks as the Basenet in the second stage and conduct extensive experiments on the Insecta dataset of iNaturalist 2017 and IP102 dataset, achieving improvements of 2.2% and 1.4%, respectively. Generalization experiments on other fine-grained image classification datasets such as CUB200-2011 and Stanford Dogs also demonstrate positive effects. These experiments validate the pertinence and effectiveness of our framework PCAM-Frame in fine-grained image classification tasks under complex conditions, particularly in insect scenes.

Structural Damage Identification Based on Preprocessing Samples Mixed Training of Multi-Channel and Multi-Source Dynamic Response Data

Previous studies have shown that sampling processing and identification methods for neural networks damage identification in multi-channel and multi-source structural dynamic response data exhibit diversity, and the robustness and generalization issues of the model have not been effectively solved. This paper proposes a sample preprocessing technique and mixed training method suitable for multi-channel and multi-source dynamic response data to optimize the current structural damage identification methods based on neural networks. Through multi-dimensional discrete autocorrelation processing and Fourier transform, the preprocessed datasets from multiple sensors with multi-channel can access multi-channel CNN. Furthermore, the measured datasets from the actual structures are mixed with numerical simulation datasets before being used for neural network model training to address the model calibration and the imbalanced sample size under each classification label. The results of extensive model experiments and finite element verification of specimens show that the method performs outstandingly in detecting and identifying damage to simply supported beam structures using multi-channel CNN. The neural network model trained with preprocessed samples exhibits excellent robustness. The model using the mixed training method still performs well in identifying the accuracy of damage location and degree in simply supported beam structures after changing beam section and initial excitation. The method has certain generalization ability in detecting unknown damage in simply supported beam structures.

A Comparative Study of Physically Accurate Synthetic Shadow Datasets in Agricultural Settings with Human Activity.

Shadow, a natural phenomenon resulting from the absence of light, plays a pivotal role in agriculture, particularly in processes such as photosynthesis in plants. Despite the availability of generic shadow datasets, many suffer from annotation errors and lack detailed representations of agricultural shadows with possible human activity inside, excluding those derived from satellite or drone views. In this paper, we present an evaluation of a synthetically generated top-down shadow segmentation dataset characterized by photorealistic rendering and accurate shadow masks. We aim to determine its efficacy compared to real-world datasets and assess how factors such as annotation quality and image domain influence neural network model training. To establish a baseline, we trained numerous baseline architectures and subsequently explored transfer learning using various freely available shadow datasets. We further evaluated the out-of-domain performance compared to the training set of other shadow datasets. Our findings suggest that AgroSegNet demonstrates competitive performance and is effective for transfer learning, particularly in domains similar to agriculture.

Acoustic emission source location for composite tubes using finite element simulation and machine learning

ABSTRACT Acoustic emission is widely used in engineering structural health monitoring, where damage location based on acoustic emission have great advantages. Currently, most of the acoustic emission location work requires extensive experimental studies. In this study, acoustic emission source location model was proposed based on BP neural network and acoustic emission wave propagation simulation, thus reduced the experiments required for acoustic emission localisation and improved the location accuracy. Initially, finite element simulation was conducted to study the propagation of acoustic emission waves in composite tubes. The calculated findings showed a high level of agreement with the experimental data. Additionally, the first arrival time of the signal was extracted according to the Akaike information criterion, and the neural network was trained and tested using the simulated data, while the trained network model was validated using the experimental data. The results show that the finite element method is a reliable alternative test method. More than 94% of the sound source location results have absolute location error within ± 3 mm, and the all points is less than ± 5 mm. The deviation of the location results showed significant improvement compared to the traditional time difference of arrival method, verifying the feasibility of the method.

USES-Net: An Infrared Dim and Small Target Detection Network with Embedded Knowledge Priors

Detecting and identifying small infrared targets has always been a crucial technology for many applications. To address the low accuracy, high false-alarm rate, and poor environmental adaptability that commonly exist in infrared target detection methods, this paper proposes a composite infrared dim and small target detection model called USES-Net, which combines the target prior knowledge and conventional data-driven deep learning networks to make use of both labeled data and the domain knowledge. Based on the typical encoder–decoder structure, USES-Net firstly introduces the self-attention mechanism of Swin Transformer to replace the universal convolution kernel at the encoder end. This helps to extract potential features related to dim, small targets in a larger receptive field. In addition, USES-Net includes an embedded patch-based contrast learning module (EPCLM) to integrate the spatial distribution of the target as a knowledge prior in the training network model. This guides the training process of the constrained network model with clear physical interpretability. Finally, USES-Net also designs a bottom-up cross-layer feature fusion module (AFM) as the decoder of the network, and a data-slicing-aided enhancement and inference method based on Slicing Aided Hyper Inference (SAHI) is utilized to further improve the model’s detection accuracy. An experimental comparative analysis shows that USES-Net achieves the best results on three typical infrared weak-target datasets: NUAA-SIRST, NUDT-SIRST, and IRSTD-1K. The results of the target segmentation are complete and sufficient, which demonstrates the validity and practicality of the proposed method in comparison to others.

Maximum Thinning Rate Prediction of Friction Heat Single-Point Incremental Forming for AZ31B Magnesium Alloy Based on BP Neural Network

The purpose of this project is to build an accurate prediction model for the maximum thinning rate of the truncated cone made of magnesium alloy AZ31B by single-point incremental forming. A truncated cone’s thickness distribution is analyzed using ABAQUS. Orthogonal experiments and supplementary data were performed as training samples within the maximum refinement rate obtained from the finite element (FE) results. Back-propagation (BP) neural network is established by taking into account tool diameter, initial sheet thickness, and forming angle as input responses, and maximum thinning rate as output responses, and nine hidden layers are involved. The results showed that the incidence of process parameters on the maximum thinning rate of the truncated cone is in the order of forming angle, initial thickness of sheet, and diameter of tool from large to small; The correlation coefficient of BP neural network model training [Formula: see text]. The test results and FE results contrast, and the average error rate of the maximum thinning rate is only 0.97%. The results indicate that the model is reliable and accurate for the prediction of incremental forming, and it is a useful reference value.

Deep-time temperature field simulation of hot dry rock: A deep learning method in both time and space dimensions

Hot dry rock has emerged as a crucial renewable resource with a potential to provide significant energy to China. Accurately simulating the temperature field of hot dry rocks in space is essential in estimating their reserves. Moreover, predicting the temperature change curve of hot dry rocks over time is crucial for research in the field. Although these tasks are interrelated and share a common ground of simulating the temperature of hot dry rocks, previous approaches have dealt with them separately, each being challenging. In this study, we propose a 3D U-shaped deep-time neural network that can extract the nonlinear relationship between the temperature field and rock strata parameters, thereby enabling the simulation of the temperature field of hot dry rocks in both space and time dimensions. The proposed neural network, which incorporates an attention mechanism and residual structure, can incorporate multiple rock parameters and geological structures, leading to more accurate simulation of the temperature field of hot dry rocks. To train the network, we constructed extensive datasets based on granite rock parameters and actual geological structures, with the temperature fields simulated by the finite element method serving as labels. The resulting trained network model can successfully emulate the complex hot dry rock model, reducing the workload by combining the two networks into one. Furthermore, the simulated temperature field allows the observation of the effect of different geological activities on hot dry rocks. Network simulation has shown notable advancements in efficiency compared to traditional finite element methods. Moreover, it offers the advantage of exploring the impact of distinct geological structures on geothermal reservoirs in a spatial context. Additionally, it allows for the observation of temporal variations in the temperature distribution across different geological strata during different time periods. Thereby laying the groundwork for further prediction and exploitation of hot dry rock geothermal reserves.

Research on a Small-Sample Fault Diagnosis Method for UAV Engines Based on an MSSST and ACS-BPNN Optimized Deep Convolutional Network

Regarding the difficulty of extracting fault information in the faulty status of UAV (unmanned aerial vehicle) engines and the high time cost and large data requirement of the existing deep learning fault diagnosis algorithms with many training parameters, in this paper, a small-sample transfer learning fault diagnosis algorithm is proposed. First, vibration signals under the engine fault status are converted into a two-dimensional time-frequency map by multiple simultaneous squeezing S-transform (MSSST), which reduces the randomness of manually extracted features. Second, to address the problems of slow network model training and large data sample requirement, a transfer diagnosis strategy using the fine-tuned time-frequency map samples as the pre-training model of the ResNet-18 convolutional neural network is proposed. In addition, in order to improve the training effect of the network model, an agent model is introduced to optimize the hyperparameter network autonomously. Finally, experiments show that the algorithm proposed in this paper can obtain high classification accuracy in fault diagnosis of UAV engines compared to other commonly used methods, with a classification accuracy of faults as high as 97.1751%; in addition, we show that it maintains a very stable small-sample migratory learning capability under this condition.

Improving Deep Learning Anomaly Diagnostics with a Physics-Based Simulation Model

Deep learning algorithms often struggle to accurately distinguish between healthy and anomalous states due to the scarcity of high-quality data in real-world applications. However, these data can be obtained through a physics-based simulation model. In this research, the model serves a dual purpose: detecting anomalies in industrial processes and replicating the machine’s operational behavior with high fidelity in terms of a simulated torque signal. When anomalous behaviors are detected, their patterns are utilized to generate anomalous events, contributing to the enhancement of deep neural network model training. This research proposes a method, named Simulation-Enhanced Anomaly Diagnostics (SEAD), to detect anomalies and further create high-quality data related to the diagnosed faults in the machine’s operation. The findings of this study suggest that employing a physics-based simulation model as a synthetic-anomaly signal generator can significantly improve the classification accuracy of identified anomalous states, thereby enhancing the deep learning model’s ability to recognize deviating behavior at an earlier stage when more high-quality data of the identified anomaly has been available for the learning process. This research measures the classification capability of a Long Short-Term Memory (LSTM) autoencoder to classify anomalous behavior in different SEAD stages. The validated results clearly demonstrate that simulated data can contribute to the LSTM autoencoder’s ability to classify anomalies in a peripheral milling machine. The SEAD method is employed to test its effectiveness in detecting and replicating a failure in the support element of the peripheral milling machine.

Renewable-Aware Frequency Scaling Approach for Energy-Efficient Deep Learning Clusters

Recently, renewable energy has emerged as an attractive means to reduce energy consumption costs for deep learning (DL) job processing in modern GPU-based clusters. In this paper, we propose a novel Renewable-Aware Frequency Scaling (RA-FS) approach for energy-efficient DL clusters. We have developed a real-time GPU core and memory frequency scaling method that finely tunes the training performance of DL jobs while maximizing renewable energy utilization. We introduce quantitative metrics: Deep Learning Job Requirement (DJR) and Deep Learning Job Completion per Slot (DJCS) to accurately evaluate the service quality of DL job processing. Additionally, we present a log-transformation technique to convert our non-convex optimization problem into a solvable one, ensuring the rigorous optimality of the derived solution. Through experiments involving deep neural network (DNN) model training jobs such as SqueezeNet, PreActResNet, and SEResNet on NVIDIA GPU devices like RTX3060, RTX3090, and RTX4090, we validate the superiority of our RA-FS approach. The experimental results show that our approach significantly improves performance requirement satisfaction by about 71% and renewable energy utilization by about 31% on average, compared to recent competitors.

Evaluating NetMHCpan performance on non-European HLA alleles not present in training data.

Bias in neural network model training datasets has been observed to decrease prediction accuracy for groups underrepresented in training data. Thus, investigating the composition of training datasets used in machine learning models with healthcare applications is vital to ensure equity. Two such machine learning models are NetMHCpan-4.1 and NetMHCIIpan-4.0, used to predict antigen binding scores to major histocompatibility complex class I and II molecules, respectively. As antigen presentation is a critical step in mounting the adaptive immune response, previous work has used these or similar predictions models in a broad array of applications, from explaining asymptomatic viral infection to cancer neoantigen prediction. However, these models have also been shown to be biased toward hydrophobic peptides, suggesting the network could also contain other sources of bias. Here, we report the composition of the networks' training datasets are heavily biased toward European Caucasian individuals and against Asian and Pacific Islander individuals. We test the ability of NetMHCpan-4.1 and NetMHCpan-4.0 to distinguish true binders from randomly generated peptides on alleles not included in the training datasets. Unexpectedly, we fail to find evidence that the disparities in training data lead to a meaningful difference in prediction quality for alleles not present in the training data. We attempt to explain this result by mapping the HLA sequence space to determine the sequence diversity of the training dataset. Furthermore, we link the residues which have the greatest impact on NetMHCpan predictions to structural features for three alleles (HLA-A*34:01, HLA-C*04:03, HLA-DRB1*12:02).

Automatic segmentation of ameloblastoma on ct images using deep learning with limited data

BackgroundAmeloblastoma, a common benign tumor found in the jaw bone, necessitates accurate localization and segmentation for effective diagnosis and treatment. However, the traditional manual segmentation method is plagued with inefficiencies and drawbacks. Hence, the implementation of an AI-based automatic segmentation approach is crucial to enhance clinical diagnosis and treatment procedures.MethodsWe collected CT images from 79 patients diagnosed with ameloblastoma and employed a deep learning neural network model for training and testing purposes. Specifically, we utilized the Mask R-CNN neural network structure and implemented image preprocessing and enhancement techniques. During the testing phase, cross-validation methods were employed for evaluation, and the experimental results were verified using an external validation set. Finally, we obtained an additional dataset comprising 200 CT images of ameloblastoma from a different dental center to evaluate the model's generalization performance.ResultsDuring extensive testing and evaluation, our model successfully demonstrated the capability to automatically segment ameloblastoma. The DICE index achieved an impressive value of 0.874. Moreover, when the IoU threshold ranged from 0.5 to 0.95, the model's AP was 0.741. For a specific IoU threshold of 0.5, the model achieved an AP of 0.914, and for another IoU threshold of 0.75, the AP was 0.826. Our validation using external data confirms the model's strong generalization performance.ConclusionIn this study, we successfully applied a neural network model based on deep learning that effectively performs automatic segmentation of ameloblastoma. The proposed method offers notable advantages in terms of efficiency, accuracy, and speed, rendering it a promising tool for clinical diagnosis and treatment.

Enhancing The Accuracy of Image Classification Using Deep Learning and Preprocessing Methods

Deep learning is one of many methods in Artificial Intelligence (AI) that computers can use to process information like text, images, and audio. This manuscript will be focusing on image preprocessing, one of the many different techniques that are used to modify the neural network model training process, and how it affects the training speed and accuracy of the neural network. Six different image preprocessing techniques were picked for use in this study: Grayscale, Smoothing, Unmask Sharpening, Laplacian and Equalization, and Random Cropping and Rotation all of which were implemented using Python and the libraries NumPy, OpenCV, and PyTorch. For the dataset, a batch of 10000 images from the CIFAR10 dataset were used to train the model. This study explored the impact of preprocessing techniques on a deep learning model, employing the RESNET50 architecture. Notable improvements in model accuracy were observed, particularly with normalization and random cropping accompanied by rotation. The efficiency gains attributed to preprocessing were highlighted, leading to a more rapid training process and significant resource savings. This research underscores the importance of thoughtful preprocessing in enhancing the performance of deep learning models, offering valuable insights for practitioners in imageclassification tasks.

Research on Supply Chain Demand Prediction Model Based on LSTM

The supply chain regards suppliers, producers, and consumers as an organic whole, unifying and coordinating the information flow, logistics, and capital flow of all members, and achieving the goal of win-win for all members in the overall operation of cross organization. Demand forecasting is an important factor driving the entire supply chain, and low error rates in forecasting are a common goal pursued by the industry. In order to improve the quality of demand forecasting, enhance the efficiency of supply chain operations, and leverage the important role of machine learning in the era of artificial intelligence, this paper conducts research based on LSTM. Firstly, this paper determines the objective function and constraints for supply chain demand forecasting; Then, this paper constructs a supply chain demand prediction model, based on the LSTM network structure, determine the network training method and model construction process; Finally, this paper conducts simulation experiments and result analysis, configure LSTM parameters, determine model performance evaluation indicators, and compare and analyze actual values with predicted values. The results indicate that the supply chain demand prediction model constructed in this article has very good performance and has promotional value in practice.

MFE-transformer: Adaptive English text named entity recognition method based on multi-feature extraction and transformer

English text named entity recognition aims to alleviate the problem of insufficient labeling data in the target domain. Existing methods usually use feature representation or model parameter sharing to realize cross-domain transfer of entity recognition capability, but there is still a lack of full utilization of structured knowledge in text sequences. Therefore, this paper proposes an adaptive English named text entity recognition method based on multi-feature extraction and transformer. Firstly, a bidirectional long term memory conditional random field entity recognition model based on BERT pre-trained language model is constructed on a generic domain dataset. In the training process, the weights of two character vectors of text words are dynamically calculated and combined, which makes the model make full use of the information in the character granularity, and the parts-of-speech information and block analysis are added as additional features. The word vectors, character-level features and additional features are spliced into the BiLSTM-CRF neural network model for training. Finally, experiments are carried out on five English datasets and specific cross-domain named entity recognition datasets respectively. The results show that the average performance of the proposed model is improved by 0.43% and 1.47% compared with the current cross-domain model, indicating that the structured knowledge in feature representation can effectively improve the entity recognition capability of the target domain.

Increasing the Effectiveness of Fake News Detection: An Educational Program for High School Students Using Interactive Neural Network Training and Collective Intelligence

This paper presents the project results designed to provide high school students with essential ICT tools to identify and counteract fake news and disinformation commonly found on the Internet, especially on platforms like X/Twitter. Additionally, it introduces an educational program that utilizes interactive neural network training and collective intelligence to combat fake news. For this project, there was developed an IT framework enabling the collective training of a specialized neural network. In order to conduct a quasi-experiment, we engaged three research groups of high school students, each containing app. 10-15 members. Through a set of comprehensive workshops, the students were trained to recognize harmful online content. After this training, students actively participated in data classification on various topics, laying the foundation for the neural network's training model. The presented results underscore the efficacy of this immersive method in imparting digital literacy and enhancing the group intelligence. Moreover, the results highlight the promising potential of machine learning in assisting youth to navigate the complex digital terrain safely and responsibly. The final phase of the conducted research involved testing the trained neural network in detecting disinformation, particularily in the topics of 5G technologies and immigration problems in Poland.