Neural Network Research Articles

Deep Learning-Based Acoustic Emission Signal Filtration Model in Reinforced Concrete

AbstractAcoustic emission is a nondestructive testing (NDT) technique, widely used to monitor the condition of structures for safety reasons especially in real time. The method utilizes the electrical signals generated by the elastic waves in a material under load to detect and locate damage in structures. However, identifying the sources of AE signals in concrete or composite materials can be challenging due to the anisotropic properties of materials and interpreting a large amount of AE data, leading to data misinterpretation and inaccurate detection of damage. Hence, the need for filtering out noise-induced signals from recorded data and emphasizing the actual AE source is crucial for monitoring and source localization of damage in real time. This study proposed a one-dimensional convolutional neural network (1D-CNN) deep learning approach to filter around 22,000 AE data in a reinforced concrete (RC) beam. The model utilizes significant AE parameters identified through neighborhood component analysis (NCA) to classify true AE signals from noise-induced signals. By using the optimized network parameters, a high classification accuracy of 97% and 96.29% was achieved during the training and testing phases, respectively. To check the reliability of the proposed AE filtering model in the real world, it was evaluated and verified using source location AE activities collected during a four-point bending test on a shear-deficient beam. The outcomes suggest that the proposed AE filtration model has the potential to accurately classify AE signals with an accuracy of 92.8% and proved that the filtration model provides accurate and valuable insight into source location determination.

A Detection Transformer-Based Intelligent Identification Method for Multiple Types of Road Traffic Safety Facilities

Road traffic safety facilities (TSFs) are of significant importance in the management and maintenance of traffic safety. The complexity and variety of TSFs make it challenging to detect them manually, which renders the work unsustainable. To achieve the objective of automatic TSF detection, a target detection dataset, designated TSF-CQU (TSF data collected by Chongqing University), was constructed based on images collected by a car recorder. This dataset comprises six types of TSFs and 8410 instance samples. A detection transformer with an improved denoising anchor box (DINO) was selected to construct a model that would be suitable for this scenario. For comparison purposes, Faster R-CNN (Region Convolutional Neural Network) and Yolov7 (You Only Look Once version 7) were employed. The DINO model demonstrated the highest performance on the TSF-CQU dataset, with a mean average precision (mAP) of 82.2%. All of the average precision (AP) values exceeded 0.8, except for streetlights (AP = 0.77) and rods (AP = 0.648). The DINO model exhibits minimal instances of erroneous recognition, which substantiates the efficacy of the contrastive denoising training approach. The DINO model rarely makes misjudgments, but a few missed detection.

Self-supervised learning of accelerometer data provides new insights for sleep and its association with mortality

Sleep is essential to life. Accurate measurement and classification of sleep/wake and sleep stages is important in clinical studies for sleep disorder diagnoses and in the interpretation of data from consumer devices for monitoring physical and mental well-being. Existing non-polysomnography sleep classification techniques mainly rely on heuristic methods developed in relatively small cohorts. Thus, we aimed to establish the accuracy of wrist-worn accelerometers for sleep stage classification and subsequently describe the association between sleep duration and efficiency (proportion of total time asleep when in bed) with mortality outcomes. We developed a self-supervised deep neural network for sleep stage classification using concurrent laboratory-based polysomnography and accelerometry. After exclusion, 1448 participant nights of data were used for training. The difference between polysomnography and the model classifications on the external validation was 34.7 min (95% limits of agreement (LoA): −37.8–107.2 min) for total sleep duration, 2.6 min for REM duration (95% LoA: −68.4–73.4 min) and 32.1 min (95% LoA: −54.4–118.5 min) for NREM duration. The sleep classifier was deployed in the UK Biobank with 100,000 participants to study the association of sleep duration and sleep efficiency with all-cause mortality. Among 66,214 UK Biobank participants, 1642 mortality events were observed. Short sleepers (<6 h) had a higher risk of mortality compared to participants with normal sleep duration of 6–7.9 h, regardless of whether they had low sleep efficiency (Hazard ratios (HRs): 1.58; 95% confidence intervals (CIs): 1.19–2.11) or high sleep efficiency (HRs: 1.45; 95% CIs: 1.16–1.81). Deep-learning-based sleep classification using accelerometers has a fair to moderate agreement with polysomnography. Our findings suggest that having short overnight sleep confers mortality risk irrespective of sleep continuity.

Super resolution reconstruction of background oriented schlieren displacement field based on the convolution neural network with the gradient loss function

Abstract In order to refine the displacement field of the background oriented schlieren method, a super-resolution method based on deep learning has been newly proposed in the study, and were compared with the bicubic interpolation. The gradient loss functions were firstly introduced into the hybrid downsampled skip-connection/multi-scale (DSC/MS) model to improve the reconstruction effect. The reconstruction effects of the new loss functions were compared with that of the traditional MSE loss function. The results showed that the Laplace operator with average pooling had better performance than the origin loss function in all the indexes including PSNR, MSE, MSE of the gradient and the max MSE. In these four indexes, MSE of the gradient and the max MSE performed especially better than the others, where the MSE of the gradient reduced from 3.90E-05 to 3.30E-05 and the max MSE reduced from 0.392 to 0.360.&amp;#xD;

Time division multiplexing based multi-spectral semantic camera for LiDAR applications

The recent progress in the development of measurement systems for autonomous recognition had a substantial impact on emerging technology in numerous fields, especially robotics and automotive applications. In particular, time-of-flight (TOF) based light detection and ranging (LiDAR) systems enable to map the surrounding environmental information over long distances and with high accuracy. The combination of advanced LiDAR with an artificial intelligence platform allows enhanced object recognition and classification, which however still suffers from limitations of inaccuracy and misidentification. Recently, multi-spectral LiDAR systems have been employed to increase the object recognition performance by additionally providing material information in the short-wave infrared (SWIR) range where the reflection spectrum characteristics are typically very sensitive to material properties. However, previous multi-spectral LiDAR systems utilized band-pass filters or complex dispersive optical systems and even required multiple photodetectors, adding complexity and cost. In this work, we propose a time-division-multiplexing (TDM) based multi-spectral LiDAR system for semantic object inference by the simultaneous acquisition of spatial and spectral information. By utilizing the TDM method, we enable the simultaneous acquisition of spatial and spectral information as well as a TOF based distance map with minimized optical loss using only a single photodetector. Our LiDAR system utilizes nanosecond pulses of five different wavelengths in the SWIR range to acquire sufficient material information in addition to 3D spatial information. To demonstrate the recognition performance, we map the multi-spectral image from a human hand, a mannequin hand, a fabric gloved hand, a nitrile gloved hand, and a printed human hand onto an RGB-color encoded image, which clearly visualizes spectral differences as RGB color depending on the material while having a similar shape. Additionally, the classification performance of the multi-spectral image is demonstrated with a convolution neural network (CNN) model using the full multi-spectral data set. Our work presents a compact novel spectroscopic LiDAR system, which provides increased recognition performance and thus a great potential to improve safety and reliability in autonomous driving.

Height Prediction of Water-Conducting Fracture Zone in Jurassic Coalfield of Ordos Basin Based on Improved Radial Movement Optimization Algorithm Back-Propagation Neural Network

The development height of the water-conducting fracture zone (WCFZ) is crucial for the safe production of coal mines. The back-propagation neural network (BP-NN) can be utilized to forecast the WCFZ height, aiding coal mines in water hazard prevention and control efforts. However, the stochastic generation of initial weights and thresholds in BP-NN usually leads to local optima, which might reduce the prediction accuracy. This study thus invokes the excellent global optimization capability of the Improved Radial Movement Optimization (IRMO) algorithm to optimize BP-NN. The influences of mining thickness, coal seam depth, working width, and hard rock lithology proportion coefficient on the height of WCFZ are investigated through 75 groups of in situ data of WCFZ heights measured in the Jurassic coalfield of the Ordos Basin. Consequently, an IRMO-BP-NN model for predicting WCFZ height in the Jurassic coalfield of the Ordos Basin was constructed. The proposed IRMO-BP-NN model was validated through monitoring data from the 4−2216 working faces of Jianbei Coal Mine, followed by a comparative analysis with empirical formulas and conventional BP-NN models. The relative error of the IRMO-BP-NN prediction model is 4.93%, outperforming both the BP-NN prediction model, the SVR prediction model, and empirical formulas. The results demonstrate that the IRMO-BP-NN model enhances the accuracy of predicting WCFZ height, providing an application foundation for predicting such heights in the Jurassic coalfield of the Ordos Basin and protecting the ecological environment of Ordos Basin mining areas.

Machine learning approaches for simultaneous spectrophotometric determination of heavy metal ions in water samples

In this study, the simultaneous determination of Co, Cd, Ni, Cu, and Pb was carried out as a color complex with 4-(2-pyridylazo) resorcinol in an aqueous solution under the assesting of machine learning. A partial least-squares multivariate linear regression and artificial neuron network for the analysis of mixtures of metals were developed. MATLAB is a powerful software machine learning program that was used to support matrix calculations and displays. The benefit of MATLAB in the construction of the machine learning model allows the development of a rapid and highly effective analysis of multiple components in the mixtures without separation and enrichment. For individual determinations, the working ranges were discovered as the important information for choosing the initial concentration of each heavy metal in a mixture, r. The results of analysis of Ni2+, Pb2+, and Cd2+ by two methods Partial Least Squares - PLS and Artificial Neural Networks - ANN are sensitive and accurate for simultaneous determination of the concentration of these ions in the synthesis mixture with a high regression coefficient of 0.993, respectively, 0.997, 0.997 for Ni2+, Pb2+ and Cd2+. As for Cu2+ and Co2+, the accuracy is higher when using the ANN method.

STFEformer: Spatial–Temporal Fusion Embedding Transformer for Traffic Flow Prediction

In the realm of Intelligent Transportation Systems (ITSs), traffic flow prediction is crucial for multiple applications. The primary challenge in traffic flow prediction lies in the handling and modeling of the intricate spatial–temporal correlations inherent in transport data. In recent years, many studies have focused on developing various Spatial–Temporal Graph Neural Networks (STGNNs), and researchers have also begun to explore the application of transformers to capture spatial–temporal correlations in traffic data. However, GNN-based methods mainly focus on modeling spatial correlations statically, which significantly limits their capacity to discover dynamic and long-range spatial patterns. Transformer-based methods have not sufficiently extracted the comprehensive representation of traffic data features. To explore dynamic spatial dependencies and comprehensively characterize traffic data, the Spatial–Temporal Fusion Embedding Transformer (STFEformer) is proposed for traffic flow prediction. Specifically, we propose a fusion embedding layer to capture and fuse both native information and spatial–temporal features, aiming to achieve a comprehensive representation of traffic data characteristics. Then, we introduce a spatial self-attention module designed to enhance detection of dynamic and long-range spatial correlations by focusing on interactions between similar nodes. Extensive experiments conducted on three real-world datasets demonstrate that STFEformer significantly outperforms various baseline models, notably achieving up to a 5.6% reduction in Mean Absolute Error (MAE) on the PeMS08 dataset compared to the next-best model. Furthermore, the results of ablation experiments and visualizations are employed to clarify and highlight our model’s performance. STFEformer represents a meaningful advancement in traffic flow prediction, potentially influencing future research and applications in ITSs by providing a more robust framework for managing and analyzing traffic data.

Distributed training of CosPlace for large-scale visual place recognition

Visual place recognition (VPR) is a popular computer vision task aimed at recognizing the geographic location of a visual query, usually within a tolerance of a few meters. Modern approaches address VPR from an image retrieval standpoint using a kNN on top of embeddings extracted by a deep neural network from both the query and images in a database. Although most of these approaches rely on contrastive learning, which limits their ability to be trained on large-scale datasets (due to mining), the recently reported CosPlace proposes an alternative training paradigm using a classification task as the proxy. This has been shown to be effective in expanding the potential of VPR models to learn from large-scale and fine-grained datasets. In this work, we experimentally analyze CosPlace from a continual learning perspective and show that its sequential training procedure leads to suboptimal results. As a solution, we propose a different formulation that not only solves the pitfalls of the original training strategy effectively but also enables faster and more efficient distributed training. Finally, we discuss the open challenges in further speeding up large-scale image retrieval for VPR.

Medical history predicts phenome-wide disease onset and enables the rapid response to emerging health threats

The COVID-19 pandemic exposed a global deficiency of systematic, data-driven guidance to identify high-risk individuals. Here, we illustrate the utility of routinely recorded medical history to predict the risk for 1883 diseases across clinical specialties and support the rapid response to emerging health threats such as COVID-19. We developed a neural network to learn from health records of 502,460 UK Biobank. Importantly, we observed discriminative improvements over basic demographic predictors for 1774 (94.3%) endpoints. After transferring the unmodified risk models to the All of US cohort, we replicated these improvements for 1347 (89.8%) of 1500 investigated endpoints, demonstrating generalizability across healthcare systems and historically underrepresented groups. Ultimately, we showed how this approach could have been used to identify individuals vulnerable to severe COVID-19. Our study demonstrates the potential of medical history to support guidance for emerging pandemics by systematically estimating risk for thousands of diseases at once at minimal cost.

Separate-layer injection scheme optimization based on integrated injection information with artificial neural network and residual network

Abstract Separate-layer injection technology is a highly significant approach for enhancing oil recovery in the later stages of oilfield production. Both separate-layer and general injection information are crucial parameters in multi-layer oilfield injection systems. However, the significance of general injection information is usually overlooked during the optimization process of separate-layer injection. Moreover, conventional optimization schemes for separate-layer injection fails to meet the immediate and dynamic demands of well production. Consequently, a separate-layer injection optimization method based on ANN-Res model was proposed. Firstly, the primary controlling factors for production were identified through grey correlation analysis and ablation experiments. Then, a data-driven model was established with artificial neural network (ANN), in which the residual block was utilized to incorporate general injection information, eventually formed an ANN-Res model that integrates separate-layer and general injection information. Finally, a workflow for separate-layer injection optimization was designed in association with the ANN-Res model. Analysis of primary controlling factor for production shows that the combination of separate-layer and general injection information for production prediction leads to redundancy. The results of injection-production prediction demonstrate that the ANN-Res model is significantly better than that of the ANN model which only inputs separate-layer or general injection information. Furthermore, the result of optimization proves the proposed method can be successfully applied to injection optimization, realizing the purpose of increasing oil production and decreasing water cut, thereby improving oilfield development.

A real-time multiple tunneling parameter prediction method of TBM steady phase based on dual recurrent neural networks

A real-time multiple tunneling parameter prediction method of TBM steady phase based on dual recurrent neural networks

Comparative Study of Keras CNNs for Tuberculosis Detection from Chest X-rays

As a major global health burden, tuberculosis (TB) requires prompt and accurate diagnostic solutions. The effectiveness of Convolutional Neural Networks (CNNs) for TB prediction from chest X-ray images is investigated in this work using a variety of Keras applications. We compare the accuracy, efficiency, and computational resource usage of the VGG16, ResNet50, and EfficientNetV2B2 architectures. By means of comprehensive testing and analysis on a wide range of datasets, we determine the applicability of every model for tuberculosis identification. The findings show that EfficientNetV2B2 is the most promising architecture, achieving remarkable accuracy (99.5%) with enhanced computational efficiency, while VGG16 and ResNet50 show competitive performance. These results highlight how deep learning-based methods have the potential to transform tuberculosis diagnosis by providing doctors with a trustworthy and usable instrument for early detection and treatment. Our research has implications for improving healthcare outcomes and bolstering international efforts to fight tuberculosis. Keywords—EfficientNet, CNN, VGG16,Keras, Accuracy

Comparative Analysis of CNN Architectures for SIBI Image Classification

The classification of images from the Indonesian Sign Language System (SIBI) using VGG16, ResNet50, Inception, Xception, and MobileNetV2 Convolutional Neural Network (CNN) architectures is evaluated in this paper. With Google Colab Pro, a 224 × 224-pixel picture dataset was used for the study. A five-stage technique consisting of Dataset Collection, Dataset Preprocessing, Model Design, Model Training, and Model Testing was applied. Performance evaluation focused on accuracy, precision, recall, and F1-Score. The results identified VGG16 as the top-performing model with an accuracy of 99.60% and an equivalent F1-Score, followed closely by ResNet50 with nearly similar performance. Inception, XCeption, and MobileNetV2 demonstrated balanced performance but with lower accuracy. This study sheds light on the best CNN models to choose for SIBI image classification, and it makes recommendations for further research that include using sophisticated data augmentation methods, investigating novel CNN architectures, and putting the models to practical use.

Sampling the user controls in neural modeling of audio devices

This work studies neural modeling of nonlinear parametric audio circuits, focusing on how the diversity of settings of the target device user controls seen during training affects network generalization. To study the problem, a large corpus of training datasets is synthetically generated using SPICE simulations of two distinct devices, an analog equalizer and an analog distortion pedal. A proven recurrent neural network architecture is trained using each dataset. The difference in the datasets is in the sampling resolution of the device user controls and in their overall size. Based on objective and subjective evaluation of the trained models, a sampling resolution of five for the device parameters is found to be sufficient to capture the behavior of the target systems for the types of devices considered during the study. This result is desirable, since a dense sampling grid can be impractical to realize in the general case when no automated way of setting the device parameters is available, while collecting large amounts of data using a sparse grid only incurs small additional costs. Thus, the result provides guidance for efficient collection of training data for neural modeling of other similar audio devices.

Terahertz Nondestructive Measurement of Heat Radiation Performance of Thermal Barrier Coatings Based on Hybrid Artificial Neural Network

Effective control of the micro- and nanostructure of thermal barrier coatings is essential to enhance the thermal radiation performance of the coating, which helps to determine the remaining service life of the coating. This paper proposed a method to measure the radiation properties of thermal barrier coatings by terahertz nondestructive testing technique, using APS-prepared thermal barrier coatings as the object of study. Radiative properties were a comprehensive set of properties characterized by the diffuse reflectance, transmittance, and absorptance of the thermal barrier coating. The coating data in actual service were obtained by scanning electron microscopy and metallographic experiments, and the data were used as the simulation model critical value. The terahertz time-domain simulation data of coatings with different microstructural features were obtained using the finite-different time-domain (FDTD) method. In simulating the real test signals, white noise with a signal-to-noise ratio of 20 dB was added, and fast Fourier transform (FFT), short-time Fourier transform (STFT), and wavelet transform (WT) were used to reduce the noise and compare their noise reduction effects. Different machine learning methods were used to build the model, including support vector machine algorithm (SVM) and k-nearest neighbor algorithm (KNN). The principal component algorithm (PCA) was used to reduce the dimensionality of terahertz time-domain data, and the SVM algorithm and KNN algorithm were optimized using the particle swarm optimization algorithm (PSO) and the ant colony optimization algorithm (ACO), respectively, to improve the robustness of the system. The K-fold cross-validation method was used to construct the model to improve the adaptability of the model. It could be clearly seen that the novel hybrid PCA-ACO-SVM model had superior prediction performance. Finally, this work proposed a novel, convenient, nondestructive, online, safe and highly accurate method for measuring the radiation performance of thermal barrier coatings, which could be used for the judgment of the service life of thermal barrier coatings.

Improving the Concrete Crack Detection Process via a Hybrid Visual Transformer Algorithm

Inspections of concrete bridges across the United States represent a significant commitment of resources, given their biannual mandate for many structures. With a notable number of aging bridges, there is an imperative need to enhance the efficiency of these inspections. This study harnessed the power of computer vision to streamline the inspection process. Our experiment examined the efficacy of a state-of-the-art Visual Transformer (ViT) model combined with distinct image enhancement detector algorithms. We benchmarked against a deep learning Convolutional Neural Network (CNN) model. These models were applied to over 20,000 high-quality images from the Concrete Images for Classification dataset. Traditional crack detection methods often fall short due to their heavy reliance on time and resources. This research pioneers bridge inspection by integrating ViT with diverse image enhancement detectors, significantly improving concrete crack detection accuracy. Notably, a custom-built CNN achieves over 99% accuracy with substantially lower training time than ViT, making it an efficient solution for enhancing safety and resource conservation in infrastructure management. These advancements enhance safety by enabling reliable detection and timely maintenance, but they also align with Industry 4.0 objectives, automating manual inspections, reducing costs, and advancing technological integration in public infrastructure management.

Probabilistic deep learning approach for fatigue crack width estimation and prognosis in lap joint using acoustic waves

Abstract Accurate fatigue crack width estimation is crucial for aircraft safety, however, limited research exists on (i) the direct relationship between fatigue crack width and Lamb wave signatures and (ii) probabilistic artificial intelligence (AI) approach for automated analysis using acoustic emission waveforms. This paper presents a probabilistic deep learning approach for fatigue crack width estimation, employing an automated wavelet feature extractor and probabilistic Bayesian neural network. A dataset constituting the fatigue experiment on aluminum lap-joint specimens is considered, in which Lamb wave signals were recorded at several time instants for each specimen. Signals acquired from the piezo actuator-receiver sensor pairs are related to the optically measured surface crack length. The sensitive features are automatically extracted from the signals using decomposition techniques called maximal overlap discrete wavelet transform (MODWT). The extracted features are then mapped through the deep learning model, which incorporates Bayesian inference to account for both aleatoric as well as epistemic uncertainty, that provides outcomes in the form of providing probabilistic estimates of crack width with uncertainty quantification. Thus, employing an automated wavelet feature extractor (MODWT) on a dataset of fatigue experiments, the framework learns the relationship between Lamb wave signals and crack width. Validation on unseen in-situ data demonstrates the efficacy of the approach for practical implementation, paving the way for more reliable fatigue life prognosis.

Analyzing time series to forecast hot rolled coil steel price in Spain by means of neural non-linear models

Abstract In the industrial context, steel is a broadly-used raw material with applications in many different fields. Due to its high impact in the activity of many industries all over the world, forecasting its price is of utmost importance for a huge amount of companies. In this work, non-linear neural models are applied for the first time to different datasets in order to validate their suitability when predicting the price of this commodity. In particular, the NAR, NIO and NARX neural network models are innovatively applied for the first time to forecast the price of hot rolled steel in Spain. Besides these variety of models, different datasets consisting of a set of heterogenous variables from the last seven years and related to the price of this commodity are benchmarked and analyzed. The results showed that NARX is the best performing model when the price of raw materials used to produce steel and the stock market prices of three major global steel producing companies are employed as input to this predictive model. Consequently, this result may boost the application of Machine Learning in companies, in order to schedule the supplying operations according to the price forecasting.

An evolutionary gravitational neocognitron neural network for earthquake parameters observation in iot system-based earthquake early warning

An evolutionary gravitational neocognitron neural network for earthquake parameters observation in iot system-based earthquake early warning