Input To Convolutional Neural Network Research Articles

An Integrated Method of Rolling Bearing Fault Diagnosis Based on Convolutional Neural Network Optimized by Sparrow Optimization Algorithm

Intending to solve the problems including poor self-adaptive ability and generalization ability of the traditional categorizing method under big data, a parameter-optimized Convolutional Neural Network (CNN) based on Sparrow Search Algorithm (SSA) is proposed in this research. Initially, the raw data regarding a series of bearing vibration signals are processed with Fast Fourier Transform (FFT) and Continuous Wavelet Transform (CWT) to attain groups of time-frequency maps. Then, Locally Linear Embedding (LLE) and linear normalization are introduced to make these maps proper for the input of CNN. Next, the preprocessed data sets are utilized as training and testing samples for CNN, and the accuracy rate of the testing is considered as the fitness of SSA, which is used to search for optimal parameter combinations for CNN by SAA. Meanwhile, the construction of the CNN is determined by experience and other previous researches. Finally, an NN-based defect diagnosis model for bearings will be constructed after the SAA has determined the appropriate parameters. The model’s accuracy rate may reach 99.4 percent after repeated testing using samples, which is significantly superior to the classic fault detection approach and the fault diagnostic method based solely on shallow networks. This experimental result demonstrates that the suggested strategy may significantly increase the model’s self-adaptive feature extraction capacity and accuracy rate, implying a higher performance in defect diagnosis in the presence of huge data.

Multiband Joint Spectrum Sensing via Covariance Matrix-Aware Convolutional Neural Network

This work investigates multiband spectrum sensing in a cognitive radio network where a multi-antenna secondary user attempts to detect several consecutive frequency bands occupied by multiple primary users. Specifically, we propose a multiband joint spectrum sensing approach based on the covariance matrix-aware convolutional neural network (CNN), in which the multiband sample covariance matrices are concatenated as the input of CNN. The proposed approach is free of model assumptions and the hidden correlation features between subbands can be learned to improve the spectrum sensing performance. Simulation results show that the proposed algorithm outperforms the state-of-the-art spectrum sensing methods in cases with and without the noise uncertainty.

SSPNet: An interpretable 3D-CNN for classification of schizophrenia using phase maps of resting-state complex-valued fMRI data.

Convolutional neural networks (CNNs) have shown promising results in classifying individuals with mental disorders such as schizophrenia using resting-state fMRI data. However, complex-valued fMRI data is rarely used since additional phase data introduces high-level noise though it is potentially useful information for the context of classification. As such, we propose to use spatial source phase (SSP) maps derived from complex-valued fMRI data as the CNN input. The SSP maps are not only less noisy, but also more sensitive to spatial activation changes caused by mental disorders than magnitude maps. We build a 3D-CNN framework with two convolutional layers (named SSPNet) to fully explore the 3D structure and voxel-level relationships from the SSP maps. Two interpretability modules, consisting of saliency map generation and gradient-weighted class activation mapping (Grad-CAM), are incorporated into the well-trained SSPNet to provide additional information helpful for understanding the output. Experimental results from classifying schizophrenia patients (SZs) and healthy controls (HCs) show that the proposed SSPNet significantly improved accuracy and AUC compared to CNN using magnitude maps extracted from either magnitude-only (by 23.4 and 23.6% for DMN) or complex-valued fMRI data (by 10.6 and 5.8% for DMN). SSPNet captured more prominent HC-SZ differences in saliency maps, and Grad-CAM localized all contributing brain regions with opposite strengths for HCs and SZs within SSP maps. These results indicate the potential of SSPNet as a sensitive tool that may be useful for the development of brain-based biomarkers of mental disorders.

Turn-to-Turn Short Circuit Fault Localization in Transformer Winding via Image Processing and Deep Learning Method

Frequency response analysis (FRA) suffers from the interpretation of results despite its potential ability to detect faults related to the power transformer windings. This article presents a technique for interpreting frequency responses, which is based on image processing and a deep learning method called graph convolutional neural network (CNN). The proposed procedure transfers frequency responses into 2-D images through a visualization technique. The resulting images are aggregated into a dataset to be used as the CNN input. The proposed technique is applied on frequency responses of two different winding models with short circuit (SC) faults. The SC faults with different intensities are applied on different sections of a simulated ladder model winding and a 20 kV winding of a 1.6 MVA distribution transformer. After determining the frequency response for each faulty case and applying the visualization technique, the precise locating of the SC faults is performed by the CNN. Then, the results are analyzed by performance evaluation metrics. At this stage, the high performance of the CNN in the use of 2-D images instead of the conventional method is observed. Finally, by testing the high impedance SC faults in different sections of the simulated winding model and applying the suggested method step by step, early detection of the SC fault is also performed in this article. It should be noted that the suggested technique, in addition to its accuracy and high detection speed, can be considered as an important step in automatic interpretation of frequency responses for online monitoring of transformers.

Reconstruction of missing spring discharge by using deep learning models with ensemble empirical mode decomposition of precipitation.

A continuous and complete spring discharge record is critical in understanding the hydrodynamic behavior of karst aquifers and the variability of freshwater resources. However, due to equipment errors, failure of observation and other reasons, missing data is a common problem for spring discharge monitoring and further hydrological investigations and data analysis. In this study, a novel approach that integrates deep learning algorithms and ensemble empirical mode decomposition (EEMD) is proposed to reconstruct the missing spring discharge data with a given local precipitation record. Using EEMD, the local precipitation data is decomposed into several intrinsic mode functions (IMFs) from high to low frequencies and a residual function, which are served as the input of convolutional neural network (CNN), long short-term memory (LSTM), and hybrid CNN-LSTM models to reconstruct the missing discharge data. Evaluation metrics, including root mean squared error (RMSE), mean absolute error (MAE), and Nash-Sutcliffe efficiency coefficient (NSE), are calculated to evaluate the reconstruction performance. The monthly spring discharge and precipitation data from March 1978 to October 2021 collected at Barton Springs in Texas are used for the validation and evaluation of newly proposed deep learning models. The results indicate that deep learning models coupled with EEMD overperform the models without EEMD and significantly improve the reconstruction results. The LSTM-EEMD model obtains the best reconstruction results among three deep learning algorithms. For models with monthly data, the missing rate affects the reconstruction performance because of the number of data samples: the best reconstruction results are achieved when the missing rate was low. If the missing rate was 50%, the reconstruction results become notably poorer. However, when the daily precipitation and discharge data are used, the models can obtain satisfactory reconstruction results with missing rate ranged from 10 to 50%.

BPCNN: Bi-Point Input for Convolutional Neural Networks in Speaker Spoofing Detection.

We propose a method, called bi-point input, for convolutional neural networks (CNNs) that handle variable-length input features (e.g., speech utterances). Feeding input features into a CNN in a mini-batch unit requires that all features in each mini-batch have the same shape. A set of variable-length features cannot be directly fed into a CNN because they commonly have different lengths. Feature segmentation is a dominant method for CNNs to handle variable-length features, where each feature is decomposed into fixed-length segments. A CNN receives one segment as an input at one time. However, a CNN can consider only the information of one segment at one time, not the entire feature. This drawback limits the amount of information available at one time and consequently results in suboptimal solutions. Our proposed method alleviates this problem by increasing the amount of information available at one time. With the proposed method, a CNN receives a pair of two segments obtained from a feature as an input at one time. Each of the two segments generally covers different time ranges and therefore has different information. We also propose various combination methods and provide a rough guidance to set a proper segment length without evaluation. We evaluate the proposed method on the spoofing detection tasks using the ASVspoof 2019 database under various conditions. The experimental results reveal that the proposed method reduces the relative equal error rate (EER) by approximately 17.2% and 43.8% on average for the logical access (LA) and physical access (PA) tasks, respectively.

Interference Quality Assessment of Speech Communication Based on Deep Learning

In this article, interferencequality assessment is of great significance to reflect the communication environment and improve speech communication performance. However, most traditional assessment approaches aimed at the degraded speech produced in the communication telephone network, but lacking of methods in extreme communication environment with ultralow SNR. Therefore, in this article, we proposed a convolutional neural network (CNN) model evaluation method based on Log-Mel spectrogram to evaluate interfered speech quality. In this method, the Mel frequency cepstrum coefficients of interference speech are converted into images, which are used as input of CNN. In order to verify the performance of this method, we collected a speech dataset in real interfered communication scenarios and finished manual annotation. Experiments are carried out on this dataset to evaluate the interference speech, and the performance of this method is compared with that of machine learning evaluation method under different features. Experimental results show that the proposed method gives the better evaluation accuracy. Compared with the previous machine learning methods, the accuracy is improved by 12.5% from 75% to 87.5%.

Membrane fouling diagnosis of membrane components based on multi-feature information fusion

CBAM-MUL-CNN (convolutional block attention module - multiple - convolutional neural networks) model based on attention mechanism is proposed to solve the problem that the membrane fouling feature extraction capability of membrane bioreactor membrane component is insufficient, which resulted in the complex structure of the membrane fouling data, so that the efficient localization and classification of membrane fouling in membrane bioreactor could not be achieved. First, the time domain and frequency domain information about the fault data is used as the input of CNN (convolutional neural networks), and the features are extracted by convolution layer. Then, the input classifier is classified by splicing the time domain and frequency domain features using the full connection layer. BN (batch normalization) layer in the model can effectively prevent the disappearance of gradients, ReLU (rectified linear uint) layer can improve the non-linear model expression ability, CBAM (convolutional block attention module) can simplify the model complexity, improve the network features expression ability, and pooling layer can improve the model fault tolerance. The comparison results show that the model has excellent comprehensive performance in the membrane fouling diagnosis experiments of series tubular membrane devices and parallel hollow fiber membrane devices, and can effectively classify and locate all membrane fouling, making the treatment of water by membrane process improve the quality of effluent while reducing energy consumption, which provides a theoretical basis for actual production.

Deep Learning for Infant Cry Recognition.

Recognizing why an infant cries is challenging as babies cannot communicate verbally with others to express their wishes or needs. This leads to difficulties for parents in identifying the needs and the health of their infants. This study used deep learning (DL) algorithms such as the convolutional neural network (CNN) and long short-term memory (LSTM) to recognize infants’ necessities such as hunger/thirst, need for a diaper change, emotional needs (e.g., need for touch/holding), and pain caused by medical treatment (e.g., injection). The classical artificial neural network (ANN) was also used for comparison. The inputs of ANN, CNN, and LSTM were the features extracted from 1607 10 s audio recordings of infants using mel-frequency cepstral coefficients (MFCC). Results showed that CNN and LSTM both provided decent performance, around 95% in accuracy, precision, and recall, in differentiating healthy and sick infants. For recognizing infants’ specific needs, CNN reached up to 60% accuracy, outperforming LSTM and ANN in almost all measures. These results could be applied as indicators for future applications to help parents understand their infant’s condition and needs.

Instantaneous Brain Strain Estimation for Automotive Head Impacts via Deep Learning.

Efficient brain strain estimation is critical for routine application of a head injury model. Lately, a convolutional neural network (CNN) has been successfully developed to estimate spatially detailed brain strains instantly and accurately in contact sports. Here, we extend its application to automotive head impacts, where impact profiles are typically more complex with longer durations. Head impact kinematics (N=458) from two public databases were used to generate augmented impacts (N=2694). They were simulated using the anisotropic Worcester Head Injury Model (WHIM) V1.0, which provided baseline elementwise peak maximum principal strain (MPS). For each augmented impact, rotational velocity (vrot) and the corresponding rotational acceleration (arot) profiles were concatenated as static images to serve as CNN input. Three training strategies were evaluated: 1) "baseline", using random initial weights; 2) "transfer learning", using weight transfer from a previous CNN model trained on head impacts drawn from contact sports; and 3) "combined training", combining previous training data from contact sports (N=5661) for training. The combined training achieved the best performances. For peak MPS, the CNN achieved a coefficient of determination (R2) of 0.932 and root mean squared error (RMSE) of 0.031 for the real-world testing dataset. It also achieved a success rate of 60.5% and 94.8% for elementwise MPS, where the linear regression slope, k, and correlation coefficient, r, between estimated and simulated MPS did not deviate from 1.0 (when identical) by more than 0.1 and 0.2, respectively. Cumulative strain damage measure (CSDM) from the CNN estimation was also highly accurate compared to those from direct simulation across a range of thresholds (R2 of 0.899-0.943 with RMSE of 0.054-0.069). Finally, the CNN achieved an average k and r of 0.98±0.12 and 0.90±0.07, respectively, for six reconstructed car crash impacts drawn from two other sources independent of the training dataset. Importantly, the CNN is able to efficiently estimate elementwise MPS with sufficient accuracy while conventional kinematic injury metrics cannot. Therefore, the CNN has the potential to supersede current kinematic injury metrics that can only approximate a global peak MPS or CSDM. The CNN technique developed here may offer enhanced utility in the design and development of head protective countermeasures, including in the automotive industry. This is the first study aimed at instantly estimating spatially detailed brain strains for automotive head impacts, which employs >8.8 thousand impact simulations generated from ~1.5 years of nonstop computations on a high-performance computing platform.

Investigating the Performance of Various Deep Neural Networks-based Approaches Designed to Identify Game Events in Gameplay Footage

Video games, in addition to representing an extremely relevant field of entertainment and market, have been widely used as a case study in artificial intelligence for representing a problem with a high degree of complexity. In such studies, the investigation of approaches that endow player agents with the ability to retrieve relevant information from game scenes stands out, since such information can be very useful to improve their learning ability. This work proposes and analyses new deep learning-based models to identify game events occurring in Super Mario Bros gameplay footage. The architecture of each model is composed of a feature extractor convolutional neural network (CNN) and a classifier neural network (NN). The extracting CNN aims to produce a feature-based representation for game scenes and submit it to the classifier, so that the latter can identify the game event present in each scene. The models differ from each other according to the following elements: the type of the CNN; the type of the NN classifier; and the type of the game scene representation at the CNN input, being either single frames, or chunks, which are n-sequential frames (in this paper 6 frames were used per chunk) grouped into a single input. The main contribution of this article is to demonstrate the greater performance reached by the models which combines the chunk representation for the game scenes with the resources of the classifier recurrent neural networks (RNN).

3D Input Convolutional Neural Network for SSVEP Classification in Design of Brain Computer Interface for Patient User.

This research was aimed at presenting performance of 3-dimensional input convolutional neural networks for steady-state visual evoked potential classification in a wireless EEG-based brain-computer interface system. Overall performance of a brain-computer interface system depends on information transfer rate. Parameters such as signal classification accuracy rate, signal stimulator structure, and user task completion time affect information transfer rate. In this study, we used 3 types of signal classification methods that are 1-dimensional, 2-dimensional, and 3-dimensional input convolutional neural network. According to online experiment with using 3-dimensional input convolutional neural network, we reached average classification accuracy rate and average information transfer rate as 93.75% and 58.35 bit/min, respectively. This both results significantly higher than the other methods that we used in experiments. Moreover, user task completion time was reduced with using 3-dimensional input convolutional neural network. Our proposed method is novel and state-of-art model for steady-state visual evoked potential classification.

Research on Pattern Recognition of Lower Limb Motion Based on Convolutional Neural Network

Accurate motion recognition is essential for assist devices such as exoskeletons to achieve human-robot communion. However, at present, the technology of lower limb motion pattern recognition still has the problems of small amount of data and low recognition accuracy. In this paper, the lower limb motion was taken as the object, and the surface electromyography (sEMG) signals of five gaits of going upstairs without weight, going downstairs without weight, going upstairs with weight, going downstairs with weight, and walking on a level surface without weight were collected. Based on the feature extraction of the sEMG signal, a convolutional neural network (CNN) with a feature set as the input is constructed, and a new lower limb motion pattern recognition method is proposed. The recognition accuracy and work feature of the proposed method are compared with several other classification and recognition methods. The experimental results show that, compared with the traditional methods, the method of using the feature set as input for CNN can better represent the features of the prediction model, and the pattern recognition accuracy is higher. The recognition accuracy of the five gaits are all greater than 96.96%, and the error rate is less than 7%, indicating that the proposed method has higher recognition accuracy. This method provides theoretical support for achieving compliant power assistance and promoting motor function rehabilitation with rehabilitation robots, power-assisted robots, and other equipment.

Joint discriminative learning and classification for monitoring flow patterns of supersonic inlet

A supersonic inlet is one of the key components in a supersonic air-breathing propulsion system and is the basis for protection control. The overall system performance can be greatly influenced by inlet flow patterns, so it's necessary to develop methods for monitoring them to ensure stable and safe operation. In this paper, time-frequency analysis and deep learning are combined to determine inlet flow patterns from the dynamic sensor signals. Continuous Wavelet Transform is first used for preliminary signal processing by converting the dynamic sensor signals into the 2-D time-frequency spectrogram, and then the spectrogram is taken as input of Convolutional Neural Network (CNN) for classification. In order to reduce the classification error, Doublet/Triplet Convolutional Neural Network combined with Discriminative Learning (DDL-CNN/TDL-CNN) are proposed that take both the cross entropy loss and discriminative learning of features into account simultaneously. The proposed methods encourage CNN to map the spectrogram into a feature space where different flow patterns become more separable. The experimental results demonstrate that DDL-CNN/TDL-CNN have better performance for monitoring the flow patterns of supersonic inlet. Compared with the baseline CNN, three multi-classification metrics of DDL-CNN/TDL-CNN are higher on most of 31 sensors, and under the statistical comparisons, the performance of DDL-CNN/TDL-CNN is also remarkably better. From the two-dimensional representations obtained by t-sne, different flow patterns become more separable for DDL-CNN/TDL-CNN after taking the constraints of discriminative learning into consideration.

Research on Community Risk Prediction Model and Management Based on Deep Learning

The influencing factors of community risk are complex. For the low accuracy of traditional prediction model, a multichannel convolutional neural network community risk prediction model is proposed by improving convolutional neural network of deep learning. First of all, in the community risk prediction model, the structure of multichannel input convolutional neural network is selected. Then, add it into the full connection layer. Subsequently, the DenseNet layer is added to establish connections between different network layers. Finally, the receptive field is improved, and the gradient disappearance is solved. Thus, the prediction accuracy of model is improved. Compared with the traditional model, the proposed multichannel convolutional neural network model has better prediction accuracy. In addition, it performs better on the three indicators, namely, correlation coefficient R , coefficient of determination R 2 , and mean square root error RMSE. Compared with the commonly used LSTM model and logic regression model, the proposed model also has certain advantages, which is more suitable for community risk prediction.

Image classification using convolutional neural network with wavelet domain inputs

Commonly used convolutional neural networks (CNNs) usually compress high-resolution input images. Although it reduces the computation requirements into a reasonable range, the downsampling operation causes information loss, which affects the accuracy of image classification. How to adopt high-resolution image inputs to improve the quality of input information and thus improve the classification accuracy without changing the overall structure of the pre-defined CNN model or increasing the model parameters is an important issue. Here, a CNN model with wavelet domain inputs is proposed to provide a solving scheme. Specifically, the proposed method applies wavelet packet transform or dual-tree complex wavelet transform to extract information from input images with higher resolutions in the image pre-processing stage. Some subband image channels are selected as the inputs of conventional CNNs where the first several convolutional layers are removed, so that the networks directly learn in the wavelet domain. Experiment results on the Caltech-256 dataset and the Describable Textures Dataset with the ResNet-50 show that the classification accuracy of our method can have a maximum improvement of 2.15% and 10.26%, respectively. These validate the effectiveness of our proposed scheme. This code is publicly available at https://github.com/BeBeBerr/wavelet-cnn.

Inferring safety critical events from vehicle kinematics in naturalistic driving environment: Application of deep learning Algorithms

Advances in sensing technology has enabled the collection of countless terabytes of second-by-second kinematics data. Such data provides opportunities for real-time monitoring of driving behavior and identification of safety critical events (SCEs) including crashes and near crashes. The concept of volatility is relevant in this context, which identifies instability and erratic variations in driving behavior prior to involvement in SCEs. This study utilized vehicle kinematics from a large-scale naturalistic driving data to develop a deep learning approach based on 1D convolutional neural networks (CNN) for inferring SCEs. The data are unique in the sense that such accurate pre-crash data at high fidelity are not available in traditional crash repositories. This study contributes to the literature by providing a first attempt at predicting responses to SCEs by developing deep learning-based CNN architectures using novel driving volatility based kinematic thresholds for a sample of 9553 events. The key contribution lies in developing a volatility-based CNN input layout that is acceptable to CNN schemes and represents the motion kinematics such as speed, acceleration and volatility measures. Several 1D-CNN architectures were developed using layers, numbers of convolutions, layer patterns, and kernels. Shallow and deep architectures were tested, revealing higher accuracy of shallow architectures in detecting SCEs. The optimal number of epochs were identified using an early stopping method while the CNN performance was improved by increasing the number of epochs. The ensemble CNN had the highest predictive accuracy of 95.6% for detection of crashes and near crashes, which was 2.5% higher than the optimal CNN using 20% hold out test data. The ensemble CNN also outperformed classical machine learning models and model performance reported in past studies on detection of SCEs. These results have implications for identification of safety hotspots and providing real-time alerts and warnings in connected and highly automated vehicle environment including society of automotive engineers levels 3–5.

Fault diagnosis of power grid based on variational mode decomposition and convolutional neural network

The distribution network has complex topological structure and many branches. So, the fault location is easy to be wrongly located. Therefore, a novel hybrid method of combining variational mode decomposition (VMD) and convolutional neural network (CNN) for fault location and fault type identification is proposed in power grid. A fault feature extraction method based on VMD and Hilbert-Huang transform (HHT) is designed. In this method, the VMD is used to analyze the characteristic features from fault transient signals of the positive sequence current. The fault features of the intrinsic mode function with more fault features are extracted through HHT. The extracted fault feature vector is used as the input of CNN to build fault diagnosis model. Finally, the fault diagnosis report is obtained by comparing and analyzing the output results of SoftMax layer. The experimental results show that this method can identify the fault location and type in the small current grounding power system model of relay protection dynamic simulation equipment. Meanwhile, the method is less influenced by fault resistance and fault distance and has good accuracy, thus having better accuracy and generalization ability than traditional methods.

Deep learning of deformation-dependent conductance in thin films: Nanobubbles in graphene

Motivated by the ever-improving performance of deep learning techniques, we design a mixed input convolutional neural network approach to predict transport properties in deformed nanoscale materials using a height map of deformations (from scanning probe information) as input. We employ our approach to study electrical transport in a graphene nanoribbon deformed by a number of randomly positioned nano-bubbles. Our network is able to make conductance predictions valid to an average error of 4.3\%. We demonstrate that such low average errors are achieved by including additional inputs like energy in a highly redundant fashion, which allows predictions that are 30-40\% more accurate than conventional architectures. We demonstrate that the same method can learn to predict the valley-resolved conductance, with success specifically in identifying the energy at which inter-valley scattering becomes prominent. We demonstrate the robustness of the approach by testing the pre-trained network on samples with deformations differing in number and shape from the training data. We employ a graph theoretical analysis of the structure and outputs of the network and conclude that a tight-binding Hamiltonian is effectively encoded in the first layer of the network. We confirm our graph theoretical analysis numerically for different hopping processes in a trained network and find the result to be accurate within an error of 1\%. Our approach contributes a new theoretical understanding and a refined methodology to the application of deep learning for the determination transport properties based on real-space disorder information.

Scenario Based Run-Time Switching for Adaptive CNN-Based Applications at the Edge

Convolutional Neural Networks (CNNs) are biologically inspired computational models that are at the heart of many modern computer vision and natural language processing applications. Some of the CNN-based applications are executed on mobile and embedded devices. Execution of CNNs on such devices places numerous demands on the CNNs, such as high accuracy, high throughput, low memory cost, and low energy consumption. These requirements are very difficult to satisfy at the same time, so CNN execution at the edge typically involves trade-offs (e.g., high CNN throughput is achieved at the cost of decreased CNN accuracy). In existing methodologies, such trade-offs are either chosen once and remain unchanged during a CNN-based application execution, or are adapted to the properties of the CNN input data. However, the application needs can also be significantly affected by the changes in the application environment, such as a change of the battery level in the edge device. Thus, CNN-based applications need a mechanism that allows to dynamically adapt their characteristics to the changes in the application environment at run-time. Therefore, in this article, we propose a scenario-based run-time switching (SBRS) methodology, that implements such a mechanism.