Two-stream Convolutional Neural Network Model Research Articles

Bolt-Loosening Detection Using 1D and 2D Input Data Based on Two-Stream Convolutional Neural Networks.

At present, the detection accuracy of bolt-loosening diagnoses is still not high. In order to improve the detection accuracy, this paper proposes a fault diagnosis model based on the TSCNN model, which can simultaneously extract fault features from vibration signals and time-frequency images and can precisely detect the bolt-loosening states. In this paper, the LeNet-5 network is improved by adjusting the size and number of the convolution kernels, introducing the dropout operation, and building a two-dimensional convolutional neural network (2DCNN) model. Combining the advantages of a one-dimensional convolutional neural network (1DCNN) with wide first-layer kernels to suppress high-frequency noise, a two-stream convolutional neural network (TSCNN) is proposed based on 1D and 2D input data. The proposed model uses raw vibration signals and time-frequency images as input and automatically extracts sensitive features and representative information. Finally, the effectiveness and superiority of the proposed approach are verified by practical experiments that are carried out on a machine tool guideway. The experimental results show that the proposed approach can effectively achieve end-to-end bolt-loosening fault diagnoses, with an average recognition accuracy of 99.58%. In addition, the method can easily achieve over 93% accuracy when the SNR is over 0 dB without any denoising preprocessing. The results show that the proposed approach not only achieves high classification accuracy but also has good noise immunity.

Improved human action recognition approach based on two-stream convolutional neural network model

In order to improve the accuracy of human abnormal behavior recognition, a two-stream convolution neural network model was proposed. This model includes two main parts, VMHI and FRGB. Firstly, the motion history images are extracted and input into VGG-16 convolutional neural network for training. Then, the RGB image is input into Faster R-CNN algorithm for training using Kalman filter-assisted data annotation. Finally, the two stream VMHI and FRGB results are fused. The algorithm can recognize not only single person behavior, but also two person interaction behavior and improve the recognition accuracy of similar actions. Experimental results on KTH, Weizmann, UT-interaction, and TenthLab dataset showed that the proposed algorithm has higher accuracy than the other literature.

Driver Behavior Analysis via Two-Stream Deep Convolutional Neural Network

According to the World Health Organization global status report on road safety, traffic accidents are the eighth leading cause of death in the world, and nearly one-fifth of the traffic accidents were cause by driver distractions. Inspired by the famous two-stream convolutional neural network (CNN) model, we propose a driver behavior analysis system using one spatial stream ConvNet to extract the spatial features and one temporal stream ConvNet to capture the driver’s motion information. Instead of using three-dimensional (3D) ConvNet, which would suffer from large parameters and the lack of a pre-trained model, two-dimensional (2D) ConvNet is used to construct the spatial and temporal ConvNet streams, and they were pre-trained by the large-scale ImageNet. In addition, in order to integrate different modalities, the feature-level fusion methodology was applied, and a fusion network was designed to integrate the spatial and temporal features for further classification. Moreover, a self-compiled dataset of 10 actions in the vehicle was established. According to the experimental results, the proposed system can increase the accuracy rate by nearly 30% compared to the two-stream CNN model with a score-level fusion.

Two-Stream Convolutional Neural Network Based on Gradient Image for Aluminum Profile Surface Defects Classification and Recognition

In this article, a novel two-stream convolutional neural network based on gradient image is performed to effectively classify and identify aluminum profiles defects for the first time. Recent feature fusion methods based on two-stream network prove promising performance for defects classification and recognition. In this article, we use data enhancement methods to obtain a large number of samples to prevent the over fitting phenomenon in deep learning. The image gradient is calculated with the Sobel operator, and normalized to transform the data between zero and one under the same dimension. We design a two-stream convolutional neural network model adopting Wavelet transform fusion strategy to realize feature fusion on the ReLU6 layer, which uses the original RGB image of aluminum profile and the gradient image corresponding to the original RGB image as inputs to extract features through two sub-networks and fuses features on a concatenate layer to be input into SVM classifier for classification and recognition. Using Bayesian Optimization function and computing the cross-validation classification error to optimize the hyperparameters to choose the best performance configuration is performed. A series of experimental data, which include accuracy and estimated generalized classification errors of single-stream and two-stream networks with different feature fusion strategies on different fusion layers, are conducted and show that the current model has good convergence, accuracy, stability and generalization. On this basis, this article also proposes a series of innovative methods for the future research of other defects.