Two-channel Convolutional Neural Network Research Articles

Selection of Distance Measure for Visual and Long Wave Infrared Image Region Similarity using CNN Features

Similarity computation between images or image regions is a necessary precursor for several vision-based applications, such as retrieval, registration, change detection etc. A two-channel convolutional neural network architecture is designed to retrieve an appropriate visual (VS) image representative from the repository, given as query a long-wave infrared (LWIR) image patch of the same region. Both the VS and LWIR image regions are described using pretrained convolution neural network models and images are ranked by computing the dis/similarity between the feature vectors. It is essential to evaluate and identify the suitable combination of feature descriptor and distance measure, when applied to LWIR and visual image similarity, as pre-trained CNNs such as VGG16, VGG19, MobileNet are not trained for LWIR images. RoadScene dataset which contains a pair of aligned long-wave infrared and visible images is used and performance of CNN features and distance measures is objectively evaluated using computation time, number of patches in top 5 and also in computing how close the retrieved LWIR patch is to the visual patch. Results demonstrate that Cosine similarity measure is relatively better when compared to all the other distance measures in addressing the spectral variations.

Localizing False Data Injection Attacks in Smart Grid: A Spectrum-Based Neural Network Approach

Smart grid is confronted with cyberattacks due to the increasing dependence on cyberspace. False data injection attacks (FDIAs) represent a major type of cyberattacks that cannot be detected by the traditional bad data detection (BDD). The majority of existing researches focus on how to detect FDIAs, while little attention has been paid to obtaining the exact attack locations, which are crucial for deploying countermeasures in time. In this paper, we propose a neural network-based approach for online localizing FDIAs in AC power systems. The approach transforms the multi-dimensional time series measurements to their time-frequency and spectral graph representations, to jointly capture the temporal and spatial correlations in the spectral domain. The spectral decomposition could better characterize the latent properties of measurements, such as auto-correlations of a bus and key dependency between buses, which would be poorly represented in the time domain and vertex domain. Specifically, the proposed approach first adopts the short-time Fourier transform (STFT) and a two-channel convolutional neural network (2C-CNN) to model the spectral-temporal correlations. Then the spectral-spatial relationships are modeled with a graph convolutional network (GCN), which combines the nodal admittance matrix and physical property of power systems. The prior physical knowledge further improves our approach’s interpretability and localization performance. We evaluate the effectiveness of our proposed approach with comprehensive case studies on various standard test systems. Numerical results indicate the superiority of our approach over baselines.

Analysis of bridge foundation pile detection based on convolutional neural network model

Abstract In order to quickly and accurately detect the instability of foundation piles on concrete Bridges, an improved convolutional neural network based image recognition method for concrete bridge pile foundations was proposed. In order to improve the image quality, the entropy threshold method is used to process the image, and the two-channel convolutional neural network is designed to extract the image features fully. The improved traditional Relu activation function avoids model underfitting. Support vector machine (SVM) was used to replace Softmax classifier to improve computing efficiency. The experiment of bridge pile foundation image recognition shows that the improved convolutional neural network has significantly improved the recognition rate of the real bridge pile foundation. No matter how many degrees the image is rotated, the method in this article always maintains a high recognition rate, and the recognition rate does not fluctuate much, indicating that the algorithm in this article has good robustness to rotation and translation. In summary, the average recognition rate of the 5 groups was 96.26%. The feasibility of this method in identifying bridge pile foundation is proved.

A presentation attack detection network based on dynamic convolution and multi-level feature fusion with security and reliability

In the Internet of Things environment, most facial recognition systems are highly susceptible to face spoofing attacks. To increase the security and reliability of such systems, this paper proposes an anti-spoofing method for facial recognition systems based on optical flow and texture features. The spoofing-detection algorithm first generates the optical flow field map of the face area using the optical flow method and a face-detection method, based on two consecutive frames of the captured face video. Then, the original RGB face area image and optical flow field map are input into a two-channel convolutional neural network to extract and fuse the features of the face. Finally, based on the optical flow and texture features, this method classifies real and fake faces. In addition, to better generate the optical flow field map containing liveness information, a motion amplification algorithm is applied to enhance the 0.04–0.4 Hz signal in the video frame by 20 times. For texture and optical flow presentation, we propose a lightweight network with dynamic convolution and multi-level feature fusion blocks. This paper used the Replay Attack spoofing dataset from IDIAP, consisting of 1300 videos, for model training, verification, and testing. Experiments showed that the algorithm proposed in the paper performed well on the Replay Attack data set and achieved a half total error rate of 0.66%.

A New Method for False Alarm Suppression in Heterogeneous Change Detection

Heterogeneous change detection has a wide range of applications in many fields. However, to date, many existing problems of heterogeneous change detection, such as false alarm suppression, have not been specifically addressed. In this article, we discuss the problem of false alarm suppression and propose a new method based on the combination of a convolutional neural network (CNN) and graph convolutional network (GCN). This approach employs a two-channel CNN to learn the feature maps of multitemporal images and then calculates difference maps of different scales, which means that both low-level and high-level features contribute equally to the change detection. The GCN, with a newly built convolution kernel (called the partially absorbing random walk convolution kernel), classifies these difference maps to obtain the inter-feature relationships between true targets and false ones, which can be represented by an adjacent matrix. We use pseudo-label samples to train the whole network, which means our method is unsupervised. Our method is verified on two typical data sets. The experimental results indicate the superiority of our method compared to some state-of-the-art approaches, which proves the efficacy of our method in false alarm suppression.

Multi-Defect Identification of Concrete Piles Based on Low Strain Integrity Test and Two-Channel Convolutional Neural Network

Defects in different positions and degrees in pile foundations will affect the building structure’s safety and the foundation’s bearing capacity. The efficiency and accuracy of using traditional methods to identify multi-defect types of pile foundations are very low, so finding suitable methods to improve their related indicators for pile foundation safety and engineering applications is necessary. In this paper, under the condition of secondary development of finite element software ABAQUS to obtain the time-domain signal database of six kinds of multi-defect pile foundations, a multi-defect type identification method of pile foundations based on two-channel convolutional neural network (TC-CNN) and low-strain pile integrity test (LSPIT) is proposed. Firstly, simulated time-domain signals of the dynamic measurements that match the experimental results performed wavelet packet denoising. Secondly, the 1D time-domain signals before and after denoising and the corresponding 2D wavelet time–frequency maps are inputs to retain more data information and prevent overfitting. Finally, TC-CNN achieved the multi-defect type identification of concrete piles. Compared with the single-channel convolutional neural network, this method can effectively fuse 1D and 2D features, extract more potential features, and make the classification accuracy reach 99.17%.

DDCNN: A Deep Learning Model for AF Detection From a Single-Lead Short ECG Signal.

With the popularity of the wireless body sensor network, real-time and continuous collection of single-lead electrocardiogram (ECG) data becomes possible in a convenient way. Data mining from the collected single-lead ECG waves has therefore aroused extensive attention worldwide, where early detection of atrial fibrillation (AF) is a hot research topic. In this paper, a two-channel convolutional neural network combined with a data augmentation method is proposed to detect AF from single-lead short ECG recordings. It consists of three modules, the first module denoises the raw ECG signals and produces 9-s ECG signals and heart rate (HR) values. Then, the ECG signals and HR rate values are fed into the convolutional layers for feature extraction, followed by three fully connected layers to perform the classification. The data augmentation method is used to generate synthetic signals to enlarge the training set and increase the diversity of the single-lead ECG signals. Validation experiments and the comparison with state-of-the-art studies demonstrate the effectiveness and advantages of the proposed method.

Blind Quality Prediction for View Synthesis Based on Heterogeneous Distortion Perception.

The quality of synthesized images directly affects the practical application of virtual view synthesis technology, which typically uses a depth-image-based rendering (DIBR) algorithm to generate a new viewpoint based on texture and depth images. Current view synthesis quality metrics commonly evaluate the quality of DIBR-synthesized images, where the DIBR process is computationally expensive and time-consuming. In addition, the existing view synthesis quality metrics cannot achieve robustness due to the shallow hand-crafted features. To avoid the complicated DIBR process and learn more efficient features, this paper presents a blind quality prediction model for view synthesis based on HEterogeneous DIstortion Perception, dubbed HEDIP, which predicts the image quality of view synthesis from texture and depth images. Specifically, the texture and depth images are first fused based on discrete cosine transform to simulate the distortion of view synthesis images, and then the spatial and gradient domain features are extracted in a Two-Channel Convolutional Neural Network (TCCNN). Finally, a fully connected layer maps the extracted features to a quality score. Notably, the ground-truth score of the source image cannot effectively represent the labels of each image patch during training due to the presence of local distortions in view synthesis image. So, we design a Heterogeneous Distortion Perception (HDP) module to provide effective training labels for each image patch. Experiments show that with the help of the HDP module, the proposed model can effectively predict the quality of view synthesis. Experimental results demonstrate the effectiveness of the proposed model.

Fault Diagnosis Method for Rolling Bearings Based on Two-Channel CNN under Unbalanced Datasets

As a critical component in industrial systems, timely and accurate fault diagnosis of rolling bearings is closely related to reliability and safety. Since the equipment usually operates in normal conditions with few fault samples, unbalanced data distribution problems lead to poor fault diagnosis ability. To address the above problems, a two-channel convolutional neural network (TC-CNN) model is proposed. Firstly, the frequency spectrum of the vibration signal is extracted using the Fast Fourier Transform (FFT), and the frequency spectrum is used as the input to the one-dimensional convolutional neural network (1D-CNN). Secondly, the time-frequency image of the vibration signal is extracted using generalized S-transform (GST), and the time-frequency image is used as the input to the two-dimensional convolutional neural network (2D-CNN). Then, feature extraction in the convolution and pooling layers is performed in the above two CNN channels, respectively. The feature vectors obtained from the two CNN models are stitched together in the fusion layer, and the fault classes are identified using an SVM classifier. Finally, using the rolling bearing experimental dataset of Case Western Reserve University (CWRU), the fault diagnosis effect of the proposed TC-CNN model under various data imbalance conditions is verified. In comparison with other related works, the experimental results demonstrate the better fault diagnosis results and robustness of the method.

Wildfire Identification Based on an Improved Two-Channel Convolutional Neural Network

The identification of wildfires is a very complex task due to their different shapes, textures, and colours. Traditional image processing methods need to manually design feature extraction algorithms based on prior knowledge, and because fires at different stages have different characteristics, manually designed feature extraction algorithms often have insufficient generalization capabilities. A convolutional neural network (CNN) can automatically extract the deeper features of an image, avoiding the complexity and blindness of the feature extraction phase. Therefore, a wildfire identification method based on an improved two-channel CNN is proposed in this paper. Firstly, in order to solve the problem of the insufficient dataset, the dataset is processed by using PCA_Jittering, transfer learning is used to train the model and then the accuracy of the model is improved by using segmented training. Secondly, in order to achieve the effective coverage of the model for fire scenes of different sizes, a two-channel CNN based on feature fusion is designed, in which the fully connected layers are replaced by a support vector machine (SVM). Finally, in order to reduce the delay time of the model, Lasso_SVM is designed to replace the SVM in the original model. The results show that the method has the advantages of high accuracy and low latency. The accuracy of wildfire identification is 98.47% and the average delay time of fire identification is 0.051 s/frame. The wildfire identification method designed in this paper improves the accuracy of identifying wildfires and reduces the delay time in identifying them.

Feature Extraction and Identification of Calligraphy Style Based on Dual Channel Convolution Network

To improve the effect of calligraphy style feature extraction and identification, this study proposes a calligraphy style feature extraction and identification technology based on two-channel convolutional neural network and constructs an intelligent calligraphy style feature extraction and identification system. Moreover, this paper improves the C3D network model and retains 2 fully connected layers. In addition, by extracting the outline skeleton and stroke features of calligraphy characters, this paper calculates the feature weight and authenticity determination function and constructs an authenticity identification system. The experimental study shows that the calligraphy style feature extraction and identification system based on the dual-channel convolutional neural network proposed in this paper has a good performance in calligraphy style feature extraction and identification.

LPI Radar Signal Recognition Based on Dual-Channel CNN and Feature Fusion

The accuracy of low probability of intercept (LPI) radar waveform recognition is an important and challenging problem in electronic warfare. Aiming at the problem of the difficulty in feature extraction and the low recognition rates of the LPI radar signal under a low signal-to-noise ratio, and inspired by the symmetry theory, we propose a new approach for the LPI radar signal recognition method based on a dual-channel convolutional neural network (CNN) and feature fusion. Our new approach contains three main modules: the preprocessing module that converts the LPI radar waveforms into two-dimensional time-frequency images using the Choi–Williams distribution (CWD) transformation and performs image binarization, the feature extraction module that extracts different features obtained from the images, and the recognition module that utilizes a multi-layer perceptron (MLP) network to fuse these features and distinguish the type of LPI radar signals. In the feature extraction module, a two-channel CNN model is proposed that extracts Histogram of Oriented Gradients (HOG) features and deep features from time-frequency images, respectively. Finally, the recognition module recognizes the radar signals using a Softmax classifier based on the fused features from two channels. The experimental results from 12 types of LPI radar signals prove the superiority and robustness of the proposed model. Its overall recognition rate reaches 97% when the signal-to-noise ratio is −6 dB.

A Convolutional Neural Network for Human Motion Recognition and Classification Using a Millimeter-Wave Doppler Radar

Human movement detection based on millimeter-wave radar sensors is a technology of interest in various areas such as for smart surveillance, security, behavioral biometrics, biomedical systems, robotics, etc. This paper shows the feasibility and effectiveness of using a compact 24 GHz Doppler radar with a built-in low-noise microwave amplifier (LNA) for detecting and extracting signals coming from human motion. Reliability and accuracy is assessed based on a comprehensive theoretical analysis, and on simulation results followed by experimental investigations. A continuous wavelet transform (CWT) is used to decompose in-phase and quadrature ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${I/Q}$ </tex-math></inline-formula> ) baseband signals to extract information about the dynamics of the system. The dataset consists of 1000 recordings in 8 motion classes (standing, walking, sitting, etc.). We propose to apply a two-channel convolutional neural network (CNN), which is composed of two CNN channels, for learning high-level features from time-domain signals and CWT spectrograms. One channel has four one-dimensional (1D) convolutional and pooling layers, and the other channel is made of three two-dimensional (2D) convolutional and pooling layers. In addition, the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${I/Q}$ </tex-math></inline-formula> signals are denoised by using Savitzky-Golay filtering, and both noisy and denoised signals are used as input signals for deep learning data augmentation purpose. We achieved an overall classification accuracy rate of 98.85% in motion classification for a two-branch CNN architecture, and an accuracy rate of 95.3% for a one-branch 2D-CNN. Our results show that a dual-channel CNN model can greatly increase the classification capabilities of human motion recognition and classification, and the proposed method can be effectively used with various radar signal classifications.

A novel art gesture recognition model based on two channel region-based convolution neural network for explainable human-computer interaction understanding

The application development of hot technology is both an opportunity and a challenge. The vision-based gesture recognition rate is low and real-time performance is poor, so various algorithms need to be studied to improve the accuracy and speed of recognition. In this paper, we propose a novel gesture recognition based on two channel region-based convolution neural network for explainable human-computer interaction understanding. The input gesture image is extracted through two mutually independent channels. The two channels have convolution kernel with different scales, which can extract the features of different scales in the input image, and then carry out feature fusion at the fully connection layer. Finally, it is classified by the softmax classifier. The two-channel convolutional neural network model is proposed to solve the problem of insufficient feature extraction by the convolution kernel. Experimental results of gesture recognition on public data sets NTU and VIVA show that the proposed algorithm can effectively avoid the over-fitting problem of training models, and has higher recognition accuracy and stronger robustness than traditional algorithms.

AI classification of wafer map defect patterns by using dual-channel convolutional neural network

In semiconductor manufacturing, the presence of abnormal chips in wafer products will reduce the yield of wafer products. The identification of the wafer map can find related problems in the wafer production process, and post-analysis is a necessary means to improve the wafer yield. In this study, we proposed the method that image-based multi-source and two-channel convolutional neural network for wafer map classification. This method consists of the following three main steps. First, using two different deep convolution neural networks (DCNN) to form a two-channel DCNN feature extraction model and extracting multiple sets of advanced features from multi-source data. Second, processing the multi-group data features extracted from different channels to obtain two sets of new multi-source features. Third, the multi-source features in the two channels are further processed to become a new set of classification features. Then input the new features into the combined classification model of error correction code and support vector machine (ECOC-SVM) to classify the defect pattern of the wafer map. The data used in the experiment comes from data sets in actual production (WM-811K). The experimental results showed that the proposed method has a good effect on the defect pattern recognition of wafer maps, and the effectiveness of the proposed method is confirmed. There are a total of 33,256 wafer maps, and the overall classification accuracy of 6552 test sets has reached 96.4%.

Radar High-Resolution Range Profile Ship Recognition Using Two-Channel Convolutional Neural Networks Concatenated with Bidirectional Long Short-Term Memory

Radar automatic target recognition is a critical research topic in radar signal processing. Radar high-resolution range profiles (HRRPs) describe the radar characteristics of a target, that is, the characteristics of the target that is reflected by the microwave emitted by the radar are implicit in it. In conventional radar HRRP target recognition methods, prior knowledge of the radar is necessary for target recognition. The application of deep-learning methods in HRRPs began in recent years, and most of them are convolutional neural network (CNN) and its variants, and recurrent neural network (RNN) and the combination of RNN and CNN are relatively rarely used. The continuous pulses emitted by the radar hit the ship target, and the received HRRPs of the reflected wave seem to provide the geometric characteristics of the ship target structure. When the radar pulses are transmitted to the ship, different positions on the ship have different structures, so each range cell of the echo reflected in the HRRP will be different, and adjacent structures should also have continuous relational characteristics. This inspired the authors to propose a model to concatenate the features extracted by the two-channel CNN with bidirectional long short-term memory (BiLSTM). Various filters are used in two-channel CNN to extract deep features and fed into the following BiLSTM. The BiLSTM model can effectively capture long-distance dependence, because BiLSTM can be trained to retain critical information and achieve two-way timing dependence. Therefore, the two-way spatial relationship between adjacent range cells can be used to obtain excellent recognition performance. The experimental results revealed that the proposed method is robust and effective for ship recognition.

Identification of Maize Leaf Diseases based on Convolutional Neural Network

The identification and diagnosis of crop leaf disease is of great significance to improve the quality of crop cultivation. Compared with the traditional manual diagnosis method, the automatic identification of crop leaf disease based on computer vision technology has high efficiency and no subjective judgment error. But the traditional image processing technology is affected by different illumination conditions, cross shading. The algorithm’s robustness is affected. Because deep learning dose not need to set learning features manually, which greatly improves the recognition efficiency. In this paper, the two-channel Convolutional Neural Network was constructed based on VGG and ResNet. Taking the maize leaf diseases as research objects, the maize leaf disease data set has been constructed and preprocessed. And the structure and characteristics of AlexNet, VGG and ResNet are introduced respectively. By adjusting the parameters of the two-channel Convolutional Neural Network, the accuracy of identifying the maize leaf disease type in the validation set can reach 98.33%, while the VGG model can reach 93.33%. The classification results on three types of maize leaf diseases show that the two-channel Convolutional Neural Network has a better performance than the single AlexNet model.Three kinds of leaf disease data sets (big spot, gray leaf spot and rust ) are downloaded from the “kaggle”platform.

Two-Channel Feature Extraction Convolutional Neural Network for Facial Expression Recognition

An emotion recognition framework based on a two-channel convolutional neural network (CNN) is proposed to detect the affective state of humans through facial expressions. The framework consists of three parts, i.e., the frontal face detection module, the feature extraction module, and the classification module. The feature extraction module contains two channels: one is for raw face images and the other is for texture feature images. The local binary pattern (LBP) images are utilized for texture feature extraction to enrich facial features and improve the network performance. The attention mechanism is adopted in both CNN feature extraction channels to highlight the features that are related to facial expressions. Moreover, arcface loss function is integrated into the proposed network to increase the inter-class distance and decrease the inner-class distance of facial features. The experiments conducted on the two public databases, FER2013 and CK+, demonstrate that the proposed method outperforms the previous methods, with the accuracies of 72.56% and 94.24%, respectively. The improvement in emotion recognition accuracy makes our approach applicable to service robots.

Classification of hyperspectral images based on two-channel convolutional neural network combined with support vector machine algorithm

Our study uses artificial intelligence and machine learning methods for dimensionality reduction, feature extraction, and classification of hyperspectral remote sensing images to detect subtle differences between different spectra. Our study also used hyperspectral remote sensing image data gathered by airborne visible infrared imaging spectrometer over the Indian Pines and Salinas test sites. Based on experimental analysis and comparison of mainstream classifiers and advanced algorithms, we propose a new classification mode for the hyperspectral image based on a two-channel fusion of convolutional neural network-support vector machine (CNN-SVM). This model gives full play to the advantages of CNN in feature extraction and the advantages of SVM in the classification. It also maximizes the generalization and accuracy of classification. The CNN is based on two-channel feature extraction, including spatial features and spectral features, which utilizes the characteristics of hyperspectral remote sensing images for classification. Experimental results show that the research is both theoretically and practically significant for improving the quality and accuracy of identification and classification.

Adversarial unsupervised domain adaptation for cross scenario waveform recognition

Deep Learning based waveform recognition has become a topical area of research recently. In fact, due to complex and heterogeneous scenarios, it is impossible to train a universal model for all waveform recognition tasks. Unsupervised domain adaptation (UDA) has an ability to take advantage of knowledge from an available source dataset and apply it to an unlabeled target dataset. Therefore, we propose a novel waveform recognition method based on an adversarial UDA, which incorporates adversarial learning to improve the cross scenario recognition performance. Specifically, a two-channel convolutional neural network is well designed for waveform recognition task and the system gradually converges an optimized stable state via two innovative methods, namely Adversarial unsupervised domain adaptation with gradient reversal (GR-AUDA) and Feature augmentation adversarial unsupervised domain adaptation (FA-AUDA). Furthermore, we sampled a more abundant waveform dataset under various scenarios, including typical indoor, outdoor, and corridor cases, using software defined radio. We evaluate our method on both public datasets and our own datasets. The results demonstrate that our proposed adversarial UDA framework is able to significantly improve the recognition performance of target domain waveforms.