Siamese Convolutional Neural Network Research Articles

A New Method for Bearing Fault Diagnosis across Machines Based on Envelope Spectrum and Conditional Metric Learning.

In recent years, most research on bearing fault diagnosis has assumed that the source domain and target domain data come from the same machine. The differences in equipment lead to a decrease in diagnostic accuracy. To address this issue, unsupervised domain adaptation techniques have been introduced. However, most cross-device fault diagnosis models overlook the discriminative information under the marginal distribution, which restricts the performance of the models. In this paper, we propose a bearing fault diagnosis method based on envelope spectrum and conditional metric learning. First, envelope spectral analysis is used to extract frequency domain features. Then, to fully utilize the discriminative information from the label distribution, we construct a deep Siamese convolutional neural network based on conditional metric learning to eliminate the data distribution differences and extract common features from the source and target domain data. Finally, dynamic weighting factors are employed to improve the convergence performance of the model and optimize the training process. Experimental analysis is conducted on 12 cross-device tasks and compared with other relevant methods. The results show that the proposed method achieves the best performance on all three evaluation metrics.

Two-stage prediction method for capacity aging trajectories of lithium-ion batteries based on Siamese-convolutional neural network

Accurate battery lifespan prediction can guarantee battery safety in applications. Considering the complex nonlinear nature of battery degradation, it is more challenging to predict battery aging trajectories utilizing less data. A two-stage prediction method for battery capacity aging trajectories is proposed. The first stage is battery lifespan prediction based on the Siamese-Convolutional neural network (Siamese-CNN). Taking the predicted lifespan as its prior information, the second stage is predicting capacity aging trajectories with the Convolutional neural network (CNN). The method only requires partial voltage-current data from 30 cycles, of which the similarity in different batteries is obtained by the Siamese-CNN. The lifespan-unknown batteries in the testing set are fed into the Siamese-CNN to match the lifespan-known training set battery with the highest similarity. Then, CNN is employed to predict the capacity aging trajectories under different profiles. The ratio of each cell's predicted lifespan to the fixed length of the CNN output sequence is regarded as the interval for uniform sampling to keep the series consistent. The prediction accuracy of the method is validated by an open dataset. It is demonstrated that the battery lifespan and capacity aging trajectory prediction achieve a mean absolute percentage error (MAPE) of 2.44% and 1.28%, respectively.

Predicting diabetic retinopathy stage using Siamese Convolutional Neural Network

ABSTRACT Diabetic retinopathy (DR) is considered to be a leading cause of blindness in people aged 16 to 64 years, affecting around 40% of the population diagnosed with diabetes mellitus (DM). DR is usually identified through retinal image analysis, and in some countries, such as Brazil, such diagnosis is hindered by limited access to specialized care, leading to lengthy waits for ophthalmological evaluations. This scenario makes the Brazilian Diabetic Society’s annual DR check recommendation impractical for many. To address this gap, our study introduces a novel Siamese Convolutional Neural Network (SCNN) for DR prediction, usable by primary care professionals. Our SCNN analyzes pairs of eye fundus images and employs shared weights in its layers to extract essential features, facilitating similarity measurement between neural network outputs. Despite challenges with limited and imbalanced datasets, our SCNN showed effectiveness. We tested it on four datasets (IDRiD, APTOS, Messidor-1, DIARETDB0), with accuracy ranging from 67.23% (APTOS) to 96.85% (DIARETDB0). Compared to other methods, our approach consistently excelled, especially in recall analysis. These results suggest that deep learning via Siamese networks is likely to be a viable and potential DR screening tool.

A Siamese Convolutional Neural Network for Identifying Mild Traumatic Brain Injury and Predicting Recovery.

Timely diagnosis of mild traumatic brain injury (mTBI) remains challenging due to the rapid recovery of acute symptoms and the absence of evidence of injury in static neuroimaging scans. Furthermore, while longitudinal tracking of mTBI is essential in understanding how the diseases progresses/regresses over time for enhancing personalized patient care, a standardized approach for this purpose is not yet available. Recent functional neuroimaging studies have provided evidence of brain function alterations following mTBI, suggesting mTBI-detection models can be built based on these changes. Most of these models, however, rely on manual feature engineering, but the optimal set of features for detecting mTBI may be unknown. Data-driven approaches, on the other hand, may uncover hidden relationships in an automated manner, making them suitable for the problem of mTBI detection. This paper presents a data-driven framework based on Siamese Convolutional Neural Network (SCNN) to detect mTBI and to monitor the recovery state from mTBI over time. The proposed framework is tested on the cortical images of Thy1-GCaMP6s mice, obtained via widefield calcium imaging, acquired in a longitudinal study. Results show that the proposed model achieves a classification accuracy of 96.5%. To track the state of the injured brain over time, a reference distance map is constructed, which together with the SCNN model, are employed to assess the recovery state in subsequent sessions after injury, revealing that the recovery progress varies among subjects. The promising results of this work suggest that a similar approach could be potentially applicable for monitoring recovery from mTBI, in humans.

Dimension-Expanded-Based Matching Method With Siamese Convolutional Neural Networks for Gravity-Aided Navigation

Matching algorithm is the key technique of the gravity-aided inertial navigation system. With the development of artificial intelligence, many neural network-based matching methods have been extensively studied. The pattern recognition-based matching methods transforms the matching problem as pattern recognition, which cannot be used directly on datasets where the neural networks have not been trained. To improve the accuracy of navigation and positioning, it is necessary to extract muti-dimensional gravity features from the limited navigation information. In this paper, the sequence of the gravity anomaly value is expanded to two-dimensional feature map containing time series features by Gramian Angular Fields method, which preserves the numerical information of the one-dimensional sequence and extracts the correlation relationship between each element. In addition, to reduce the influence of gravity measurement instrument error on the position precision of gravity matching algorithm, affine transformation is performed on INS trajectory and a Siamese convolutional neural network model is proposed to compare the measured gravity database with the gravity anomaly value in the pre-stored gravity background map and get the matching position. Simulation results and practical tests show that the proposed method can obtain a more precise location result compared with the traditional matching algorithm.

Few-shot transfer learning for personalized atrial fibrillation detection using patient-based siamese network with single-lead ECG records

The proliferation of wearable devices has allowed the collection of electrocardiogram (ECG) recordings daily to monitor heart rhythm and rate. For example, 24-hour Holter monitors, cardiac patches, and smartwatches are widely used for ECG gathering and application. An automatic atrial fibrillation (AF) detector is required for timely ECG interpretation. Deep learning models can accurately identify AFs if large amounts of annotated data are available for model training. However, it is impractical to request sufficient labels for ECG recordings for an individual patient to train a personalized model. We propose a Siamese-network-based approach for transfer learning to address this issue. A pre-trained Siamese convolutional neural network is created by comparing two labeled ECG segments from the same patient. We sampled 30-second ECG segments with a 50% overlapping window from the ECG recordings of patients in the MIT-BIH Atrial Fibrillation Database. Subsequently, we independently detected the occurrence of AF in each patient in the Long-Term AF Database. By fine-tuning the model with the 1, 3, 5, 7, 9, or 11 ECG segments ranging from 30 to 180 s, our method achieved macro-F1 scores of 96.84%, 96.91%, 96.97%, 97.02%, 97.05%, and 97.07%, respectively.

Despeckling of SAR Images Using Residual Twin CNN and Multi-Resolution Attention Mechanism

The despeckling of synthetic aperture radar images using two different convolutional neural network architectures is presented in this paper. The first method presents a novel Siamese convolutional neural network with a dilated convolutional network in each branch. Recently, attention mechanisms have been introduced to convolutional networks to better model and recognize features. Therefore, we propose a novel design for a convolutional neural network using an attention mechanism for an encoder–decoder-type network. The framework consists of a multiscale spatial attention network to improve the modeling of semantic information at different spatial levels and an additional attention mechanism to optimize feature propagation. Both proposed methods are different in design but they provide comparable despeckling results in subjective and objective measurements in terms of correlated speckle noise. The experimental results are evaluated on both synthetically generated speckled images and real SAR images. The methods proposed in this paper are able to despeckle SAR images and preserve SAR features.

A Siamese Inverted Residuals Network Image Steganalysis Scheme based on Deep Learning

With the rapid proliferation of urbanization, massive data in social networks are collected and aggregated in real time, making it possible for criminals to use images as a cover to spread secret information on the Internet. How to determine whether these images contain secret information is a huge challenge for multimedia computing security. The steganalysis method based on deep learning can effectively judge whether the pictures transmitted on the Internet in urban scenes contain secret information, which is of great significance to safeguarding national and social security. Image steganalysis based on deep learning has powerful learning ability and classification ability, and its detection accuracy of steganography images has surpassed that of traditional steganalysis based on manual feature extraction. In recent years, it has become a hot topic of the information hiding technology. However, the detection accuracy of existing deep learning based steganalysis methods still needs to be improved, especially when detecting arbitrary-size and multi-source images, their detection efficientness is easily affected by cover mismatch. In this manuscript, we propose a steganalysis method based on Inverse Residuals structured Siamese network (abbreviated as SiaIRNet method, Sia mese- I nverted- R esiduals- Net work Based method). The SiaIRNet method uses a siamese convolutional neural network (CNN) to obtain the residual features of subgraphs, including three stages of preprocessing, feature extraction, and classification. Firstly, a preprocessing layer with high-pass filters combined with depth-wise separable convolution is designed to more accurately capture the correlation of residuals between feature channels, which can help capture rich and effective residual features. Then, a feature extraction layer based on the Inverse Residuals structure is proposed, which improves the ability of the model to obtain residual features by expanding channels and reusing features. Finally, a fully connected layer is used to classify the cover image and the stego image features. Utilizing three general datasets, BossBase-1.01, BOWS2, and ALASKA#2, as cover images, a large number of experiments are conducted comparing with the state-of-the-art steganalysis methods. The experimental results show that compared with the classical SID method and the latest SiaStegNet method, the detection accuracy of the proposed method for 15 arbitrary-size images is improved by 15.96% and 5.86% on average, respectively, which verifies the higher detection accuracy and better adaptability of the proposed method to multi-source and arbitrary-size images in urban scenes.

Sensor fault analysis of aero-engine using ensemble SCNN and Bayesian interval estimation

Sensors of an aircraft engine have the problems of multiple kinds of faults and difficulty to diagnose. Moreover, the diagnosis performance cannot meet practical demands. Therefore, this paper proposes a novel approach for sensor fault diagnosis integrating the advantages of selective ensemble learning, siamese convolutional neural network (SCNN), and Bayesian interval estimation (BIE). The Bayesian interval estimation with hypothesis testing is developed with real-time data update to reduce the confidence interval. The attention mechanism is introduced to enhance fault diagnosis performance. Moreover, ensemble selection learning with Akaike information criteria is proposed to obtain optimal SCNNs. The theorems of convergence and optimal confidence data in ensemble learning are proven. Furthermore, this paper develops a novel improved learning algorithm based on AdaMod (Adaptive and momental bound method) and SGD (stochastic gradient descent), and achieves a new fault diagnosis algorithm to improve the diagnosis performance. The sensor data of aircraft engines from China Eastern Airlines is adopted for validation and experimental results show that proposed approach reaches an excellent tradeoff between sensor diagnosis performance, confidence interval, and runtime. The ablation studies are performed to analyze the role of leaner’s components. The proposed method improves training accuracy by 6.2%, fault diagnosis performance by 5.1%, fault prediction mean result by 4.6%, and fault prediction variance is reduced by 38.1% than the state-of-the-art methods.

Automated fundus ultrasound image classification based on siamese convolutional neural networks with multi-attention

Fundus ultrasound image classification is a critical issue in the medical field. Vitreous opacity (VO) and posterior vitreous detachment (PVD) are two common eye diseases, Now, the diagnosis of these two diseases mainly relies on manual identification by doctors. This method has the disadvantages of time-consuming and manual investment, so it is very meaningful to use computer technology to assist doctors in diagnosis. This paper is the first to apply the deep learning model to VO and PVD classification tasks. Convolutional neural network (CNN) is widely used in image classification. Traditional CNN requires a large amount of training data to prevent overfitting, and it is difficult to learn the differences between two kinds of images well. In this paper, we propose an end-to-end siamese convolutional neural network with multi-attention (SVK_MA) for automatic classification of VO and PVD fundus ultrasound images. SVK_MA is a siamese-structure network in which each branch is mainly composed of pretrained VGG16 embedded with multiple attention models. Each image first is normalized, then is sent to SVK_MA to extract features from the normalized images, and finally gets the classification result. Our approach has been validated on the dataset provided by the cooperative hospital. The experimental results show that our approach achieves the accuracy of 0.940, precision of 0.941, recall of 0.940, F1 of 0.939 which are respectively increased by 2.5%, 1.9%, 3.4% and 2.5% compared with the second highest model.

Fault Diagnosis of Wind Turbine with Alarms Based on Word Embedding and Siamese Convolutional Neural Network

Alarms generated by a wind turbine alarm system indicate the need for emergency action by operators to protect the turbine from running into risky conditions. However, it can be challenging for operators to identify the fault types that trigger alarms, particularly with few labeled fault samples. This paper proposes a novel fault diagnosis method for wind turbines with alarms that collaboratively uses labeled and unlabeled alarms to improve diagnosis accuracy. First, the proposed method distinguishes different alarm sequences using a designed Siamese convolutional neural network with an embedding layer (S-ECNN) model. Then, the fault category of an unknown alarm sequence is diagnosed based on similarity scores. Specifically, the Skip-gram model is used to mine potential relationships among alarms in unlabeled alarm sequences, and pretrained alarm vectors are obtained. In the S-ECNN model, the pretrained alarm vectors are further optimized and trained using labeled alarm sequences. The similarity scores are calculated based on the distance between the extracted discriminative features of alarm sequences. The effectiveness of the proposed method is validated using actual alarm data from a wind farm.

Using a lightweight Siamese neural network for generating a feature vector in a vascular authentication system

The article analyzes the possibility of using a Siamese convolutional neural network to solve the problem of vascular authentication on an embedded hardware platform with limited computing resources (Orange Pi One). The authors give a brief review of modern methods for calculating image feature vectors used in the tasks of classifying, comparing or searching for images by content: based on variational series (histograms), local descriptors, singular point descriptors, descriptors based on hash functions, neural network descriptors. They suggest using the architecture of a biometric authentication system (BAS) based on images of palms in the visible and near-IR spectra based on a Siamese convolutional neural network. The developed software solution allows using the Siamese neural network in the "full network" (both symmetrical channels of the neural network are used) and "half of the neural network" (only one channel is used) modes to reduce the time for comparing biometric data vectors - images of the palms of registered BAS users. The authors demonstrate advantages of the neural network features: universality, scalability and competitiveness, including on embedded hardware and software solutions with limited computing resources without graphics accelerators. The studies have shown that using the Siamese neural network, the "overall accuracy" of palm image classification can be improved from 0.929 to 0.968 when compared with the image vectorization method based on a perceptual hash, while showing a comparable authentication time for individuals registered in BAS. In the experiments, the authors use a database of 2,000 images for 400 people.

Contrastive fusion representation learning for foreground object detection

This paper investigates the problem of an effective and robust fusion representation method for foreground moving object detection. Many fusion representation learning approaches pay attention to the similarity measurement between the fusion results and source images in texture details and pixel intensity, ignoring the harmful information, e.g., noise, blur, and extreme illumination. Therefore, the aggregated features of infrared and visible images will introduce much harmful information, affecting the model performance of downstream visual tasks. This paper tackles these problems by proposing a contrastive fusion representation learning method for the foreground moving object detection task, which consists of two major modules: the upstream fusion representation module (FRM) and the downstream foreground moving object detection module (FODM). Unlike the traditional fusion optimization mechanism, the former aims to extract valuable features and reject harmful features via the maximum mutual information theory. The latter is a siamese convolutional neural network to detect foreground moving objects by aggregating the time-sequence images generated by FRM. Experimental results and comparisons with the state-of-the-art on three public datasets (i.e., TNO, MF, and cross-modal FOD dataset of infrared and visible images), validate the effectiveness, robustness, and overall superiority of the proposed contrast fusion representation learning method. Concisely, our contrastive fusion representation learning has gained 53.9%,43.2%,46.4%,52.3%,2.2%,87.1%,3.5% on EI, SF, DF, AG, MI, and Nabf metrics compared with the best competitors.

Fusion representation learning for foreground moving object detection

This paper investigates the problem of effective and robust fusion representation for foreground moving object detection. Many deep learning-based approaches pay attention to network architecture based on a single modality image, ignoring the complementary mechanism of cross-modal images or only considering single-frame prediction without temporal association. We tackle these problems by proposing a fusion representation learning method for the foreground moving object detection task, which consists of two major modules: the upstream fusion representation module (FRM) and the downstream foreground moving object detection module (FODM). Unlike traditional feature aggregate methods, the FRM module is a quality-aware and online learnable fusion module which can aggregate valuable features while rejecting the harmful information in the source images. Specifically, the FODM module is a siamese convolutional neural network to detect foreground moving objects by aggregating the time-sequence images generated by FRM. Moreover, a new aligned foreground moving object detection dataset of infrared and visible images is constructed to provide a new option for benchmark evaluation. Experimental results and comparisons with the state-of-the-art on three public datasets validate the effectiveness, robustness, and overall superiority of our method.

Implementing a deep-learning model using Google street view to combine social and physical indicators of gentrification

While physical changes have been empirically recognised as a fundamental component of neighbourhood change, data and modelling constraints have limited the quantification of these indicators. Recently however, the proliferation of Big Data and advancements in deep learning (DL) techniques have enabled mass image processing. It is in this context that we build a Siamese Convolutional Neural Network using Google Street View (GSV) images to detect upgrades to properties as evidence of gentrification. The model achieves 84.8% test accuracy and 74.6% AUC. Building upgrades detected using the model are mapped using Kernel Density Estimation (KDE) and validated against Development Approvals. The DL GSV model is subsequently combined with the socioeconomic-based predictions of seven gentrifying suburbs from a prior Sydney-based gentrification study. The maps of predicted social change are validated against the spatial patterns of building upgrades detected by the DL GSV model. Of the five suburbs with sufficient data, the socioeconomic trends were affirmed with physical indicators of gentrification in four and questioned in one. The paper provides the first machine learning approach to combine social and physical indicators of gentrification. This automated, multi-dimensional approach enables the distinction of gentrification from other forms of neighbourhood change, helping to develop a more comprehensive understanding of gentrification occurring within a city.

An Easy-To-Update Pulse-Like Ground Motion Identification Method Based on Siamese Convolutional Neural Networks

ABSTRACT A pulse-like ground motion identification method is proposed based on the Siamese Convolutional Neural Networks (SCNNs). The wavelet coefficient graphs of pulse-like ground motions are used as the input data, and the features extracted by the SCNNs are used for the identification. Based on the time-domain features, pulse-like and non-pulse-like ground motions are classified into several classes. The results indicated that all the identified pulse-like ground motions have similar pulse features to the pre-selected training data. The principal advantage of the method is that the misclassification problem can be minimized by an updating procedure proposed by this paper.

Review Paper on an Authentication System using Siamese Convolutional Neural Networks

Due to its distinct advantages, finger vein verification has lately drawn more attention. Focusing on the characteristics of finger vein verification, construct a Siamese structure combining with a modified contrastive loss function for training the above CNN, which effectively improves the network's performance. The experimental findings demonstrate that the lightweight CNN's size shrinks to 1/6th of the pretrained-weights based CNN and that it achieves an equal error rate of 75% in the SDUMLA-HMT dataset, which outperforms cutting-edge techniques and nearly maintains the same performance as CNN that is based on pretrained weights.

CNN-Siam: multimodal siamese CNN-based deep learning approach for drug‒drug interaction prediction

BackgroundDrug‒drug interactions (DDIs) are reactions between two or more drugs, i.e., possible situations that occur when two or more drugs are used simultaneously. DDIs act as an important link in both drug development and clinical treatment. Since it is not possible to study the interactions of such a large number of drugs using experimental means, a computer-based deep learning solution is always worth investigating. We propose a deep learning-based model that uses twin convolutional neural networks to learn representations from multimodal drug data and to make predictions about the possible types of drug effects.ResultsIn this paper, we propose a novel convolutional neural network algorithm using a Siamese network architecture called CNN-Siam. CNN-Siam uses a convolutional neural network (CNN) as a backbone network in the form of a twin network architecture to learn the feature representation of drug pairs from multimodal data of drugs (including chemical substructures, targets and enzymes). Moreover, this network is used to predict the types of drug interactions with the best optimization algorithms available (RAdam and LookAhead). The experimental data show that the CNN-Siam achieves an area under the precision-recall (AUPR) curve score of 0.96 on the benchmark dataset and a correct rate of 92%. These results are significant improvements compared to the state-of-the-art method (from 86 to 92%) and demonstrate the robustness of the CNN-Siam and the superiority of the new optimization algorithm through ablation experiments.ConclusionThe experimental results show that our multimodal siamese convolutional neural network can accurately predict DDIs, and the Siamese network architecture is able to learn the feature representation of drug pairs better than individual networks. CNN-Siam outperforms other state-of-the-art algorithms with the combination of data enhancement and better optimizers. But at the same time, CNN-Siam has some drawbacks, longer training time, generalization needs to be improved, and poorer classification results on some classes.

One Step of Gait Information From Sensing Walking Surface for Personal Identification

The accuracy of gait sensing detection and identification determines the usability and acceptability in field applications. Four inertial measurement units (IMUs) recorded from heel contact to toe-off as a footstep in gait. The 1400 footsteps from 70 participants were collected, and the 13 footsteps from each were randomly selected for the Siamese convolutional neural network (SCNN) model training. The inputs in the SCNN model were a matrix of 4 (sensor) <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times150$ </tex-math></inline-formula> (sample) with six channels, which were associated with three axes of acceleration and gyroscopes of one footstep. Subsequently, different numbers of enrolled people were used in the training model, and different footsteps were set as a threshold for ingroup or imposter detections. Even in the smallest number of enrolled people dataset in ingroup detection, the performance of the SCNN was significantly better than the long short-term memory (LSTM). The highest accuracy rate of ingroup detection was 96.33% in ten enrolled people, and the highest for imposter detection was 81.93% in 60 enrolled people under the 13 footsteps threshold. The best accuracy rate was in 30 enrolled people when employed ingroup and imposter detections together, significantly higher than the other enrolled numbers. The present study highlighted two contributions of an embedded IMU surface measurement system in gait information collection. First, only 13 steps of each participant were required in training, which was considered a minimal data input. Second, only one footstep achieved very promising personal identification for each person in the ingroup population and imposter detection, making our system highly applicable to gait biometrics.

Learning a Novel LiDAR Submap-Based Observation Model for Global Positioning in Long-Term Changing Environments

In complex environments with long-term changes such as light, seasonal, and viewpoint changes, robust, accurate, and high-frequency global positioning based on light detection and ranging (LiDAR) map is still a challenge, which is crucial for autonomous vehicles or robots. To this end, a novel observation model that relies on the siamese multitask convolutional neural networks (CNNs) with multimodule cascaded is creatively presented in this article. In particular, a new pseudoimage representing LiDAR submap is designed to enrich scene texture and enhance rotation invariance. Besides, novel siamese CNNs that are coupled by NeXtVLAD and long short-term memory is designed for the first time, which can reliably predict similarity and quaternion at the same time. Finally, the predicted quaternion observation is integrated into the extended Kalman filter framework for multisensor fusion to achieve robust high-frequency global pose estimation. Extensive evaluations on KITTI, NCLT, and real-world datasets suggest that the proposed method not only obtains the remarkable precision-recall performance, but also effectively enhances and improves the robustness and accuracy of long-term positioning.