Siamese Convolutional Neural Network Research Articles

Similarity learning to enable building searches in post‐event image data

AbstractReconnaissance teams collect perishable data after each disaster to learn about building performance. However, often these large image sets are not adequately curated, nor do they have sufficient metadata (e.g., GPS), hindering any chance to identify images from the same building when collected by different reconnaissance teams. In this study, Siamese convolutional neural networks (S‐CNN) are implemented and repurposed to establish a building search capability suitable for post‐disaster imagery. This method can automatically rank and retrieve corresponding building images in response to a single query using an image. In the demonstration, we utilize real‐world images collected from 174 reinforced‐concrete buildings affected by the 2016 Southern Taiwan and the 2017 Pohang (South Korea) earthquake events. A quantitative performance evaluation is conducted by examining two metrics introduced for this application: Similarity Score (SS) and Similarity Rank (SR).

Vision based human fall detection with Siamese convolutional neural networks

Fall detection is drawing serious attention all across the globe, as unattended fall of senior citizens creates long lasting injuries. This necessitates the deployment of automatic fall detection systems to facilitate smart care health environments for the elderly people living in various settings, viz., living independently in their homes, hospitalized or living in care homes. The proposed work employs Siamese network with one shot classification for human fall detection. Unlike the neural network that classifies the video sequences, this network learns to differentiate the video sequences by computing the similarity score. The network contains two identical CNNs, receiving pair of video sequences as the input. The features of these networks are merged at the final layer through the similarity function. Two different architectures viz., one with 2D convolutional filters and the other with depth wise convolutional filters, each operated on two set of features, RGB and optical flow features are developed. Experimental results demonstrate the effectiveness and feasibility of the proposed work compared to state-of-the methods.

A Siamese convolutional neural network with high–low level feature fusion for change detection in remotely sensed images

ABSTRACT Automatic change detection is an important and difficult task in the field of remote sensing. In this study, a deep Siamese convolutional network based on the fusion of high- and low-level features is proposed for change detection in remote sensing images. Given that low-level features correspond to low-order ones (e.g., texture) that are sensitive to change and that high-level features can accurately reflect image category information (e.g., semantic information), we fuse these features to enhance the abstractness and robustness of the extracted features in the change detection framework. The whole system is end-to-end and does not require any pre- or post-processing. Experimental results on three datasets show that our method is superior to other advanced methods by adding a high- and low-level fusion framework.

Comparative analysis of image projection-based descriptors in Siamese neural networks

Low-level object matching can be done using projection signatures. In case of a large number of projections, the matching algorithm has to deal with less significant slices. A trivial approach would be to do statistical analysis or apply machine learning to determine the significant features. To take adjacent values of the projection matrices into account, a convolutional neural network should be used. To compare two matrices, a Siamese-structure of convolutional heads can be applied. In this paper, an experiment is designed and implemented to analyze the object matching performance of Siamese Convolutional Neural Networks based on multi-directional image projection data. A backtracking search-based Neural Architecture Generation method is used to create convolutional architectures, and a Master/Worker structured distributed processing with highly efficient scheduling based on the Longest Processing Times-heuristics is used for parallel training and evaluation of the models. Results show that the projection-based methods are Pareto optimal in terms of one-shot classification accuracy and memory consumption.

Siamese Neural Network Based Few-Shot Learning for Anomaly Detection in Industrial Cyber-Physical Systems

With the increasing population of Industry 4.0, both AI and smart techniques have been applied and become hotly discussed topics in industrial cyber-physical systems (CPS). Intelligent anomaly detection for identifying cyber-physical attacks to guarantee the work efficiency and safety is still a challenging issue, especially when dealing with few labeled data for cyber-physical security protection. In this article, we propose a few-shot learning model with Siamese convolutional neural network (FSL-SCNN), to alleviate the over-fitting issue and enhance the accuracy for intelligent anomaly detection in industrial CPS. A Siamese CNN encoding network is constructed to measure distances of input samples based on their optimized feature representations. A robust cost function design including three specific losses is then proposed to enhance the efficiency of training process. An intelligent anomaly detection algorithm is developed finally. Experiment results based on a fully labeled public dataset and a few labeled dataset demonstrate that our proposed FSL-SCNN can significantly improve false alarm rate (FAR) and F1 scores when detecting intrusion signals for industrial CPS security protection.

Learning deep features for task-independent EEG-based biometric verification

Abstract Considerable interest has been recently devoted to the exploitation of brain activity as biometric identifier in automatic recognition systems, with a major focus on data acquired through electroencephalography (EEG). Several researches have in fact confirmed the presence of discriminative characteristics within brain signals recorded while performing specific mental tasks. Yet, to make EEG-based recognition appealing for practical applications, it would be highly advisable to investigate the existence and permanence of such distinctive traits while performing different mental tasks. In this regard, the present study evaluates the feasibility of performing task-independent EEG-based biometric recognition. A deep learning approach using siamese convolutional neural networks is employed to extract, from the considered EEG recordings, subject-specific template representations. An extensive set of experimental tests, performed on a multi-session database comprising EEG data acquired from 45 subjects while performing six different tasks, is employed to evaluate whether it is actually possible to verify the identity of a subject using brain signals, regardless the performed mental task.

A deep learning framework for 18F-FDG PET imaging diagnosis in pediatric patients with temporal lobe epilepsy

PurposeEpilepsy is one of the most disabling neurological disorders, which affects all age groups and often results in severe consequences. Since misdiagnoses are common, many pediatric patients fail to receive the correct treatment. Recently, 18F-fluorodeoxyglucose positron emission tomography (18F-FDG PET) imaging has been used for the evaluation of pediatric epilepsy. However, the epileptic focus is very difficult to be identified by visual assessment since it may present either hypo- or hyper-metabolic abnormality with unclear boundary. This study aimed to develop a novel symmetricity-driven deep learning framework of PET imaging for the identification of epileptic foci in pediatric patients with temporal lobe epilepsy (TLE).MethodsWe retrospectively included 201 pediatric patients with TLE and 24 age-matched controls who underwent 18F-FDG PET-CT studies. 18F-FDG PET images were quantitatively investigated using 386 symmetricity features, and a pair-of-cube (PoC)-based Siamese convolutional neural network (CNN) was proposed for precise localization of epileptic focus, and then metabolic abnormality level of the predicted focus was calculated automatically by asymmetric index (AI). Performances of the proposed framework were compared with visual assessment, statistical parametric mapping (SPM) software, and Jensen-Shannon divergence-based logistic regression (JS-LR) analysis.ResultsThe proposed deep learning framework could detect the epileptic foci accurately with the dice coefficient of 0.51, which was significantly higher than that of SPM (0.24, P < 0.01) and significantly (or marginally) higher than that of visual assessment (0.31–0.44, P = 0.005–0.27). The area under the curve (AUC) of the PoC classification was higher than that of the JS-LR (0.93 vs. 0.72). The metabolic level detection accuracy of the proposed method was significantly higher than that of visual assessment blinded or unblinded to clinical information (90% vs. 56% or 68%, P < 0.01).ConclusionThe proposed deep learning framework for 18F-FDG PET imaging could identify epileptic foci accurately and efficiently, which might be applied as a computer-assisted approach for the future diagnosis of epilepsy patients.Trial registrationNCT04169581. Registered November 13, 2019Public site: https://clinicaltrials.gov/ct2/show/NCT04169581

Lightweight Underwater Visual Loop Detection and Classification using a Siamese Convolutional Neural Network

This paper presents an end-to-end Neural Network (NN) to estimate the overlap between two scenes observed by an underwater robot endowed with a bottom-looking camera. This information is extremely valuable to perform visual loop detection in Simultaneous Localization and Mapping (SLAM). Contrarily to other existing approaches, this study does not depend on handcrafted features or similarity metrics, but jointly optimizes the image description and the loop detection by means of a Siamese NN architecture. Twelve different configurations have been experimentally tested using large balanced datasets synthetically generated from real data. These experiments demonstrate the ability of our proposal to properly estimate the overlap with precisions, recalls and fall-outs close to 95%, 98% and 5% respectively and execution times close to 0.7 ms per loop in a standard laptop computer. The source code of this proposal is publicly available.

Host-Based Intrusion Detection Model Using Siamese Network

As cyberattacks become more intelligent, the difficulty increases for traditional intrusion detection systems to detect advanced attacks that deviate from previously stored patterns. To solve this problem, a deep learning-based intrusion detection system model has emerged that analyzes intelligent attack patterns through data learning. However, deep learning models have the disadvantage of having to re-learn each time a new cyberattack method emerges. The time required to learn a large amount of data is not efficient. In this paper, an experiment was conducted using the Leipzig Intrusion Detection Data Set (LID-DS), which is a host-based intrusion detection data set released in 2018. In addition, in order to evaluate and improve the performance of the system, a host-based intrusion detection model consisting of pre-processing, vector-to-image processing, training and testing steps is proposed. In the training and testing steps, a Siamese Convolutional Neural Network (Siamese-CNN) is constructed using the few-shot learning method, which shows excellent performance by learning a small amount of data. Siamese-CNN determines whether the attack type is the same based on the similarity score of each cyberattack sample converted to an image. The accuracy was calculated using the few-shot learning technique. The performance of the Vanilla Convolutional Neural Network (Vanilla-CNN) and Siamese-CNN are compared to confirm the performance of Siamese-CNN. As a result of measuring the accuracy, precision, recall, and F1-score indicators, it was confirmed that the recall of the Siamese-CNN model proposed in this study increased by about 6% compared to the Vanilla-CNN model.

Image fusion algorithm based on multi-scale detail siamese convolutional neural network

Image fusion algorithm based on multi-scale detail siamese convolutional neural network

Deep learning supported disease detection with multi-modality image fusion.

Multi-modal image fusion techniques aid the medical experts in better disease diagnosis by providing adequate complementary information from multi-modal medical images. These techniques enhance the effectiveness of medical disorder analysis and classification of results. This study aims at proposing a novel technique using deep learning for the fusion of multi-modal medical images. The modified 2D Adaptive Bilateral Filters (M-2D-ABF) algorithm is used in the image pre-processing for filtering various types of noises. The contrast and brightness are improved by applying the proposed Energy-based CLAHE algorithm in order to preserve the high energy regions of the multimodal images. Images from two different modalities are first registered using mutual information and then registered images are fused to form a single image. In the proposed fusion scheme, images are fused using Siamese Neural Network and Entropy (SNNE)-based image fusion algorithm. Particularly, the medical images are fused by using Siamese convolutional neural network structure and the entropy of the images. Fusion is done on the basis of score of the SoftMax layer and the entropy of the image. The fused image is segmented using Fast Fuzzy C Means Clustering Algorithm (FFCMC) and Otsu Thresholding. Finally, various features are extracted from the segmented regions. Using the extracted features, classification is done using Logistic Regression classifier. Evaluation is performed using publicly available benchmark dataset. Experimental results using various pairs of multi-modal medical images reveal that the proposed multi-modal image fusion and classification techniques compete the existing state-of-the-art techniques reported in the literature.

Deep Siamese Cross-Residual Learning for Robust Visual Tracking

The sixth-generation (6G) wireless technology contributes to the establishment of the Internet of Things (IoT). Recently, the IoT has become popular because of its smart architectures and various applications. Among these applications, intelligent urban surveillance systems for smart cities are becoming more and more important. Therefore, designing a robust visual tracking method has become an urgent task. Deep Siamese convolutional neural networks have been applied to visual tracking recently because of their advantageous abilities to learn a matching function between the template and the target candidate. Unlike traditional Siamese networks, which separately treat the two branches, we propose deep Siamese cross-residual learning to entangle the two branches from the beginning to the end of the Siamese network. This strategy can make the two branches exchange instance-specific information at different nodes of the network and learn a more compact representation of the target. In addition, we propose a combined loss function, which consists of two complementary tasks. One task is to learn a matching function directly and the other one is to learn a classification function. Moreover, our model does not need to load any pretrained weights and is trained with limited sequences from scratch. Plenty of experiments show that our tracker performs favorably against many state-of-the-art tracking methods.

Optical and SAR Image Matching Using Pixelwise Deep Dense Features

Image matching is a primary technology to fuse the complementary information from optical and SAR images. Due to the high nonlinear radiometric and geometric relationship, the optical and SAR image matching task remains a widely unsolved challenge. In this study, we propose to use a Siamese convolutional neural network (CNN) architecture to learn pixelwise deep dense features. The proposed network is able to balance the learning of high-level semantic information and low-level fine-grained information, which is nonnegligible for feature matching task. Under the local searching framework, the loss function is defined based on the score map produced by the sum of squared differences (SSDs) between the learned pixelwise dense features of local optical and the SAR image patches, with a fast implementation in the frequency domain. The hardest negative mining strategy is adopted to increase the discrimination of the network. Extensive experiments are conducted on optical and SAR image pairs of different spatial resolution and different landcover types, verifying the superiority and robustness of the proposed method in terms of matching accuracy and matching precision.

WITHDRAWN: Contactless attendance system using Siamese neural network based face recognition

Abstract One of the most important and fast growing biometrical technological advancement is face recognition, used for identification of the person without any human intervention. In the proposed work, one shot learning has been used on Siamese convolution neural network (SCNN) for face recognition and attendance of the employee is entered in the MySQL database. Application Programmable interface used for uploading database is APACHE Web Server. The advantage of one shot learning algorithm is that it requires single training set for similarity measure. Get well soon message is send to absent employees through Gmail using send mail transfer protocol. Twelve images have been successfully tested and the system observed state-of –art performance with all face recognized correctly.

Deep neural network based approach for detection of defective solar cell

Abstract In the current scenario, renewable energies play a crucial factor to fulfil the increasing power demand in accordance with environment protection. With the advancement of solar energy, this reduces the gap of providing the power supply with the protection to the environmental parameter. However, it is also analyzed that the efficiency of power supply degrades due to the defects of solar cell. Electroluminescence (EL) image can be found to be the source for visually identifying the various kinds of defects. Although manual approach for classifying the EL imaging for identifying the defect solar is possible but those methods suffers from the time and cost factors. In order to overcome these constraints, a simple and elegant deep learning based approach is proposed in this paper to classify the different kinds of defects present in the EL image of solar cell. In this work, we first pre-process the EL image for removing the noise and distortions, then used Deep Siamese convolution neural network (CNN) for classifying the different kinds of defects present in the solar cell. Proposed model is tested on the standard EL image dataset. Simulation results show the proposed model provides better classification accuracy towards finding the defective solar cell.

Hierarchical Paired Channel Fusion Network for Street Scene Change Detection.

Street Scene Change Detection (SSCD) aims to locate the changed regions between a given street-view image pair captured at different times, which is an important yet challenging task in the computer vision community. The intuitive way to solve the SSCD task is to fuse the extracted image feature pairs, and then directly measure the dissimilarity parts for producing a change map. Therefore, the key for the SSCD task is to design an effective feature fusion method that can improve the accuracy of the corresponding change maps. To this end, we present a novel Hierarchical Paired Channel Fusion Network (HPCFNet), which utilizes the adaptive fusion of paired feature channels. Specifically, the features of a given image pair are jointly extracted by a Siamese Convolutional Neural Network (SCNN) and hierarchically combined by exploring the fusion of channel pairs at multiple feature levels. In addition, based on the observation that the distribution of scene changes is diverse, we further propose a Multi-Part Feature Learning (MPFL) strategy to detect diverse changes. Based on the MPFL strategy, our framework achieves a novel approach to adapt to the scale and location diversities of the scene change regions. Extensive experiments on three public datasets (i.e., PCD, VL-CMU-CD and CDnet2014) demonstrate that the proposed framework achieves superior performance which outperforms other state-of-the-art methods with a considerable margin.

Multi-View Instance Matching with Learned Geometric Soft-Constraints

We present a new approach for matching urban object instances across multiple ground-level images for the ultimate goal of city-scale mapping of objects with high positioning accuracy. What makes this task challenging is the strong change in view-point, different lighting conditions, high similarity of neighboring objects, and variability in scale. We propose to turn object instance matching into a learning task, where image-appearance and geometric relationships between views fruitfully interact. Our approach constructs a Siamese convolutional neural network that learns to match two views of the same object given many candidate image cut-outs. In addition to image features, we propose utilizing location information about the camera and the object to support image evidence via soft geometric constraints. Our method is compared to existing patch matching methods to prove its edge over state-of-the-art. This takes us one step closer to the ultimate goal of city-wide object mapping from street-level imagery to benefit city administration.

Data association framework based on biconnected gated recurrent unit network for multiple object tracking

In the tracking-by-detection scheme of multiple object tracking (MOT), the data association process in which existing tracking data and new detections are matched over time is very important. A framework is proposed to solve the data association problem in MOT in scenarios where there are potential target interactions and occlusions in crowded environments. This framework consists of an input layer and an association layer. The input layer is an end-to-end feature-map extraction model that incorporates a simplified Siamese convolutional neural network, which effectively distinguishes similar objects based on their appearance and motion. The association layer is composed of a bidirectional gated recurrent unit network with three layers of fully connected networks (FCNs), whose outputs are fed into the FCNs and transformed into an association matrix that reflects the matching scores between the detections and existing tracks. The matrix is then used to minimize the loss of the framework. The experimental results show that the proposed framework demonstrates outstanding performance for MOT, with its accuracy and precision in MOT reaching values as high as 26.1% and 71.2%, respectively.

Siamese Convolutional Neural Network for ASL Alphabet Recognition

American sign language is an important communication way to convey information among the deaf community in North America and is primarily used by people who have hearing or speech impairments. The deaf community faces a struggle in schools and other institutions because they usually consist primarily of hearing people. Besides, deaf people often feel misunderstood by people who do not knowsign language, for example, family members. In the last two decades, researchers have been proposing automatic sign language recognition systems to facilitate the learning of sign language, and nowadays, computer scientists have focused on using artificial intelligence in order to develop a system capable of reducing the communication gap between hearing and deaf people. In this paper, it is proposed a siamese convolutional neural network for American sign language alphabet recognition. This siamese architecture allows the computer to reduce the high interclass similarity and high intraclass variations. The results show that the proposed method outperforms the state of the art systems.

Siamese hyperspectral target detection using synthetic training data

This Letter presents a method for training convolutional neural networks (CNNs) that detect targets of interest in hyperspectral images. Collecting suitable and abundant training data has been the main obstacle to the successful application of CNNs to hyperspectral target detection. To solve the problem, the authors propose a scheme to generate synthetic training data. Publicly available spectral reflectance library and an easy-to-obtain radiative transfer model are utilised in their scheme. Using the synthetic training data only, a Siamese CNN is trained to learn robust features for the detection task. Experimental results on the real hyperspectral image show the effectiveness of the proposed method.