Siamese Model Research Articles

Merging Biomedical Ontologies with BioSTransformers.

Ontologies play a key role in representing and structuring domain knowledge. In the biomedical domain, the need for this type of representation is crucial for structuring, coding, and retrieving data. However, available ontologies do not encompass all the relevant concepts and relationships. In this paper, we propose the framework SiMHOMer (Siamese Models for Health Ontologies Merging) to semantically merge and integrate the most relevant ontologies in the healthcare domain, with a first focus on diseases, symptoms, drugs, and adverse events. We propose to rely on the siamese neural models we developed and trained on biomedical data, BioSTransformers, to identify new relevant relations between concepts and to create new semantic relations, the objective being to build a new merging ontology that could be used in applications. To validate the proposed approach and the new relations, we relied on the UMLS Metathesaurus and the Semantic Network. Our first results show promising improvements for future research.

Efficient Siamese model for visual object tracking with attention-based fusion modules

Efficient Siamese model for visual object tracking with attention-based fusion modules

Simple Siamese Model with Long Short-Term Memory for User Authentication with Field-Programmable Gate Arrays

Recent studies have focused on user authentication methods that use biometric signals such as electrocardiogram (ECG) and photo-plethysmography (PPG). These authentication technologies have advantages such as ease of acquisition, strong security, and the capability for non-aware authentication. This study addresses user authentication using electromyogram (EMG) signals, which are particularly easy to acquire, can be fabricated in a wearable form such as a wristwatch, and are readily expandable with technologies such as human–machine interface. However, despite their potential, they often exhibit lower accuracy (approximately 90%) than traditional methods such as fingerprint recognition. Accuracy can be improved using complex algorithms and multiple biometric authentication technologies; however, complex algorithms use substantial hardware resources, making their application to wearable devices difficult. In this study, a simple Siamese model with long short-term memory (LSTM) (SSiamese-LSTM) is proposed to achieve a high accuracy of over 99% with limited resources suitable for wearable devices. The hardware implementation was accomplished using field-programmable gate arrays (FPGAs). In terms of accuracy, EMG measurement results from Chosun University were used, and data from 100 individuals were employed for verification. The proposed digital logic will be integrated with an EMG sensor in the form of a watch that can be used for user authentication.

A Siamese network optimization using genetic algorithm for brain diseases

AbstractThe complex nature of human brain tissues is important in ensuring accurate diagnosis to save human lives. Research on early detection of brain diseases has gained significant prominence within medical intelligence using highly complex model architectures with only a single label that cannot be verified. This paper introduces an innovative approach to Siamese Network Genetic Algorithm (SN‐GA) leveraging Siamese contrastive learning for classifying brain images across diverse diseases. Our core architecture is a Bi‐Convolutional Neural Network (Bi‐CNN) optimized by a genetic algorithm to enhance brain image classification. Specifically, five widely recognized transfer learning‐based architectures, namely AlexNet, Efficient‐B0, VGG‐16, ResNet‐50, and Inception‐v3, have been incorporated to evaluate the effectiveness of the proposed SN‐GA system. The performance of these models has been rigorously analyzed and compared using two distinct datasets: Brain tumors and Alzheimer's datasets. The experimental results robustly affirm the efficacy of the proposed Siamese model, yielding exceptional levels of accuracy, precision, and recall, all peaking at 97%. These findings underscore the potential and resilience of the optimized Siamese network in the context of brain disease classification, emphasizing its significance in advancing the field of medical imaging and diagnosis, with implications for early intervention and patient care.

Intelligent Packet Priority Module for a Network of Unmanned Aerial Vehicles Using Manhattan Long Short-Term Memory

Unmanned aerial vehicles (UAVs) are becoming more common in wireless communication networks. Using UAVs can lead to network problems. An issue arises when the UAVs function in a network-access-limited environment with nodes causing interference. This issue could potentially hinder UAV network connectivity. This paper introduces an intelligent packet priority module (IPPM) to minimize network latency. This study analyzed Network Simulator–3 (NS-3) network modules utilizing Manhattan long short-term memory (MaLSTM) for packet classification of critical UAV, ground control station (GCS), or interfering nodes. To minimize network latency and packet delivery ratio (PDR) issues caused by interfering nodes, packets from prioritized nodes are transmitted first. Simulation results and evaluation show that our proposed intelligent packet priority module (IPPM) method outperformed previous approaches. The proposed IPPM based on MaLSTM implementation for the priority packet module led to a lower network delay and a higher packet delivery ratio. The performance of the IPPM averaged 62.2 ms network delay and 0.97 packet delivery ratio (PDR). The MaLSTM peaked at 97.5% accuracy. Upon further evaluation, the stability of LSTM Siamese models was observed to be consistent across diverse similarity functions, including cosine and Euclidean distances.

Application and analysis of face matching based on the Siamese model in face recognition

In recent years, face recognition technology, and face matching particularly have broadened the application fields in various aspects of society. It is considered a combination of deep learning architecture and face recognition technology, which has been used for personal information security and safety efficiently for many years. For this, this paper aims to investigate the practical method of utilizing Siamese models to enhance the accuracy and efficiency of face matching systems. The existing challenges of low accuracy and slow recognition rates in face matching applications have been approved to be solvable by utilizing the capabilities of the Siamese model. Experimental analysis and comments from relevant practitioners demonstrate the effectiveness and potential of the Siamese model in enhancing the performance of face matching systems. To conclude, the Siamese model is introduced as a robust and efficient tool in the field of face recognition. It provides higher accuracy and efficiency compared to the traditional feature-based models. Its adaptability and advancements bring the potential to revolutionize face-matching applications and overcome current limitations. The findings from the experiments demonstrate that the utilization of the joint model can significantly enhance the performance of the matching system. The proposed model offers a potential solution to address the issue of low accuracy during the face matching phase.

Siamese model for collateral score prediction from computed tomography angiography images in acute ischemic stroke.

Imaging biomarkers, such as the collateral score as determined from Computed Tomography Angiography (CTA) images, play a role in treatment decision making for acute stroke patients. In this manuscript, we present an end-to-end learning approach for automatic determination of a collateral score from a CTA image. Our aim was to investigate whether such end-to-end learning approaches can be used for this classification task, and whether the resulting classification can be used in existing outcome prediction models. The method consists of a preprocessing step, where the CTA image is aligned to an atlas and divided in the two hemispheres: the affected side and the healthy side. Subsequently, a VoxResNet based convolutional neural network is used to extract features at various resolutions from the input images. This is done by using a Siamese model, such that the classification is driven by the comparison between the affected and healthy using a unique set of features for both hemispheres. After masking the resulting features for both sides with the vascular region and global average pooling (per hemisphere) and concatenation of the resulting features, a fully connected layer is used to determine the categorized collateral score. Several experiments have been performed to optimize the model hyperparameters and training procedure, and to validate the final model performance. The hyperparameter optimization and subsequent model training was done using CTA images from the MR CLEAN Registry, a Dutch multi-center multi-vendor registry of acute stroke patients that underwent endovascular treatment. A separate set of images, from the MR CLEAN Trial, served as an external validation set, where collateral scoring was assessed and compared with both human observers and a recent more traditional model. In addition, the automated collateral scores have been used in an existing functional outcome prediction model that uses both imaging and non-imaging clinical parameters. The results show that end-to-end learning of collateral scoring in CTA images is feasible, and does perform similar to more traditional methods, and the performance also is within the inter-observer variation. Furthermore, the results demonstrate that the end-to-end classification results also can be used in an existing functional outcome prediction model.

Span prompt dense passage retrieval for Chinese open domain question answering

Dense passage retrieval is a popular method in information retrieval recently, especially in open domain question answering. It aims to retrieve related articles from massive passages to answer the question. Retriever can increase retrieval speed with less loss of accuracy compared to other methods. However, the pretrained language models used in recent research are often ineffective in semantic embedding, which will reduce accuracy. In addition, we find that contrastive learning will diverge the representation space, and Siamese models with independent parameters on both sides will decrease generalization performance. Therefore, we propose span prompt dense passage retrieval (SPDPR) based on span mask prompt tuning and parameter sharing in Chinese open-domain dense retrieval. This model can generate more efficient representation embeddings and effectively counteract the separation tendency between positive samples. We evaluate the effectiveness of SPDPR in DYKzh, as well as two Chinese datasets. SPDPR surpasses all SOTAs implemented in DYKzh and achieves a competitive result in other datasets.

Real-time Face Recognition System Using Deep Learning Method

Face recognition is one of the most popular methods currently used for biometric systems. The selection of a suitable method greatly affects the reliability of the biometrics system. This research will use Deep learning to improve the reliability of the biometric system and will compare it with the SVM method. The Deep Learning method will be adopted using the Siamese Network with the YoloV5 detection method as a real-time face detector. There are two stages in this research: the registration process and the recognition process. The registration process is image acquisition using YoloV5. The image result will be saved in the storage folder, and the preprocessing and training process will use the Siamese Network. The face feature model will be stored in the database. The recognition process is the same as the registration, but the feature extraction result will be embedded and compared with the already trained models. The accuracy rate using the Siamese model was 94%.

Spatiotemporal Mammography-based Deep Learning Model for Improved Breast Cancer Risk Prediction.

Breast cancer is one of the most prevalent cancers among women. It is the second leading cause of death in cancer-related deaths. Early detection and personalized risk assessment can reduce the mortality rate and improve survival rates. Classical risk prediction models which rely on traditional risk factors produce inconsistent results among the different populations. Thus, they are not routinely used in screening programs. Deep learning was proven to improve the results of breast cancer risk prediction. CNNs can detect risk cues from screening mammograms. However, the deep learning models utilize the spatial information of each screening mammogram independently. This study aims to further improve the risk prediction models by exploiting the spatiotemporal information in multiple screening time points. We implemented a Siamese neural network for spatiotemporal risk prediction and compared the results against CNN trained using two different time points (T1 and T2) independently. We tested our results on 191 cases, 61 of which were diagnosed with cancer. The Siamese model showed a superior AUC of 0.81 against 0.75 and 0.77 at T1 and T2 respectively. The Siamese network also exhibited higher accuracy and F1-score with values of 0.78 and 0.61 while CNNs have the same accuracy of 0.76 with an F1-score of 0.54 at T1, and 0.59 at T2. The results suggest that spatiotemporal risk prediction can be a more reliable risk assessment tool.

SiamixFormer: a fully-transformer Siamese network with temporal Fusion for accurate building detection and change detection in bi-temporal remote sensing images

ABSTRACT Building detection and change detection using remote sensing images can help urban and rescue planning. Moreover, they can be used for building damage assessment after natural disasters. Currently, most of the existing models for building detection use only one image (pre-disaster image) to detect buildings. This is based on the idea that post-disaster images reduce the model’s performance because of the presence of destroyed buildings. In this paper, we propose a siamese model, called SiamixFormer, which uses pre- and post-disaster images as input. Our model has two encoders and has a hierarchical transformer architecture. The output of each stage in both encoders is given to a temporal transformer for feature fusion in a way that query is generated from pre-disaster images and (key, value) is generated from post-disaster images. To this end, temporal features are also considered in feature fusion. Another advantage of using temporal transformers in feature fusion is that they can better maintain large receptive fields generated by transformer encoders compared with CNNs. Finally, the output of the temporal transformer is given to a simple MLP decoder at each stage. The SiamixFormer model is evaluated on xBD, and WHU datasets, for building detection and on LEVIR-CD and CDD datasets for change detection and could outperform the state-of-the-art.

Deep learning-based markerless lung tumor tracking in stereotactic radiotherapy using Siamese networks.

Radiotherapy (RT) is involved in about 50% of all cancer patients, making it a very important treatment modality. The most common type of RT is external beam RT, which consists of delivering the radiation to the tumor from outside the body. One novel treatment delivery method is volumetric modulated arc therapy (VMAT), where the gantry continuously rotates around the patient during the radiation delivery. Accurate tumor position monitoring during stereotactic body radiotherapy (SBRT) for lung tumors can help to ensure that the tumor is only irradiated when it is inside the planning target volume. This can maximize tumor control and reduce uncertainty margins, lowering organ-at-risk dose. Conventional tracking methods are prone to errors, or have a low tracking rate, especially for small tumors that are in close vicinity to bonystructures. We investigated patient-specific deep Siamese networks for real-time tumor tracking, during VMAT. Due to lack of ground truth tumor locations in the kilovoltage (kV) images, each patient-specific model was trained on synthetic data (DRRs), generated from the 4D planning CT scans, and evaluated on clinical data (x-rays). Since there are no annotated datasets with kV images, we evaluated the model on a 3D printed anthropomorphic phantom but also on six patients by computing the correlation coefficient with the breathing-related vertical displacement of the surface-mounted marker (RPM). For each patient/phantom, we used 80% of DRRs for training and 20% forvalidation. The proposed Siamese model outperformed the conventional benchmark template matching-based method (RTR): (1) when evaluating both methods on the 3D phantom, the Siamese model obtained a 0.57-0.79-mm mean absolute distance to the ground truth tumor locations, compared to 1.04-1.56 mm obtained by RTR; (2) on patient data, the Siamese-determined longitudinal tumor position had a correlation coefficient of 0.71-0.98 with the RPM, compared to 0.07-0.85 for RTR; (3) the Siamese model had a 100% tracking rate, compared to 62%-82% forRTR. Based on these results, we argue that Siamese-based real-time 2D markerless tumor tracking during radiation delivery is possible. Further investigation and development of 3D tracking iswarranted.

SPT: Single Pedestrian Tracking Framework with Re-Identification-Based Learning Using the Siamese Model.

Pedestrian tracking is a challenging task in the area of visual object tracking research and it is a vital component of various vision-based applications such as surveillance systems, human-following robots, and autonomous vehicles. In this paper, we proposed a single pedestrian tracking (SPT) framework for identifying each instance of a person across all video frames through a tracking-by-detection paradigm that combines deep learning and metric learning-based approaches. The SPT framework comprises three main modules: detection, re-identification, and tracking. Our contribution is a significant improvement in the results by designing two compact metric learning-based models using Siamese architecture in the pedestrian re-identification module and combining one of the most robust re-identification models for data associated with the pedestrian detector in the tracking module. We carried out several analyses to evaluate the performance of our SPT framework for single pedestrian tracking in the videos. The results of the re-identification module validate that our two proposed re-identification models surpass existing state-of-the-art models with increased accuracies of 79.2% and 83.9% on the large dataset and 92% and 96% on the small dataset. Moreover, the proposed SPT tracker, along with six state-of-the-art (SOTA) tracking models, has been tested on various indoor and outdoor video sequences. A qualitative analysis considering six major environmental factors verifies the effectiveness of our SPT tracker under illumination changes, appearance variations due to pose changes, changes in target position, and partial occlusions. In addition, quantitative analysis based on experimental results also demonstrates that our proposed SPT tracker outperforms the GOTURN, CSRT, KCF, and SiamFC trackers with a success rate of 79.7% while beating the DiamSiamRPN, SiamFC, CSRT, GOTURN, and SiamMask trackers with an average of 18 tracking frames per second.

EFFECT OF TRANSFERRING PRE-TRAINED WEIGHTS ON A SIAMESE CHANGE DETECTION NETWORK

Abstract. Change Detection (CD) is one of the most crucial applications in remote sensing which identifies meaningful changes from bitemporal images taken from the same location. Enhancing the temporal efficiency and accuracy of this task is of great importance and one way to achieve this is through transfer learning. In this study, we investigate the influence of transferring pre-trained weights on the performance of a Siamese CD network using a benchmark dataset. For this purpose, an autoencoder with the same encoder architecture as in the Siamese model is trained on the whole dataset. Then, the encoder weights are transferred from the autoencoder and the Siamese model is trained in two modes. In the first mode, the transferred weights are frozen and only the decoder section of the Siamese models is trained while the second mode trains the whole model without freezing any part of the model. Moreover, the Siamese model is also trained without using the pre-trained weights to set the basis for comparisons. The results indicate that freezing the encoder results in a relatively lower performance but offers a considerable amount of temporal efficiency in the training phase. On the other hand, training the whole model after the weight transfer acquires the best result with an improvement of 12.43% in the Intersection over Union (IoU) metric.

Ensemble learning for glioma patients overall survival prediction using pre-operative MRIs

Objective: Gliomas are the most common primary brain tumors. Approximately 70% of the glioma patients diagnosed with glioblastoma have an averaged overall survival (OS) of only ∼16 months. Early survival prediction is essential for treatment decision-making in glioma patients. Here we proposed an ensemble learning approach to predict the post-operative OS of glioma patients using only pre-operative MRIs. Approach: Our dataset was from the Medical Image Computing and Computer Assisted Intervention Brain Tumor Segmentation challenge 2020, which consists of multimodal pre-operative MRI scans of 235 glioma patients with survival days recorded. The backbone of our approach was a Siamese network consisting of twinned ResNet-based feature extractors followed by a 3-layer classifier. During training, the feature extractors explored traits of intra and inter-class by minimizing contrastive loss of randomly paired 2D pre-operative MRIs, and the classifier utilized the extracted features to generate labels with cost defined by cross-entropy loss. During testing, the extracted features were also utilized to define distance between the test sample and the reference composed of training data, to generate an additional predictor via K-NN classification. The final label was the ensemble classification from both the Siamese model and the K-NN model. Main results: Our approach classifies the glioma patients into 3 OS classes: long-survivors (>15 months), mid-survivors (between 10 and 15 months) and short-survivors (<10 months). The performance is assessed by the accuracy (ACC) and the area under the curve (AUC) of 3-class classification. The final result achieved an ACC of 65.22% and AUC of 0.81. Significance: Our Siamese network based ensemble learning approach demonstrated promising ability in mining discriminative features with minimal manual processing and generalization requirement. This prediction strategy can be potentially applied to assist timely clinical decision-making.

Furniture Style Compatibility Estimation by Multi-Branch Deep Siamese Network

As demands for understanding visual style among interior scenes increase, estimating style compatibility is becoming challenging. In particular, furniture styles are difficult to define due to their various elements, such as color and shape. As a result, furniture style is an ambiguous concept. To reduce ambiguity, Siamese networks have frequently been used to estimate style compatibility by adding various features that represent the style. However, it is still difficult to accurately represent a furniture’s style, even when using alternate features associated with the images. In this paper, we propose a new Siamese model that can learn from several furniture images simultaneously. Specifically, we propose a one-to-many ratio input method to maintain high performance even when inputs are ambiguous. We also propose a new metric for evaluating Siamese networks. The conventional metric, the area under the ROC curve (AUC), does not reveal the actual distance between styles. Therefore, the proposed metric quantitatively evaluates the distance between styles by using the distance between the embedding of each furniture image. Experiments show that the proposed model improved the AUC from 0.672 to 0.721 and outperformed the conventional Siamese model in terms of the proposed metric.

Textline alignment on the image domain

Editing and publishing a historical manuscript involves a research phase to recover the original manuscript and reconstruct the transmission of its text based on the relations between its surviving copies. Manuscript alignment, which aims to locate the shared and the different text among a set of copies of the same manuscript, is essential for this phase. In this paper, we present an alignment algorithm for historical handwritten documents that works directly on the image domain due to the absence of an accurate handwritten text recognition (HTR) system for handwritten historical documents and the necessity to visualize the original manuscripts in parallel to examine features beyond the transcribed text. Our approach extracts subwords, estimates the similarity among these subwords, and establishes an alignment among them. We extract subwords from textlines images and convert them into sequences of subword images. It estimates the similarity between two subwords using a Siamese network model and applies Longest Common Subsequence (LCS) to establish the alignment between two image sequences. We have implemented our algorithm, trained the Siamese model, and evaluate its performance using textline images from historical documents. Our algorithm outperformed the state-of-the-art by large margins. Unlike the state-of-the-art, the framework builds the alignment from scratch without requiring any prior knowledge concern subwords boundaries. In addition, we build a new dataset for textline alignment for historical documents, which include ten pairs of pages taken from two copies of two Arabic manuscripts and annotated at the subword level.

Bilateral Analysis Boosts the Performance of Mammography-based Deep Learning Models in Breast Cancer Risk Prediction.

Breast cancer is one of the leading causes of death among women. Early prediction of breast cancer can significantly improve the survival rates. Breast density was proven as a reliable risk factor. Deep learning models can learn subtle cues in the mammogram images. CNN models were recently shown to improve the risk discrimination in full-field mammograms. This study aims to improve risk prediction models using bilateral analysis. Bilateral analysis is the process of comparing two breasts to verify presence of anomalies. We developed a Siamese neural network to leverage the bilateral information and asymmetries between the two mammograms of the same patient. We tested our model on 271 patients and compared the results of our Siamese model against the traditional unilateral CNN model. Our results showed AUCs of 0.75 and 0.70 respectively (p = 0.0056). The Siamese model also exhibits higher sensitivity, specificity, precision, and false positive rate with values of 0.68, 0.69, 0.71, 0.31 respectively. While the CNN values were 0.61, 0.66, 0.67, 0.34 respectively. We merged both models by two techniques using pre-trained weights and weighted voting ensemble. The merging technique boosted the AUC to 0.78. The results suggest that bilateral analysis can significantly improve the risk discrimination.

Cross-Domain Few-Shot Learning Approach for Lithium-Ion Battery Surface Defects Classification Using an Improved Siamese Network

It is difficult to detect the surface defects of a lithium battery with an aluminum/steel shell. The reflectivity, lack of 3D information on the battery surface, and the shortage of many datasets make the 2D detection method hard to apply in this field. In this paper, a cross-domain few-shot learning (FSL) approach for lithium-ion battery defect classification using an improved siamese network (BSR-SNet) is proposed. To obtain the critical 3D surface of the lithium-ion battery, a multiexposure-based structured light method is utilized. Then, the heights of the 3D cloud points are transferred to grayscale information and are saved as 8-bit 2D images. For the FSL task, the DAGM 2007 datasets are used as the source domain to pre-train the improved siamese model. To avoid negative mitigation in the target domain, batch spectral regularization (BSR) is added as a penalizer in the loss function. The accuracies of the experimental results are 93.3% for 10-shot batteries and 91.0% for 5-shot batteries, which means that our method can be used to classify the surface defects of lithium batteries well.

BUILDING DAMAGE ASSESSMENT WITH DEEP LEARNING

Abstract. Global warming modifies the climate balance. Warming parameters are observed by many Earth Observation satellite systems, and the huge amount of data modifies the way to process them. This paper presents a few studies relative to damage detection on buildings, occurred during natural disasters. Recent advances in deep learning techniques are used for the building detection such as EfficientNet networks. Additional networks as Siamese models are used to evaluate the damage level with pre- and post-event images. Different techniques to merge detection masks are described and compared to a multiclass segmentation network. Results are presented and performances of the different solutions are compared.