Triplet Network Research Articles

WKNN-Based Wi-Fi Fingerprinting with Deep Distance Metric Learning via Siamese Triplet Network for Indoor Positioning

Weighted k-nearest neighbor (WKNN)-based Wi-Fi fingerprinting is popular in indoor location-based services due to its ease of implementation and low computational cost. KNN-based methods rely on distance metrics to select the nearest neighbors. However, traditional metrics often fail to capture the complexity of indoor environments and have limitations in identifying non-linear relationships. To address these issues, we propose a novel WKNN-based Wi-Fi fingerprinting method that incorporates distance metric learning. In the offline phase, our method utilizes a Siamese network with a triplet loss function to learn a meaningful distance metric from training fingerprints (FPs). This process employs a unique triplet mining strategy to handle the inherent noise in FPs. Subsequently, in the online phase, the learned metric is used to calculate the embedding distance, followed by a signal-space distance filtering step to optimally select neighbors and estimate the user’s location. The filtering step mitigates issues from an overfitted distance metric influenced by hard triplets, which could lead to incorrect neighbor selection. We evaluate the proposed method on three benchmark datasets, UJIIndoorLoc, Tampere, and UTSIndoorLoc, and compare it with four WKNN models. The results show a mean positioning error reduction of 3.55% on UJIIndoorLoc, 16.21% on Tampere, and 16.49% on UTSIndoorLoc, demonstrating enhanced positioning accuracy.

Chasing the Neutrino Blazar Candidates

In our study of the correlations between IceCube-detected neutrino events and γ-ray properties of blazars, we recognize the inherent challenges posed by the limited detection of neutrinos. In this paper, we explore few-shot learning to deal with the class imbalance and few-shot issues presented in the incremental version of the 12 yr Fermi-LAT γ-ray source catalog (4FGL_ DR3). Specifically, we train a triplet network to transform the blazars with neutrino emission (NBs) and nonblazar samples into an embedding space where their similarities can be measured. With two-way three-shot learning, 199 out of 3708 blazars without neutrino emission (non-NBs) are considered as the potential blazars emitting neutrinos (NB candidates, or NBCs for short), with a similarity score against NBs exceeding 98%. Moreover, the Kolmogorov–Smirnov test supports our identification of NBCs.

Prior semantic-embedding representation learning for on-the-fly FG-SBIR

In fine-grained sketch-based image retrieval (FG-SBIR) framework, sketch drawing is a time-consuming and skill-intensive process that greatly restricts its practical application. Our study aims to enhance the early-stage retrieval efficiency by utilizing partial sketches with a minimum number of strokes in the on-the-fly FG-SBIR framework. Nevertheless, partial sketches with few strokes usually lack global information and consist only of sparse local details. This lack of detail results in ambiguous semantic content and indistinct differences among images, which consequently impairs early-stage retrieval. To acquire an efficient representation for these sketches, we propose a multi-granularity feature representation model that utilizes prior knowledge embedding to deal with the sketch drawing process. In the first stage,we design a triplet network to learn the joint a coarse representation embedding space that is common between the complete sketch and the image. In the second stage, We enhanced the semantic representation of sketches by utilizing prior knowledge, which aids in supplementing missing fine-grained features. This was achieved by formalizing the prior knowledge and integrating it into the feature vector obtained in the initial stage, thereby acquiring the final representation of a subset of sketches. Experiments conducted on two public datasets indicate that our method outperforms the state-of-the-art baseline during the early retrieval phase. In practical applications, our approach has also achieved good results.

Unsupervised Content Mining in CBIR: Harnessing Latent Diffusion for Complex Text-Based Query Interpretation.

The paper demonstrates a novel methodology for Content-Based Image Retrieval (CBIR), which shifts the focus from conventional domain-specific image queries to more complex text-based query processing. Latent diffusion models are employed to interpret complex textual prompts and address the requirements of effectively interpreting the complex textual query. Latent Diffusion models successfully transform complex textual queries into visually engaging representations, establishing a seamless connection between textual descriptions and visual content. Custom triplet network design is at the heart of our retrieval method. When trained well, a triplet network will represent the generated query image and the different images in the database. The cosine similarity metric is used to assess the similarity between the feature representations in order to find and retrieve the relevant images. Our experiments results show that latent diffusion models can successfully bridge the gap between complex textual prompts for image retrieval without relying on labels or metadata that are attached to database images. This advancement sets the stage for future explorations in image retrieval, leveraging the generative AI capabilities to cater to the ever-evolving demands of big data and complex query interpretations.

Advanced hyperspectral image classification via adaptive triplet networks and chaotic quasi oppositional optimization

Advanced hyperspectral image classification via adaptive triplet networks and chaotic quasi oppositional optimization

Handwritten Signature Verification System Using User-dependent Approach: A Comparative Study

Handwritten signature verification is a critical aspect of personal authentication in numerous sectors, including banking, legal, and access control systems. This paper provides a comprehensive investigation into writer-dependent signature verification techniques, spanning from traditional approaches such as Histogram of Oriented Gradients (HOG) and Scale-Invariant Feature Transform (SIFT) to cutting-edge deep learning architectures like Siamese and triplet networks. Through extensive experimentation and meticulous analysis conducted on benchmark datasets, we rigorously compare and evaluate the efficacy of these approaches. Our study reveals intriguing insights into the performance characteristics of different methodologies, shedding light on their strengths, weaknesses, and areas for improvement. Notably, we observe that while traditional feature-based methods excel in capturing discriminative information from signature images, deep learning architectures offer enhanced flexibility and adaptability, particularly in scenarios with large and diverse datasets. By combining the strengths of both paradigms, we demonstrate the potential for building robust and reliable signature verification systems capable of handling real-world challenges and variations. We highlight the importance of carefully curated datasets and well-designed network architectures in achieving optimal results, thereby providing valuable guidelines for practitioners and researchers in the field. In conclusion, this study contributes significant insights to the advancement of handwritten signature verification systems, emphasizing the importance of leveraging both traditional methodologies and modern deep learning techniques for building secure and dependable authentication mechanisms. By addressing key challenges and exploring novel approaches, our research aims to foster innovation and enhance the security measures associated with handwritten signature authentication in diverse applications. Keywords—Handwritten signature verification, Writer-dependent approach, Feature-based methods, Deep learning, Siamese networks, Triplet networks, Benchmark datasets, Performance evaluation, Authentication mechanisms, Security measures.

Triplet Network and Unsupervised-Clustering-Based Zero-Shot Radio Frequency Fingerprint Identification With Extremely Small Sample Size

Triplet Network and Unsupervised-Clustering-Based Zero-Shot Radio Frequency Fingerprint Identification With Extremely Small Sample Size

Deep learning-based retinal abnormality detection from OCT images with limited data

In the realm of medical diagnosis, the challenge posed by retinal diseases is considerable, given their potential to complicate vision and overall ocular health. A promising avenue for achieving highly accurate classifiers in detecting retinal diseases involves the application of deep learning models. However, overfitting issues often undermine the performance of these models due to the scarcity of image samples in retinal disease datasets. To address this challenge, a novel deep triplet network is proposed as a metric learning approach for detecting retinal diseases using Optical Coherence Tomography (OCT) images. Incorporating a conditional loss function tailored to the constraints of limited data samples, this deep triplet network enhances the model’s accuracy. Drawing inspiration from pre-trained models such as VGG16, the foundational architecture of our model is established. Experiments use open-access datasets comprising retinal OCT images to validate our proposed approach. The performance of the suggested model is demonstrated to surpass that of state-of-the-art models in terms of accuracy. This substantiates the effectiveness of the deep triplet network in addressing overfitting issues associated with limited data samples in retinal disease datasets.

Non-invasive load monitoring based on deep learning to identify unknown loads.

With the rapid development of smart grids, society has become increasingly urgent to solve the problems of low energy utilization efficiency and high energy consumption. In this context, load identification has become a key element in formulating scientific and effective energy consumption plans and reducing unnecessary energy waste. However, traditional load identification methods mainly focus on known electrical equipment, and accurate identification of unknown electrical equipment still faces significant challenges. A new encoding feature space based on Triplet neural networks is proposed in this paper to detect unknown electrical appliances using convex hull coincidence degree. Additionally, transfer learning is introduced for the rapid updating of the pre-classification model's self-incrementing class with the unknown load. In experiments, the effectiveness of our method is successfully tested on the PLAID dataset. The accuracy of unknown load identification reached 99.23%. Through this research, we expect to bring a new idea to the field of load identification to meet the urgent need for the identification of unknown electrical appliances in the development of smart grids.

FlowCorrGCN: Enhancing Flow Correlation Through Graph Convolutional Networks and Triplet Networks

Anonymous network tracing is a significant research subject in the field of network security, and flow correlation technology serves as a fundamental technique for deanonymizing network traffic. Existing flow correlation techniques are considered ineffective and unreliable when applied on a large scale because they exhibit high false‐positive rates or require impractically long periods of traffic observation to achieve reliable correlations. To address this issue, this paper proposed an innovative flow correlation approach for the typical and most widely used Tor anonymous network by combining graph convolutional neural networks with triplet networks. Our proposed method involves extracting features such as packet intervals, packet lengths, and directions from Tor network traffic and encoding each flow into a graph representation. The integration of triplet networks enhances the internode relationships, which can effectively fuse flow representations with node associations. The graph convolutional neural network extracts features from the input graph topology, mapping them to distinct representations in the embedding space, thus effectively distinguishing different Tor flows. Experimental results demonstrate that with a false‐positive rate as low as 0.1%, the correlation accuracy reaches 86.4%, showcasing a 5.1% accuracy improvement compared to the existing state‐of‐the‐art methods.

Transfer Learning and Few-Shot Learning Based Deep Neural Network Models for Underwater Sonar Image Classification With a Few Samples

Acoustic imaging sonar systems are widely used for long-range underwater surveillance in various civilian and military applications. They provide 2-D images of underwater objects, even in turbid water conditions where optical underwater imaging systems fail. Achieving high accuracy in automatic deep learning based underwater image classification remains an open problem due to insufficient data availability, poor image resolution, low signal-to-noise ratio surroundings, etc. In this study, we conduct a comparative analysis of different advanced deep learning approaches, i.e., transfer learning and few-shot learning, to address the problem of automatic object classification in sonar images, using a few samples of data. Specifically, two metric learning-based approaches, i.e., siamese network and triplet network as well as library-based approaches, are studied under the few-shot learning paradigm. Extensive experiments are conducted on a novel custom-made dataset developed in-house, along with the publicly available SeabedObjectsKLSG dataset. In addition, the effectiveness of the sampling technique in handling class imbalance during model training is also investigated in this work. Our experimental results highlight that the few-shot learning based approach is a promising direction for future research on underwater image classification with a few samples.

Self-Supervised Enhancement for Named Entity Disambiguation via Multimodal Graph Convolution.

Named entity disambiguation (NED) finds the specific meaning of an entity mention in a particular context and links it to a target entity. With the emergence of multimedia, the modalities of content on the Internet have become more diverse, which poses difficulties for traditional NED, and the vast amounts of information make it impossible to manually label every kind of ambiguous data to train a practical NED model. In response to this situation, we present MMGraph, which uses multimodal graph convolution to aggregate visual and contextual language information for accurate entity disambiguation for short texts, and a self-supervised simple triplet network (SimTri) that can learn useful representations in multimodal unlabeled data to enhance the effectiveness of NED models. We evaluated these approaches on a new dataset, MMFi, which contains multimodal supervised data and large amounts of unlabeled data. Our experiments confirm the state-of-the-art performance of MMGraph on two widely used benchmarks and MMFi. SimTri further improves the performance of NED methods. The dataset and code are available at https://github.com/LanceZPF/NNED_MMGraph.

Fault Tolerance in Triplet Network Training: Analysis, Evaluation and Protection Methods

Fault Tolerance in Triplet Network Training: Analysis, Evaluation and Protection Methods

A Triplet Network Fusing Optical and SAR Images for Colored Steel Building Extraction.

The identification of colored steel buildings in images is crucial for managing the construction sector, environmental protection, and sustainable urban development. Current deep learning methods for optical remote sensing images often encounter challenges such as confusion between the roof color or shape of regular buildings and colored steel structures. Additionally, common semantic segmentation networks exhibit poor generalization and inadequate boundary regularization when extracting colored steel buildings. To overcome these limitations, we utilized the metal detection and differentiation capabilities inherent in synthetic aperture radar (SAR) data to develop a network that integrates optical and SAR data. This network, employing a triple-input structure, effectively captures the unique features of colored steel buildings. We designed a multimodal hybrid attention module in the network that discerns the varying importance of each data source depending on the context. Additionally, a boundary refinement (BR) module was introduced to extract the boundaries of the colored steel buildings in a more regular manner, and a deep supervision strategy was implemented to improve the performance of the network in the colored steel building extraction task. A BR module and deep supervision strategy were also implemented to sharpen the extraction of building boundaries, thereby enhancing the network's accuracy and adaptability. The results indicate that, compared to mainstream semantic segmentation, this method effectively enhances the precision of colored steel building detection, achieving an accuracy rate of 83.19%. This improvement marks a significant advancement in monitoring illegal constructions and supporting the sustainable development of the Beijing-Tianjin-Hebei metropolitan region.

Triplet attention-enhanced residual tree-inspired decision network: A hierarchical fault diagnosis model for unbalanced bearing datasets

In fault classification tasks, deep neural networks (DNNs) have remarkable recognition performance. Nevertheless, the classification decision processes of DNNs lack hierarchical logical reasoning abilities and their diagnostic performance significantly deteriorates when dealing with imbalanced bearing fault datasets. To further address this issue, a novel model, termed the triplet attention-enhanced residual tree-inspired decision network (TARTDN) which is not a simple combination of the DNN and decision tree model, is developed to diagnose unbalanced bearing faults and provides a rational decision-making and reasoning process in this study. First, a triplet attention-enhanced residual network (TARN) is designed as the backbone network to capture key information more accurately. Second, a novel tree-inspired decision layer (TDL) is construed to infer and decide bearing data categories. Subsequently, the probability distribution values obtained by pre-training TARN are flowed into the TDL as thresholds for seed and leaf nodes. The parameters of the TARN are continuously updated with the node thresholds of the TDL, resulting in an integrated TARTDN model that combines high-quality feature extraction and inferable decision-making. In the end, the trained TARTDN progressively determines the fault types and severity levels in unbalanced bearing fault datasets. The developed model tested on two bearing fault datasets with three unbalanced ratios, has consistently achieved recognition rates exceeding 97.5%. The proposed approach has been validated through ablation experiments and comparisons with other advanced methods to exhibit higher recognition rates, superior hierarchical classification reasoning, and more stable generalization capabilities on unbalanced bearing datasets.

Data-to-text generation using conditional generative adversarial with enhanced transformer

Abstract In this paper, we propose an enhanced version of the vanilla transformer for data-to-text generation and then use it as the generator of a conditional generative adversarial model to improve the semantic quality and diversity of output sentences. Specifically, by adding a diagonal mask matrix to the attention scores of the encoder and using the history of the attention weights in the decoder, this enhanced version of the vanilla transformer prevents semantic defects in the output text. Also, using this enhanced transformer along with a triplet network, respectively, as the generator and discriminator of conditional generative adversarial network, diversity and semantic quality of sentences are guaranteed. To prove the effectiveness of the proposed model, called conditional generative adversarial with enhanced transformer (CGA-ET), we performed experiments on three different datasets and observed that our proposed model is able to achieve better results than the baselines models in terms of BLEU, METEOR, NIST, ROUGE-L, CIDEr, BERTScore, and SER automatic evaluation metrics as well as human evaluation.

Multi-Stage Approach Using Convolutional Triplet Network and Ensemble Model for Fault Diagnosis in Oil Plant Rotary Machines

Ensuring the operational safety and reliability of rotary machinery systems, especially in oil plants, has become a focal point in both academic and industry arenas. Specifically, in terms of key rotary machinery components such as shafts, the diagnosis of these systems is paramount for achieving enhanced generalization capabilities in fault diagnosis, encompassing multiple sensor-derived variables with their respective fault patterns. This study introduces a multi-stage approach to generalize capabilities for fault diagnosis that considers multiple sensor-derived variables and their fault patterns. This method combines the Convolutional Triplet Network for feature extraction with an ensemble model for fault classification. Initially, vibration signals are processed to yield the most representative temporal and spatial features. Then, an ensemble approach is used to maximize both diversity and accuracy by balancing the contributions of the individual classifiers. The approach can detect three representative types of shaft faults more accurately than traditional single-stage machine learning models. Comprehensive experiments, detailed within, showcase the method’s efficacy in diagnosing rotary machine faults across diverse operational scenarios.

Deep Learning Perspectives on Efficient Image Matching in Natural Image Databases

With the proliferation of digital content, efficient image matching in natural image databases has become paramount. Traditional image matching techniques, while effective to a certain extent, face challenges in dealing with the high variability inherent in natural images. This research delves into the application of deep learning models, particularly Convolutional Neural Networks (CNNs), Siamese Networks, and Triplet Networks, to address these challenges. We introduce various techniques to enhance efficiency, such as data augmentation, transfer learning, dimensionality reduction, efficient sampling, and the amalgamation of traditional computer vision strategies with deep learning. Our experimental results, garnered from specific dataset, demonstrate significant improvements in image matching efficiency, as quantified by metrics like precision, recall, F1-Score, and matching time. The findings underscore the potential of deep learning as a transformative tool for natural image database matching, setting the stage for further research and optimization in this domain.

Pseudo labelling workflow, margin losses, hard triplet mining, and PENViT backbone for explainable age and biological gender estimation using dental panoramic radiographs

This study aimed to estimate human age and gender from panoramic radiographs using various deep learning techniques while using explainability to have a novel hybrid unsupervised model explain the decision-making process. The classification task involved training neural networks and vision transformers on 706 panoramic radiographs using different loss functions and backbone architectures namely ArcFace, a triplet network named TriplePENViT, and the subsequently developed model called PENViT. Pseudo labeling techniques were applied to train the models using unlabeled data. FullGrad Explainable AI was used to gain insights into the decision-making process of the developed PENViT model. The ViT Large 32 model achieved a validation accuracy of 68.21% without ArcFace, demonstrating its effectiveness in the classification task. The PENViT model outperformed other backbones, achieving the same validation accuracy without ArcFace and an improved accuracy of 70.54% with ArcFace. The TriplePENViT model achieved a validation accuracy of 67.44% using hard triplet mining techniques. Pseudo labeling techniques yielded poor performance, with a validation accuracy of 64.34%. Validation accuracy without ArcFace was established at 67.44% for Age and 84.49% for gender. The unsupervised model considered developing tooth buds, tooth proximity and mandibular shape for estimating age within deciduous and mixed dentitions. For ages 20–29, it factored permanent dentition, alveolar bone density, root apices, and third molars. Above 30, it notes occlusal deformity resulting from missing dentition and the temporomandibular joint complex as predictors for age estimation from panoramic radiographs.Graphical abstract

RDLNet: A Regularized Descriptor Learning Network.

Local image descriptor learning has been instrumental in various computer vision tasks. Recent innovations lie with similarity measurement of descriptor vectors with metric learning for randomly selected Siamese or triplet patches. Local image descriptor learning focuses more on hard samples since easy samples do not contribute much to optimization. However, few studies focus on hard samples of image patches from the perspective of loss functions and design appropriate learning algorithms to obtain a more compact descriptor representation. This article proposes a regularized descriptor learning network (RDLNet) that makes the network focus on the learning of hard samples and compact descriptor with triplet networks. A novel hard sample mining strategy is designed to select the hardest negative samples in mini-batch. Then batch margin loss concerned with hard samples is adopted to optimize the distance of extreme cases. Finally, for a more stable network and preventing network collapsing, orthogonal regularization is designed to constrain convolutional kernels and obtain rich deep features. RDLNet provides a compact discriminative low-dimensional representation and can be embedded in other pipelines easily. This article gives extensive experimental results for large benchmarks in multiple scenarios and generalization in matching applications with significant improvements.