Instance Matching Research Articles

A Schema-Free Instance Matching Algorithm Based on Virtual Document Similarity

With the continuous development of semantic web, especially of the web of data, several knowledge bases expressed by ontologies are independently created and added to the Linked Open Data (LOD) cloud, on a daily basis. A major challenge for the LOD paradigm is to discover resources that refer to the same real-world object, in order to interlink web resources and hold large scale data integration and sharing. In this context, instance matching is a promising solution. It aims to link co-referent instances belonging to heterogeneous knowledge bases with owl:sameAs links. Several state-of-the-art existing approaches addressing this issue are based on the prior schema-level matchings, which does not avoid the limitation of heterogeneity at the property-level. In this paper, we propose a schema-free, scalable and efficient instance matching approach that is independent from matching results at the schema-level. We transform the instance matching problem to a document similarity problem and we solve it by a Clustering technique that uses an Ascendant Hierarchical Clustering algorithm to group similar instances in the same clusters. Furthermore, we design multiple validating patterns that use some structural information to validate obtained mappings and eliminate wrong ones. Experiments on instance matching track from Ontology Alignment Evaluation Initiative (OAEI) show that our approach gets prominent results compared to several participating systems in OAEI’2019, OAEI’2020 and OAEI’2021.

Birds of a Feather Flock Together: Category-Divergence Guidance for Domain Adaptive Segmentation

Unsupervised domain adaptation (UDA) aims to enhance the generalization capability of a certain model from a source domain to a target domain. Present UDA models focus on alleviating the domain shift by minimizing the feature discrepancy between the source domain and the target domain but usually ignore the class confusion problem. In this work, we propose an Inter-class Separation and Intra-class Aggregation (ISIA) mechanism. It encourages the cross-domain representative consistency between the same categories and differentiation among diverse categories. In this way, the features belonging to the same categories are aligned together and the confusable categories are separated. By measuring the align complexity of each category, we design an Adaptive-weighted Instance Matching (AIM) strategy to further optimize the instance-level adaptation. Based on our proposed methods, we also raise a hierarchical unsupervised domain adaptation framework for cross-domain semantic segmentation task. Through performing the image-level, feature-level, category-level and instance-level alignment, our method achieves a stronger generalization performance of the model from the source domain to the target domain. In two typical cross-domain semantic segmentation tasks, i.e., GTA 5→ Cityscapes and SYNTHIA → Cityscapes, our method achieves the state-of-the-art segmentation accuracy. We also build two cross-domain semantic segmentation datasets based on the publicly available data, i.e., remote sensing building segmentation and road segmentation, for domain adaptive segmentation. Our code, models and datasets are available at https://github.com/HibiscusYB/BAFFT.

4G-VOS: Video Object Segmentation using guided context embedding

Video Object Segmentation (VOS) is a fundamental task required in many high-level real-world computer vision applications. VOS becomes challenging due to the presence of background distractors as well as to object appearance variations. Many existing VOS approaches use online model updates to capture the appearance variations which incurs high computational cost. Template matching and propagation-based VOS methods, although cost-effective, suffer from performance degradation under challenging scenarios such as occlusion and background clutter. In order to tackle these challenges, we propose a network architecture dubbed 4G-VOS to encode video context for improved VOS performance to tackle these challenges. To preserve long term semantic information, we propose a guided transfer embedding module. We employ a global instance matching module to generate similarity maps from the initial image and the mask. Besides, we use a generative directional appearance module to estimate and dynamically update the foreground/background class probabilities in a spherical embedding space. Moreover, during feature refinement, existing approaches may lose contextual information. Therefore, we propose a guided pooled decoder to exploit the global and local contextual information during feature refinement. The proposed framework is an end-to-end learning architecture that is trained in an offline fashion. Evaluations over three VOS benchmark datasets including DAVIS2016, DAVIS2017, and YouTube-VOS have demonstrated outstanding performance of the proposed algorithm compared to 40 existing state-of-the-art methods.

Reusing Alignments for Discovering Instances Correspondences

Recently, instance matching has become a key technology to achieve interoperability over datasets, especially in linked data. Due the rapid growth of published datasets, it attracts increasingly more research interest. In this context, several approaches have been proposed. However, they do not perform well since the problem of matching instances that possess different descriptions is not addressed. On the other hand, the usage of the identity link owl:sameAs is generally predominant in linking correspondences. Unfortunately, many existing identity links are misused. In this paper, the authors discuss these issues and propose an original instance matching approach aiming to match instances that hold diverse descriptions. Furthermore, a novel link named ViewSameAs is proposed. The key improvement compared to existing approaches is alignment reuse. Thus, two novel methods are introduced: ViewSameAs-based clustering and alignment reuse based on metadata. Experiments on datasets by considering those of OAEI show that the proposed approach achieves satisfying and highly accuracy results.

FTRLIM: Distributed Instance Matching Framework for Large-Scale Knowledge Graph Fusion.

Instance matching is a key task in knowledge graph fusion, and it is critical to improving the efficiency of instance matching, given the increasing scale of knowledge graphs. Blocking algorithms selecting candidate instance pairs for comparison is one of the effective methods to achieve the goal. In this paper, we propose a novel blocking algorithm named MultiObJ, which constructs indexes for instances based on the Ordered Joint of Multiple Objects’ features to limit the number of candidate instance pairs. Based on MultiObJ, we further propose a distributed framework named Follow-the-Regular-Leader Instance Matching (FTRLIM), which matches instances between large-scale knowledge graphs with approximately linear time complexity. FTRLIM has participated in OAEI 2019 and achieved the best matching quality with significantly efficiency. In this research, we construct three data collections based on a real-world large-scale knowledge graph. Experiment results on the constructed data collections and two real-world datasets indicate that MultiObJ and FTRLIM outperform other state-of-the-art methods.

A Graph based Data Integration and Aggregation Technique for Big Data

Data integration is a vital issue in the conditions of heterogeneous data sources. As of now, the in advance of referenced heterogeneity is getting boundless. In view of different data sources, in the event that we need to acquire valuable information and knowledge, we should take care of data integration issues to apply suitable insightful techniques to extensive and uniform data. All the more especially, we propose a novel engineering for instance matching that considers the particularities of this heterogeneous and conveyed setting. Rather than expecting that instances share a similar schema, the proposed technique works in any event, when there is no cover between schema, aside from a key name that matching instances should share. In addition, we have thought about the conveyed idea of the Semantic Web to propose another design for general data integration. The arrangement consolidates ETL innovation and a wrapper layer known from intervened frameworks. It additionally gives semantic integration through association component between data components. The outcomes accomplished in this work are especially intriguing for the Semantic Web and Data Integration people group.

Instance Matching in Knowledge Graphs through random walks and semantics

Instance Matching (IM) is the process of matching instances that refer to the same real-world object (e.g., the same person) across different independent Knowledge Bases (KBs). This process is considered as a key step, for instance, in the integration of different KBs. In this paper, we focus on the problem of IM across different KBs represented as Knowledge Graphs (KGs). We propose SBIGMat, a novel approach for the IM problem based on Markov random walks (RW). Our approach leverages both the local and global information mutually calculated from a pairwise similarity graph. Precisely, we first build an expanded association graph consisting of pairs of IM candidates. Then, we rank each candidate pair through the stationary distribution computed from the RW on the association graph. We propose semantic and bipartite graph-based post-processing strategies that operate on the obtained random walk ranks to optimize the final assignment of co-referents. We provide a scalable distributed implementation of our approach on top of the Spark framework and we evaluate it on benchmark datasets from the instance track of the Ontology Alignment Evaluation Initiative (OAEI). The experiments show the efficiency and scalability of SBIGMat compared to several state-of-the-art IM approaches.

Situation-aware recommendation system for personalized healthcare applications

Personalized healthcare applications are designed with the help of handheld devices for providing instantaneous monitoring and diagnosis. Modern healthcare applications are designed with recommendation systems for providing diagnosis assistance for the users. This article introduces a novel situation-aware recommendation system for improving the reliability of personal healthcare applications. This recommendation system relies on recurrent neural learning for identifying the situation based on different physiological vitals. The learning method identifies a similar cause from the previous history and augments a recommendation from the connected healthcare server. The process of the situation is understood for providing equal and instantaneous recommendations for the patients. In this learning, the situation’s training and the previous cases of occurrence are performed concurrently for identifying matching instances, reducing the complexity. The proposed system’s performance is verified using the metrics matching ratio, complexity, recommendation delay, and accuracy.

Triplet online instance matching loss for person re-identification

Mining the shared features of the same identity in different scenes and the unique features of different identities in the same scene are the most significant challenges in the field of person re-identification (ReID). The Online Instance Matching (OIM) loss function and triplet loss function are the main methods for person ReID. Unfortunately, both of them have drawbacks. The OIM loss treats all samples equally and puts no emphasis on hard samples. The triplet loss processes batch construction in a complicated and fussy way and converges slowly. For these problems, we propose a Triplet Online Instance Matching (TOIM) loss function, which emphasizes hard samples and improves the person ReID accuracy effectively. It combines the advantages of the OIM loss and triplet loss and simplifies the batch construction process, which leads to quicker convergence. It can be trained on-line when handling the joint detection and identification task. To validate our loss function, we collect and annotate a large-scale benchmark dataset (UESTC-PR), which contains 499 identities and 60,437 images taken from surveillance cameras. We evaluated our proposed loss function on the Duke, Marker-1501 and UESTC-PR datasets using ResNet-50, and the results show that our proposed loss function outperforms the baseline methods, including Softmax loss, OIM loss and triplet loss.

Multi-View Instance Matching with Learned Geometric Soft-Constraints

We present a new approach for matching urban object instances across multiple ground-level images for the ultimate goal of city-scale mapping of objects with high positioning accuracy. What makes this task challenging is the strong change in view-point, different lighting conditions, high similarity of neighboring objects, and variability in scale. We propose to turn object instance matching into a learning task, where image-appearance and geometric relationships between views fruitfully interact. Our approach constructs a Siamese convolutional neural network that learns to match two views of the same object given many candidate image cut-outs. In addition to image features, we propose utilizing location information about the camera and the object to support image evidence via soft geometric constraints. Our method is compared to existing patch matching methods to prove its edge over state-of-the-art. This takes us one step closer to the ultimate goal of city-wide object mapping from street-level imagery to benefit city administration.

Anchor-Free Detector and Re-Identification with Joint Loss for Person Search

Person search aims at matching a target person from a gallery of panorama images. Its performance depends on the localization accuracy and the recall rate of the pedestrian detector. FoveaBox detector which outperforms others is utilized in our framework. It generates high-quality region proposals for following re-identification (re-id). A joint loss function is proposed to train the network effectively. It is made up of on-the-fly Online Instance Matching (OIM) and proposal pair double margin contrastive (PPDMC) loss. We propose offset guided erasing instead of random erasing to solve the occlusion problem in person search task preferably. Experiments show that our method performs more effectively against the state-of-the-art methods on two widely used person search datasets.

Data linking over RDF knowledge graphs: A survey

SummaryInstance matching (IM) is the process of matching instances across Knowledge Bases (KBs) that refer to the same real‐world object (eg, the same person in two different KBs). Several approaches in the literature were developed to perform this process using different algorithmic techniques and search strategies. In this article, we aim to provide the rationale for IM and to survey the existing algorithms for performing this task. We begin by identifying the importance of such a process and define it formally. We also provide a new classification of these approaches depending on the “source of evidence,” which can be considered as the context information integrated explicitly or implicitly in the IM process. We survey and discuss the state‐of‐the‐art IM methods regarding the context information. We, furthermore, describe and compare different state‐of‐the‐art IM approaches in relation to several criteria. Such a comprehensive comparative study constitutes an asset and a guide for future research in IM.

Hierarchical Online Instance Matching for Person Search

Person Search is a challenging task which requires to retrieve a person's image and the corresponding position from an image dataset. It consists of two sub-tasks: pedestrian detection and person re-identification (re-ID). One of the key challenges is to properly combine the two sub-tasks into a unified framework. Existing works usually adopt a straightforward strategy by concatenating a detector and a re-ID model directly, either into an integrated model or into separated models. We argue that simply concatenating detection and re-ID is a sub-optimal solution, and we propose a Hierarchical Online Instance Matching (HOIM) loss which exploits the hierarchical relationship between detection and re-ID to guide the learning of our network. Our novel HOIM loss function harmonizes the objectives of the two sub-tasks and encourages better feature learning. In addition, we improve the loss update policy by introducing Selective Memory Refreshment (SMR) for unlabeled persons, which takes advantage of the potential discrimination power of unlabeled data. From the experiments on two standard person search benchmarks, i.e. CUHK-SYSU and PRW, we achieve state-of-the-art performance, which justifies the effectiveness of our proposed HOIM loss on learning robust features.

Geocoding of trees from street addresses and street-level images

We introduce an approach for updating older tree inventories with geographic coordinates using street-level panorama images and a global optimization framework for tree instance matching. Geolocations of trees in inventories until the early 2000s where recorded using street addresses whereas newer inventories use GPS. Our method retrofits older inventories with geographic coordinates to allow connecting them with newer inventories to facilitate long-term studies on tree mortality etc. What makes this problem challenging is the different number of trees per street address, the heterogeneous appearance of different tree instances in the images, ambiguous tree positions if viewed from multiple images and occlusions. To solve this assignment problem, we (i) detect trees in Google street-view panoramas using deep learning, (ii) combine multi-view detections per tree into a single representation, (iii) and match detected trees with given trees per street address with a global optimization approach. Experiments for >50000 trees in 5 cities in California, USA, show that we are able to assign geographic coordinates to 38% of the street trees, which is a good starting point for long-term studies on the ecosystem services value of street trees at large scale.

The diameter of the stable marriage polytope: Bounding from below

An upper bound on the diameter of the Stable Matching (Stable Marriage) polytope is known to be ⌊n2⌋ where n is the number of men (or women) involved in the matching. The current work complements that result by providing a lower bound and an algorithm computing it. It also presents a class of Stable Matching instances for which the lower bound coincides with the above-mentioned upper bound.

Matching formulation of the Staff Transfer Problem: meta-heuristic approaches

In this paper, the Staff Transfer Problem (STP) in Human Resource Management is addressed as a stable matching problem. Earlier, formulation of this problem was of scheduling/allocation type. Here, the stable matching formulation is completely a new and more practical approach to the problem. This new formulation involves two preference lists: the first list contains the offices/locations preferred by the employees undergoing transfer and the second list contains the employees preferred by the employer of an office/location where those employees want to be transferred. The capacity of an office/location would act as a hard constraint. While matching these two lists, the objective is to maximize the number of transfers and at the same time to stabilize the matching, i.e., to minimize the number of blocking pairs. The resulting STP instance belongs to an instance of Maximum Size Minimum Blocking Pair Stable Matching with incomplete preference list (MAX SIZE MIN BP SMI) and has been proved in this paper to be NP-hard. As the problem is new in formulation, no previous work, method or result is available. There was no preference in selecting meta-heuristics. Among a large number of existing meta-heuristics, some most widely used meta-heuristics, namely, Simulated Annealing, Genetic Algorithms, Tabu Search and some variants of them have been chosen. Based on them four meta-heuristic approaches have been proposed, namely, btSA_match, gtSA_match, GA_match and TS_match. The variants btSA_match and gtSA_match are obtained from modifications made upon Simulated Annealing. EGA_match and TS_match are based on modified Genetic Algorithms and Tabu Search respectively. As there is no previous result in the existing literature, the performance has been compared among these four methods. It is observed that, variants of Simulated Annealing (SA) outperform others w.r.t. the performance metrics. The SA-variant with greedy nature, incorporated with a tabu list (gtSA_match) has shown that the best result on the basis of statistical analysis.

Dynamic imposter based online instance matching for person search

Person search aims to locate the target person matching a given query from a list of unconstrained whole images. It is a challenging task due to the unavailable bounding boxes of pedestrians, limited samples for each labeled identity and large amount of unlabeled persons in existing datasets. To address these issues, we propose a novel end-to-end learning framework for person search. The proposed framework settles pedestrian detection and person re-identification concurrently. To achieve the goal of co-learning and utilize the information of unlabeled persons, a novel yet extremely efficient Dynamic Imposter based Online Instance Matching (DI-OIM) loss is formulated. The DI-OIM loss is inspired by the observation that pedestrians appearing in the same image obviously have different identities. Thus we assign the unlabeled persons with dynamic pseudo-labels. The pseudo-labeled persons along with the labeled persons can be used to learn powerful feature representations. Experiments on CUHK-SYSU and PRW datasets demonstrate that our method outperforms other state-of-the-art algorithms. Moreover, it is superior and efficient in terms of memory capacity comparing with existing methods.

Context-aware instance matching through graph embedding in lexical semantic space

In recent years, the growing availability of open-accessed data (e.g., Wikipedia) combined with the advances in algorithmic techniques for information extraction have facilitated the design and structuring of information giving rise to knowledge bases. A major challenge relies in the integration of these independently designed knowledge bases. Instance matching is presented as one of the solutions to facilitate this process. It aims to link co-referent instances with an owl:sameAs connection to allow knowledge bases to complement each other. In this work, we present an approach for automatic alignment of instances in knowledge bases in the form of Resource Description Framework (RDF) graphs. Our approach generates for each instance a virtual document from its local description (i.e., data-type properties) and instances related to it through object-type properties (i.e., neighbors). We transform the instance matching problem into a document matching problem and solve it by a vector space embedding technique. We consider the pre-trained word embeddings to assess words similarities at both the lexical and semantic levels. We evaluate our approach on multiple knowledge bases from the instance track of the Ontology Alignment Evaluation Initiative (OAEI). The experiments show that our approach gets prominent results compared to several state-of-the-art existing approaches.

A cascaded multitask network with deformable spatial transform on person search

This article introduces a cascaded multitask framework to improve the performance of person search by fully utilizing the combination of pedestrian detection and person re-identification tasks. Inspired by Faster R-CNN, a Pre-extracting Net is used in the front part of the framework to produce the low-level feature maps of a query or gallery. Then, a well-designed Pedestrian Proposal Network called Deformable Pedestrian Space Transformer is introduced with affine transformation combined by parameterized sampler as well as deformable pooling dealing with the challenge of spatial variance of person re-identification. At last, a Feature Sharing Net, which consists of a convolution net and a fully connected layer, is applied to produce output for both detection and re-identification. Moreover, we compare several loss functions including a specially designed Online Instance Matching loss and triplet loss, which supervise the training process. Experiments on three data sets including CUHK-SYSU, PRW and SJTU318 are implemented and the results show that our work outperforms existing frameworks.

TAPON-MT: A versatile framework for semantic labelling

Semantic labelling refers to the problem of assigning known labels to the elements of structured information from a source such as an HTML table or an RDF dump with unknown semantics. In the recent years it has become progressively more relevant due to the growth of available structured information in the Web of data that need to be labelled in order to integrate it in data systems. The existing approaches for semantic labelling have several drawbacks that make them unappealing if not impossible to use in certain scenarios: not accepting nested structures as input, being unable to label structural elements, not being customisable, requiring groups of instances when labelling, requiring matching instances to named entities in a knowledge base, not detecting numeric data, or not supporting complex features. In this article, we propose TAPON-MT, a framework for machine learning semantic labelling. Our framework does not have the former limitations, which makes it domain-independent and customisable. We have implemented it with a graphical interface that eases the creation and analysis of models, and we offer a web service API for their application. We have also validated it with a subset of the National Science Foundation awards dataset, and our conclusion is that TAPON-MT creates models to label information that are effective and efficient in practice.