Instance Matching Research Articles

Intelligently Counting Agricultural Pests by Integrating SAM with FamNet

The utilization of the large pretrained model (LPM) based on Transformer has emerged as a prominent research area in various fields, owing to its robust computational capabilities. However, there remains a need to explore how LPM can be effectively employed in the agricultural domain. This research aims to enhance agricultural pest detection with limited samples by leveraging the strong generalization performance of the LPM. Through extensive research, this study has revealed that in tasks involving the counting of a small number of samples, complex agricultural scenes with varying lighting and environmental conditions can significantly impede the accuracy of pest counting. Consequently, accurately counting pests in diverse lighting and environmental conditions with limited samples remains a challenging task. To address this issue, the present research suggests a unique approach that integrates the outstanding performance of the segment anything model in class-agnostic segmentation with the counting network. Moreover, by intelligently utilizing a straightforward TopK matching algorithm to propagate accurate labels, and drawing inspiration from the GPT model while incorporating the forgetting mechanism, a more robust model can be achieved. This approach transforms the problem of matching instances in different scenarios into a problem of matching similar instances within a single image. Experimental results demonstrate that our method enhances the accuracy of the FamNet baseline model by 69.17% on this dataset. Exploring the synergy between large models and agricultural scenes warrants further discussion and consideration.

A Map of Diverse Synthetic Stable Matching Instances

Focusing on Stable Roommates (SR), we contribute to the toolbox for conducting experiments for stable matching problems. We introduce the polynomial-time computable mutual attraction distance to measure the similarity of SR instances, analyze its properties, and use it to create a map of SR instances. This map visualizes 460 synthetic SR instances (each sampled from one of ten different statistical cultures) as follows: Each instance is a point in the plane, and two points are close on the map if the corresponding SR instances are similar with respect to our mutual attraction distance to each other. Subsequently, we conduct several illustrative experiments and depict their results on the map, illustrating the map’s usefulness as a non-aggregate visualization tool, the diversity of our generated dataset, and the need to use instances sampled from different statistical cultures. Lastly, we extend our approach to the bipartite Stable Marriage problem.

A simple and efficient approach to unsupervised instance matching and its application to linked data of power plants

Knowledge graphs store and link semantically annotated data about real-world entities from a variety of domains and on a large scale. The World Avatar is based on a dynamic decentralised knowledge graph and on semantic technologies to realise complex cross-domain scenarios. Accurate computational results for such scenarios require the availability of complete, high-quality data. This work focuses on instance matching — one of the subtasks of automatically populating the knowledge graph with data from a wide spectrum of external sources. Instance matching compares two data sets and seeks to identify instances (data, records) referring to the same real-world entity. We introduce AutoCal, a new instance matcher which does not require labelled data and runs out of the box for a wide range of domains without tuning method-specific parameters. AutoCal achieves results competitive to recently proposed unsupervised matchers from the field of Machine Learning. We also select an unsupervised state-of-the-art matcher from the field of Deep Learning for a thorough comparison. Our results show that neither AutoCal nor the state-of-the-art matcher is superior regarding matching quality while AutoCal has only moderate hardware requirements and runs 2.7 to 60 times faster. In summary, AutoCal is specifically well-suited to be used in an automated environment. We present its prototypical integration into the World Avatar and apply AutoCal to the domain of power plants which is relevant for practical environmental scenarios of the World Avatar.

Single-shot global localization via graph-theoretic correspondence matching

ABSTRACT This paper describes a method of single-shot global localization based on graph-theoretic matching of instances between a query and a prior map. The proposed framework employs correspondence matching based on the maximum clique problem (MCP). The framework is potentially applicable to other map and/or query modalities thanks to the graph-based abstraction of the problem, while many existing global localization methods rely on a query and the dataset in the same modality. We implement it with a semantically labeled 3D point cloud map, and a semantic segmentation image as a query. Leveraging the graph-theoretic framework, the proposed method realizes global localization exploiting only the map and the query. The method shows promising results on multiple large-scale simulated maps of urban scenes.

Global-aware and local-aware enhancement network for person search

Person search, one of the increasingly important tasks of computer vision, performs pedestrian localization and re-identification simultaneously. Several studies incorporate anchor-free detectors into person search models and achieve good results. However, these methods ignore learning the contextual information of images, which is essential for improving the performance. To solve this problem, we propose a global-aware and local-aware enhancement network for person search. The stem network is constructed using hierarchical vision transformers to adaptively learn the global context. A local-aware enhancement module is proposed to learn the local context and incorporate multi-level features. To facilitate adequate learning on the limited training data, we design a symmetric online instance matching (SOIM) loss for end-to-end training the whole model. We conduct experiments on two benchmark datasets (the CUHK-SYSU and the PRW dataset) in comparison with the state-of-the-art methods. The results demonstrate the comparable performance of our proposed method.

Integrated carbon footprint with cutting parameters for production scheduling

A large part of the global carbon footprint comes from manufacturing. Apart from material distribution during product assembly, rational optimization of cutting parameters and production scheduling during processing are the main ways to realize low-carbon manufacturing. The existing researches mainly focus on cutting parameters or production scheduling when reducing manufacturing carbon emissions, but seldom considers the coupling relationship between the two. This limits the reduction of emissions to some extent. On this basis, a collaborative low-carbon optimization system suitable for the actual use of enterprises was developed. The optimization of production scheduling is a discrete problem, while cutting parameters are continuous and processing time brought by different cutting parameters will cause different effects on the scheduling scheme. Therefore, a two-stage optimization strategy is adopted to achieve collaborative optimization of cutting parameters and production scheduling. To achieve multi-objective optimization of the model and speed up convergence, the Crowding Niche mechanism and elite retention strategy are introduced into the selection operator to improve the genetic algorithm based on Pareto Optimality. In addition, different from the traditional that need to manually determine the machine tool and cutting tool according to the demand, a knowledge base module is integrated into the system. The knowledge base module is qualified for the knowledge acquisition and instance matching of the model in order to meet the processing capacity and constraints and enhance the ability of intelligent decision-making during production. Finally, the interactive application interface of the collaborative low-carbon optimization system has been implemented and the effectiveness of the system is verified by taking the manufacturing case of electromechanical enterprises as an example.

Prototype-guided Instance matching for multiple pedestrian tracking

Multiple pedestrian tracking is a critical yet challenging to understand human behavior. Although existing tracking methods have achieved great advances through adopting the tracking-by-detection paradigm, they give insufficient consideration on how to improve the instance matching capability in the cluttered and crowded environment. Motivated by our observations on tracking persons that the similarity of the same target is larger than that of other persons along sequence frames, in this work, we propose a novel Prototype-guided Instance Matching (PIM) method to boost the performance of multiple pedestrian tracking task. The whole PIM framework mainly consists of a detector net, an appearance embedding module, and a self-adaptive prototype dictionary. Given the current frame as the input sample, we first adopt a classic detector to select a set of positive candidates, and their corresponding multi-scale convolutional features are then fed into the appearance embedding module for obtaining the robust instance-wise feature representation. We perform the prototype-guided instance matching optimization between the learned instance-wise feature embeddings and the prototype dictionary, where its aim is to enlarge the separability of different instances and make the distance between the same instances more compact. Here object detection and appearance embedding are jointly optimized in an end-to-end fashion. Furthermore, we maintain a self-adaptive prototype dictionary to store instance-wise prototypes, which can be adaptively updated to adapt the appearance variations of all tracked targets over time. The instance matching and prototype dictionary update are modeled into an interactive optimization process so that they can be gradually improved for each other. Comprehensive evaluations on three benchmark datasets (i.e., MOT16, MOT17, and MOT20) demonstrate the effectiveness of our proposed PIM when compared with the state-of-the-art methods for the multiple pedestrian tracking task.

Supporting the Task-driven Skill Identification in Open Source Project Issue Tracking Systems

[Background] Selecting an appropriate task is challenging for contributors to Open Source Software (OSS), mainly for those who are contributing for the first time. Therefore, researchers and OSS projects have proposed various strategies to aid newcomers, including labeling tasks. [Aims] In this research, we investigate the automatic labeling of open issues strategy to help the contributors to pick a task to contribute. We label the issues with APIdomains- categories of APIs parsed from the source code used to solve the issues. We plan to add social network analysis metrics gathered from the issues conversations as new predictors. By identifying the skills, we claim the contributor candidates should pick a task more suitable to their skill. [Method] We are employing mixed methods. We qualitatively analyzed interview transcripts and the survey's open-ended questions to comprehend the strategies communities use to assist in onboarding contributors and contributors used to pick up an issue. We applied quantitative studies to analyze the relevance of the API-domain labels in a user experiment and compare the strategies' relative importance for diverse contributor roles. We also mined project and issue data from OSS repositories to build the ground truth and predictors able to infer the API-domain labels with comparable precision, recall, and F-measure with the state-of-art. We also plan to use a skill ontology to assist the matching process between contributors and tasks. By quantitatively analyzing the confidence level of the matching instances in ontologies describing contributors' skills and tasks, we might recommend issues for contribution. In addition, we will measure the effectiveness of the API-domain labels by evaluating the issues solving time and the rate among the labeled and unlabelled ones. [Results] So far, the results showed that organizing the issues?which includes assigning labels is seen as an essential strategy for diverse roles in OSS communities. The API-domain labels are relevant, mainly for experienced practitioners. The predicted labels have an average precision of 75.5%. [Conclusions] Labeling the issues with the API-domain labels indicates the skills involved in an issue. The labels represent possible libraries (aggregated into domains) used in the source code related to an issue. By investigating this research topic, we expect to assist the new contributors in finding a task, helping OSS communities to attract and retain more contributors.

I don't see what you're saying: The maluma/takete effect does not depend on the visual appearance of phonemes as they are articulated.

In contrast to the principle of arbitrariness, recent work has shown that language can iconically depict referents being talked about. One such example is the maluma/takete effect: an association between certain phonemes (e.g., those in maluma) and round shapes, and other phonemes (e.g., those in takete and spiky shapes). An open question has been whether this association is crossmodal (arising from phonemes' sound or kinesthetics) or unimodal (arising from phonemes' visual appearance). In the latter case, individuals may associate a person's rounded lips as they pronounce the /u/ in maluma with round shapes. We examined this hypothesis by having participants pair nonwords with shapes in either an audio-only condition (they only heard nonwords) or an audiovisual condition (they both heard nonwords and saw them articulated). We found no evidence that seeing nonwords articulated enhanced the maluma/takete effect. In fact, there was evidence that it decreased it in some cases. This was confirmed with a Bayesian analysis. These results eliminate a plausible explanation for the maluma/takete effect, as an instance of visual matching. We discuss the alternate possibility that it involves crossmodal associations.

Automatic registration of point cloud and panoramic images in urban scenes based on pole matching

• Register the MLS point cloud and panorama robustly and precisely by pole instance matching. • A novel pole matching algorithm is proposed that maximizes the overlapping matching areas. • Combine the point cloud and panorama to accurately and robustly extract pole object. Given the initial calibration of multiple sensors, the fine registration between Mobile Laser Scanning (MLS) point clouds and panoramic images is still challenging due to the unforeseen movement and temporal misalignment while collecting. To tackle this issue, we proposed a novel automatic method to register the panoramic images and MLS point clouds based on the matching of pole objects. Firstly, 2D pole instances in the panoramic images are extracted by a semantic segmentation network and then optimized. Secondly, every corresponding frustum point cloud of each pole instance is obtained by a shape-adaptive buffer region in the panoramic image, and the 3D pole object is extracted via a combination of slicing, clustering, and connected domain analysis, then all 3D pole objects are fused. Finally, 2D and 3D pole objects are re-projected onto virtual images respectively, and then fine 2D-3D correspondences are collected through maximizing pole overlapping area by Particle Swarm Optimization (PSO). The accurate extrinsic orientation parameters are acquired by the Efficient Perspective-N-Point (EPnP). The experiments indicate that the proposed method performs effectively on two challenging urban scenes with an average registration error of 2.01 pixels (with RMSE 0.88) and 2.35 pixels (with RMSE 1.03), respectively.

MLstruct: principal type inference in a Boolean algebra of structural types

Intersection and union types are becoming more popular by the day, entering the mainstream in programming languages like TypeScript and Scala 3. Yet, no language so far has managed to combine these powerful types with principal polymorphic type inference. We present a solution to this problem in MLstruct, a language with subtyped records, equirecursive types, first-class unions and intersections, class-based instance matching, and ML-style principal type inference. While MLstruct is mostly structurally typed, it contains a healthy sprinkle of nominality for classes, which gives it desirable semantics, enabling the expression of a powerful form of extensible variants that does not need row variables. Technically, we define the constructs of our language using conjunction, disjunction, and negation connectives, making sure they form a Boolean algebra, and we show that the addition of a few nonstandard but sound subtyping rules gives us enough structure to derive a sound and complete type inference algorithm. With this work, we hope to foster the development of better type inference for present and future programming languages with expressive subtyping systems.

On the complexity of stable hypergraph matching, stable multicommodity flow and related problems

In this paper we study stable multicommodity flows, stable hypergraph matchings and some related problems, like the Hospital-Resident-Couple (hrc) problem. We give a simpler proof of the existence of a stable multicommodity flow by reducing it to scarf. We show that finding a stable fractional solution of a hrc or a 3-dimensional stable matching (stable family) instance is PPAD-complete. Then we derive from these results that the Fractional hypergraph matching problem is PPAD-complete even if all hyperedge sizes and vertex degrees are at most 3. Furthermore, we prove that computing stable multicommodity flows is PPAD-complete even if the number of commodities is 3, the sum of in and out degrees is at most 3 at each vertex and all terminals and sources are the same. We also show that deciding if there exists a stable integer multicommodity flow is NP-complete even with only two commodities and similar restrictions.

A Deep Learning-Based Person Search System for Real-World Camera Images

<p>A person search system was developed to identify the query person from images captured by cameras at four scenes in the study. This study analyzed three network architectures called Model Basic, Model One, and Model Two. To verify the validity of the model design, the models in the public data set and in the recorded system data set were compared to determine whether the results of the proposed model exhibited consistent performance between the camera images from the public data set and the recorded, unprocessed system data set. The detected pedestrian images then underwent distance matching relative to query person images by using the online instance matching (OIM) loss function. Based on Model Basic, Model One and Model Two were designed to further improve accuracy by incorporating different convolutional neural networks. In CUHK-SYSU data set, the testing results of Model Basic, Model One and Model Two achieved the accuracies of 72.38%, 75.96% and 75.32%, respectively. The testing results of Model Basic, Model One, and Model Two with the system data set achieved accuracies of 63.745%, 68.80%, and 69.33%, respectively.</p> <p> </p>

Bi-CMR: Bidirectional Reinforcement Guided Hashing for Effective Cross-Modal Retrieval

Cross-modal hashing has attracted considerable attention for large-scale multimodal data. Recent supervised cross-modal hashing methods using multi-label networks utilize the semantics of multi-labels to enhance retrieval accuracy, where label hash codes are learned independently. However, all these methods assume that label annotations reliably reflect the relevance between their corresponding instances, which is not true in real applications. In this paper, we propose a novel framework called Bidirectional Reinforcement Guided Hashing for Effective Cross-Modal Retrieval (Bi-CMR), which exploits a bidirectional learning to relieve the negative impact of this assumption. Specifically, in the forward learning procedure, we highlight the representative labels and learn the reinforced multi-label hash codes by intra-modal semantic information, and further adjust similarity matrix. In the backward learning procedure, the reinforced multi-label hash codes and adjusted similarity matrix are used to guide the matching of instances. We construct two datasets with explicit relevance labels that reflect the semantic relevance of instance pairs based on two benchmark datasets. The Bi-CMR is evaluated by conducting extensive experiments over these two datasets. Experimental results prove the superiority of Bi-CMR over four state-of-the-art methods in terms of effectiveness.

Hybrid Instance-Aware Temporal Fusion for Online Video Instance Segmentation

Recently, transformer-based image segmentation methods have achieved notable success against previous solutions. While for video domains, how to effectively model temporal context with the attention of object instances across frames remains an open problem. In this paper, we propose an online video instance segmentation framework with a novel instance-aware temporal fusion method. We first leverage the representation, \ie, a latent code in the global context (instance code) and CNN feature maps to represent instance- and pixel-level features. Based on this representation, we introduce a cropping-free temporal fusion approach to model the temporal consistency between video frames. Specifically, we encode global instance-specific information in the instance code and build up inter-frame contextual fusion with hybrid attentions between the instance codes and CNN feature maps. Inter-frame consistency between the instance codes is further enforced with order constraints. By leveraging the learned hybrid temporal consistency, we are able to directly retrieve and maintain instance identities across frames, eliminating the complicated frame-wise instance matching in prior methods. Extensive experiments have been conducted on popular VIS datasets, i.e. Youtube-VIS-19/21. Our model achieves the best performance among all online VIS methods. Notably, our model also eclipses all offline methods when using the ResNet-50 backbone.

One-shot multi-object tracking using CNN-based networks with spatial-channel attention mechanism

Deep learning algorithms for multi-object tracking have made great progress and have powered the emergence of state-of-the-art models to address multi-object tracking problems. Though a lot of efforts have been made, false detections (named “FP”) and missed detections (named “FN”) caused by inaccurate tracking still cannot be well addressed especially in extremely crowded driving situations. To address these problems, we develop a new online one-shot multi-object tracking system based on convolutional neural networks with spatial-channel attention mechanism. Firstly, we propose a feature combination module (FCM) that uses dilated convolution to obtain different receptive fields to adapt to the deformation of the targets, instead of introducing a large number of parameters to deal with the problem of target scale transformation like the recent feature pyramid network. Then, an attention mechanism network (AM-Net) is designed to allow the model to dynamically focus on certain parts of the input that help perform the task and ignore irrelevant information. Finally, we introduce a combination of triple loss and online instance matching loss (TOIM Loss) to distinguish similar instances within a class. Our proposed method is evaluated in three commonly used multi-object tracking datasets including 2DMOT15, MOT16 and MOT20. The results show that our proposed method is generally superior to the compared models.

Matching Knowledge Graphs with Compact Niching Evolutionary Algorithm

To address the Knowledge Graph (KG) heterogeneity issue, we need to determine a set of entity correspondences, which requires aggregating several complementary similarity measures to improve the confidence of the results. How to determine the suitable aggregating weights for the similarity measures to improve the KG alignment’s quality is called the KG meta-matching problem, whose challenge of scalability remains significant in the Semantic Web (SW) domain. To face this challenge, this work proposes a Compact Niching Evolutionary Algorithm (CNEA) based matching technique. We first propose an approximate evaluation metric on the alignment’s quality, and on this basis, a multi-modal optimization model is constructed to formally define the KG meta-matching problem. Then, a niching strategy is combined with EA’s evolutionary paradigm to address it, which is able to effectively locate and maintain multiple global optimal solutions. Moreover, a serial matching framework and the compact evolutionary mechanism are presented to improve CNEA’s efficiency. In particular, the former reduces the algorithm’s search space of the instance matching phase with schema-level alignments, and the latter uses the probability representation on the population to reduce the algorithm’s memory consumption and run time. The experiment utilizes Ontology Alignment Evaluation Initiative (OAEI)’s KG track to test our proposal’s performance, and experimental results show that CNEA-based KG matching technique is both effective and efficient.

A Context-Independent Ontological Linked Data Alignment Approach to Instance Matching

Linking data by finding matching instances in different datasets requires considering many characteristics, such as structural heterogeneity, implicit knowledge, and URI (Uniform Resource Identifier)-oriented identification. The authors propose a context-independent approach to align Linked data through an alignment process based on the ontological model’s components and considering data’s multidimensionality. The researchers experimented with the proposed approach against two methods for aligning linked data in two datasets and evaluated precision, recall, and f-measure metrics. The authors also conducted a case study in a real scenario considering a Brazilian publication dataset on computers and education. This study’s results indicate that the proposed approach overcomes the other methods (regarding the precision, recall, and f-measure metrics), requiring less work when changing the dataset domain. This work’s main contributions include enabling real datasets to be semi-automatically linked, presenting an approach capable of calculating resource similarity.

On the Coexistence Of Stability and Incentive Compatibility in Fractional Matchings

Stable matchings have been studied extensively in social choice literature. The focus has been mostly on integral matchings, in which the nodes on the two sides are wholly matched. A fractional matching, which is a convex combination of integral matchings, is a natural extension of integral matchings. The topic of stability of fractional matchings has started receiving attention only very recently. Further, incentive compatibility in the context of fractional matchings has received very little attention. With this as the backdrop, our paper studies the important topic of incentive compatibility of mechanisms to find stable fractional matchings. We work with preferences expressed in the form of cardinal utilities. Our first result is an impossibility result that there are matching instances for which no mechanism that produces a stable fractional matching can be incentive compatible or even approximately incentive compatible. This provides the motivation to seek special classes of matching instances for which there exist incentive compatible mechanisms that produce stable fractional matchings. Our study leads to a class of matching instances that admit unique stable fractional matchings. We first show that a unique stable fractional matching for a matching instance exists if and only if the given matching instance satisfies the conditional mutual first preference (CMFP) property. To this end, we provide a polynomial-time algorithm that makes ingenious use of envy-graphs to find a non-integral stable matching whenever the preferences are strict and the given instance is not a CMFP matching instance. For this class of CMFP matching instances, we prove that every mechanism that produces the unique stable fractional matching is (a) incentive compatible and further (b) resistant to coalitional manipulations.

A Schema-Free Instance Matching Algorithm Based on Virtual Document Similarity

With the continuous development of semantic web, especially of the web of data, several knowledge bases expressed by ontologies are independently created and added to the Linked Open Data (LOD) cloud, on a daily basis. A major challenge for the LOD paradigm is to discover resources that refer to the same real-world object, in order to interlink web resources and hold large scale data integration and sharing. In this context, instance matching is a promising solution. It aims to link co-referent instances belonging to heterogeneous knowledge bases with owl:sameAs links. Several state-of-the-art existing approaches addressing this issue are based on the prior schema-level matchings, which does not avoid the limitation of heterogeneity at the property-level. In this paper, we propose a schema-free, scalable and efficient instance matching approach that is independent from matching results at the schema-level. We transform the instance matching problem to a document similarity problem and we solve it by a Clustering technique that uses an Ascendant Hierarchical Clustering algorithm to group similar instances in the same clusters. Furthermore, we design multiple validating patterns that use some structural information to validate obtained mappings and eliminate wrong ones. Experiments on instance matching track from Ontology Alignment Evaluation Initiative (OAEI) show that our approach gets prominent results compared to several participating systems in OAEI’2019, OAEI’2020 and OAEI’2021.