Graph Matching Method Research Articles

Graph matching based on fast normalized cut and multiplicative update mapping

Point correspondence is a fundamental problem in pattern recognition and computer vision, which can be tackled by graph matching. Since graph matching is basically an NP-complete problem, some approximate methods are proposed to solve it. Continuous relaxation offers an effective approximate method for graph matching problem. However, the discrete constraint is not taken into consideration in the optimization step. In this paper, a fast normalized cut based graph matching method is proposed, where the discrete constraint is introduced into the optimization step. Specifically, first a semidefinite positive affinity matrix based form objective function is constructed by introducing a regularization term which is related to the discrete constraint. Then the fast normalized cut algorithm is utilized to find the continuous solution. Last, the discrete solution of graph matching is obtained by a multiplicative update algorithm. Experiments on both synthetic points and real-world images validate the effectiveness of the proposed method by comparing it with the state-of-the-art methods.

IA-GM: A Deep Bidirectional Learning Method for Graph Matching

Existing deep learning methods for graph matching(GM) problems usually considered affinity learningto assist combinatorial optimization in a feedforward pipeline, and parameter learning is executed by back-propagating the gradients of the matching loss. Such a pipeline pays little attention to the possible complementary benefit from the optimization layer to the learning component. In this paper, we overcome the above limitation under a deep bidirectional learning framework.Our method circulates the output of the GM optimization layer to fuse with the input for affinity learning. Such direct feedback enhances the input by a feature enrichment and fusion technique, which exploits andintegrates the global matching patterns from the deviation of the similarity permuted by the current matching estimate. As a result, the circulation enables the learning component to benefit from the optimization process, taking advantage of both global feature and the embedding result which is calculated by local propagationthrough node-neighbors. Moreover, circulation consistency induces an unsupervised loss that can be implemented individually or jointly to regularize the supervised loss. Experiments on challenging datasets demonstrate the effectiveness of our methods for both supervised learning and unsupervised learning.

Proxy Graph Matching with Proximal Matching Networks

Estimating feature point correspondence is a common technique in computer vision. A line of recent data-driven approaches utilizing the graph neural networks improved the matching accuracy by a large margin. However, these learning-based methods require a lot of labeled training data, which are expensive to collect. Moreover, we find most methods are sensitive to global transforms, for example, a random rotation. On the contrary, classical geometric approaches are immune to rotational transformation though their performance is generally inferior. To tackle these issues, we propose a new learning-based matching framework, which is designed to be rotationally invariant. The model only takes geometric information as input. It consists of three parts: a graph neural network to generate a high-level local feature, an attention-based module to normalize the rotational transform, and a global feature matching module based on proximal optimization. To justify our approach, we provide a convergence guarantee for the proximal method for graph matching. The overall performance is validated by numerical experiments. In particular, our approach is trained on the synthetic random graphs and then applied to several real-world datasets. The experimental results demonstrate that our method is robust to rotational transform and highlights its strong performance of matching accuracy.

DeepSeed Local Graph Matching for Densely Packed Cells Tracking.

The tracking of densely packed plant cells across microscopy image sequences is very challenging, because their appearance change greatly over time. A local graph matching algorithm was proposed to track such cells by exploiting the tight spatial topology of neighboring cells, and then an iterative searching strategy was used to grow the correspondence from a seed cell pair. Thus, the performance of the existing tracking approach heavily relies on the robustness of finding seed cell pair. However, the existing local graph matching algorithm cannot guarantee the correctness of the seed cell pair, especially in unregistered image sequences or image sequences with large time intervals. In this paper, we propose a DeepSeed local graph matching model to find seed cell pair robustly, by combining local graph matching and CNN-based similarity learning, which uses cells' spatial-temporal contextual information and cell pairs' similarity information. The CNN-based similarity learning is designed to learn cells' deep feature and measure cell pairs' similarity. Compared with the existing plant cell matching methods, the experimental results show that the DeepSeed local graph matching method can track most cells in unregistered image sequences. Moreover, the DeepSeed tracking algorithm can accurately track cells across image sequences with large time intervals.

Spatial Location in Integrated Circuits through Infrared Microscopy.

In this paper, we present an infrared microscopy based approach for structures’ location in integrated circuits, to automate their secure characterization. The use of an infrared sensor is the key device for internal integrated circuit inspection. Two main issues are addressed. The first concerns the scan of integrated circuits using a motorized optical system composed of an infrared uncooled camera combined with an optical microscope. An automated system is required to focus the conductive tracks under the silicon layer. It is solved by an autofocus system analyzing the infrared images through a discrete polynomial image transform which allows an accurate features detection to build a focus metric robust against specific image degradation inherent to the acquisition context. The second issue concerns the location of structures to be characterized on the conductive tracks. Dealing with a large amount of redundancy and noise, a graph-matching method is presented—discriminating graph labels are developed to overcome the redundancy, while a flexible assignment optimizer solves the inexact matching arising from noises on graphs. The resulting automated location system brings reproducibility for secure characterization of integrated systems, besides accuracy and time speed increase.

Graph Extraction of Batik Image Using Region Adjacency Graph Representation

Image feature extraction done in image analysis based on the traditional pixel is not extracted effectively. This is due to the extraction that only represents the content. A very promising approach and the challenge are to extract a graph from an image that represents the content and their relationship. The results of graph extraction are obtained from the process of image segmentation. The selection of an appropriate segmentation method from many graph-based image segmentation methods is interesting to be reviewed. The method of super-pixel segmentation is one way to divide the image into regions. The regions obtained in segmentation are represented by vertices and edges represent connections between adjacent regions. This representation is called Region Adjacency Graph (RAG). In this paper, graph extraction of “batik” images has been successfully performed where its representation uses RAG and segmentation uses the Minimum Spanning Tree method. The results of graph extraction have been tested in a simple image retrieval process using the VF2 graph matching method.

Graph Matching for Marker Labeling and Missing Marker Reconstruction With Bone Constraint by LSTM in Optical Motion Capture

Optical motion capture (MOCAP) is a commonly used technology to record the motion of non-rigid objects with high accuracy in 3D space. However, the MOCAP data has to be processed further before it can be used. The scattered reconstructed motion data must constitute a human configuration by labelling process according to the predefined template, and the missing markers have to be reconstructed to produce a stable motion trajectory. In this work, we propose a novel labelling method for motion sequences. First, a novel graph matching method is employed to determine the connection relationship of the scattered motion data for a single frame. Then, Kalman filtering is used for tracking in the motion sequence. As for the challenge coming from missing markers, we propose a new motion data preprocessing method considering the bone length constraint, which represents the information of variation in the relative position of adjacent markers. The processed motion data is input into a Long-Short Term Memory (LSTM) model to recover the missing markers and de-noise the motion data. The experiment conducted on our own dataset proves that our labelling method achieves a similar effect to Cortex, which is a commonly used commercial motion data analysis software. The experiment on CMU dataset demonstrates that our missing marker reconstruction method can achieve an art-of-state result. The labelling code will be pulished on https://github.com/Lijianfang6930/Graph-Matching-for-Marker-Labelling

Deep Graph Matching Meets Mixed-Integer Linear Programming: Relax at Your Own Risk?

Graph matching is an important problem that has received widespread attention, especially in the field of computer vision. Recently, state-of-the-art methods seek to incorporate graph matching with deep learning. However, there is no research to explain what role the graph matching algorithm plays in the model. Therefore, we propose an approach integrating a MILP formulation of the graph matching problem. This formulation is solved to optimal and it provides inherent baseline. Meanwhile, similar approaches are derived by releasing the optimal guarantee of the graph matching solver and by introducing a quality level. This quality level controls the quality of the solutions provided by the graph matching solver. In addition, several relaxations of the graph matching problem are put to the test. Our experimental evaluation gives several theoretical insights and guides the direction of deep graph matching methods.

Iterative closest graph matching for non-rigid 3D/2D coronary arteries registration

Background and objective Fusion of the preoperative computed tomography angiography and intraoperative X-ray angiography images can considerably enhance the visual perception of physicians during percutaneous coronary interventions. This technique can provide 3D information of the arteries and reduce the uncertainty of 2D guidance images. For this purpose, 3D/2D vascular registration with high accuracy and robustness is crucial for performing accurate surgery.Methods In this study, we propose an iterative closest graph matching (ICGM) method that utilizes an alternative iteration framework including correspondence and transformation phases. A coarse-to-fine matching approach based on redundant graph matching is proposed for the correspondence phase. The transformation phase involves rigid and non-rigid transformations, in which rigid transformation is calculated using a closed-form solution, and non-rigid transformation is achieved using a statistical shape model established from a synthetic deformation dataset.Results The proposed method is evaluated and compared with nine state-of-the-art methods on simulated data and clinical datasets. Experiments demonstrate that our method is insensitive to the pose of data and robust to noise and deformation. Moreover, it outperforms other methods in terms of registering real data.Conclusions Given its high capture range, the proposed method can register 3D vessels without prior initialization in clinical practice.

Dual Calibration Mechanism Based L2, p-Norm for Graph Matching

Unbalanced geometric structure caused by variations with deformations, rotations and outliers is a critical issue that hinders correspondence establishment between image pairs in existing graph matching methods. To deal with this problem, in this work, we propose a dual calibration mechanism (DCM) for establishing feature points correspondence in graph matching. In specific, we embed two types of calibration modules in the graph matching, which model the correspondence relationship in point and edge respectively. The point calibration module performs unary alignment over points and the edge calibration module performs local structure alignment over edges. By performing the dual calibration, the feature points correspondence between two images with deformations and rotations variations can be obtained. To enhance the robustness of correspondence establishment, the $L_{2,p}$ -norm is employed as the similarity metric in the proposed model, which is a flexible metric due to setting the different $p$ values. Finally, we incorporate the dual calibration and $L_{2,p}$ -norm based similarity metric into the graph matching model which can be optimized by an effective algorithm, and theoretically prove the convergence of the presented algorithm. Experimental results in the variety of graph matching tasks such as deformations, rotations and outliers evidence the competitive performance of the presented DCM model over the state-of-the-art approaches.

Hierarchical Bipartite Graph Matching Method for Transactive V2V Power Exchange in Distribution Power System

In this paper, a hierarchical bipartite graph (HBG) matching method is proposed to promote transactive vehicle-to-vehicle (V2V) power exchanges in a distribution power system. A distribution system which includes EVs is divided into a series of subsystems according to EV locations, where V2V power exchanges are optimized for scheduling charging/discharging periods among transacting EVs in each subsystem. Each subsystem's power surplus or deficit will then be regulated by power exchanges among neighboring subsystems and the utility grid. The one-to-multiple and multiple-to-one modes are adopted in the proposed HBG algorithm for the promotion of transactive V2V power exchanges. Also, individual EV's electricity trading price can track that of the subsystem by adjusting respective power exchange objectives adaptively. Several case studies are presented to validate the effectiveness of the proposed method, where transactive V2V power exchanges are improved and the energy efficiency in a distribution power system is enhanced.

Analysis of a printed complex image quality checking method of fabric cloth for development of an automated quality checking system

In this research paper, the method of printed complex image quality checking for fabric cloth has been proposed for the development of an automated, printed image, quality checking system. As described in this paper, this quality checking system was implemented using graph matching and the pixel calculation algorithms, and the reliability was analyzed using HALCON software. The analysis of image and defect detection of image was performed by partitioning the images. First, the image registration area was automatically selected according to the regional characteristics of the image pattern; subsequently, the system performed fast image registration, based on the shape discovered through a graph matching method. Finally, the pixel calculation method was used in the specific image registration area to detect defects of the printed image in order to identify and ensure quality of the subsequently printed complex pattern on the fabric cloth. In this research, the proposed method was carried out on the image pattern of a selected registration area. The image registration for defective detection quickly, effectively, and efficiently detected the faults. The minimum and maximum errors were found as 0.16% and 8.4%, respectively, occurring within a 10% control. Whenever the error exceeded 10%, it was regarded as a quality defect of printed fabric cloth product; conversely, when the error fell below 10%, the product was deemed a qualified product. The higher measurement accuracy by the proposed method meets the needs of actual production by providing a new method of printing defect detection for the textile and garment industry printing sectors.

Multi-View Graph Matching for 3D Model Retrieval

3D model retrieval has been widely utilized in numerous domains, such as computer-aided design, digital entertainment, and virtual reality. Recently, many graph-based methods have been proposed to address this task by using multi-view information of 3D models. However, these methods are always constrained by many-to-many graph matching for the similarity measure between pairwise models. In this article, we propose a multi-view graph matching method (MVGM) for 3D model retrieval. The proposed method can decompose the complicated multi-view graph-based similarity measure into multiple single-view graph-based similarity measures and fusion. First, we present the method for single-view graph generation, and we further propose the novel method for the similarity measure in a single-view graph by leveraging both node-wise context and model-wise context. Then, we propose multi-view fusion with diffusion, which can collaboratively integrate multiple single-view similarities w.r.t. different viewpoints and adaptively learn their weights, to compute the multi-view similarity between pairwise models. In this way, the proposed method can avoid the difficulty in the definition and computation of the traditional high-order graph. Moreover, this method is unsupervised and does not require a large-scale 3D dataset for model learning. We conduct evaluations on four popular and challenging datasets. The extensive experiments demonstrate the superiority and effectiveness of the proposed method compared against the state of the art. In particular, this unsupervised method can achieve competitive performances against the most recent supervised and deep learning method.

Probing into the Physical Layer: Moving Tag Detection for Large-Scale RFID Systems

Logistics monitoring is a fundamental application that utilizes RFID systems to manage numerous tagged-objects. Due to the frequent rearrangement of tagged-objects, a fast RFID-based tracking approach is highly desired for accurate logistics distribution. However, traditional RFID systems usually take tens of seconds to interrogate hundreds of RFID tags, not to mention the time delay involved to locate all the tags, which severely prevents from in-time tracking. To address this issue, we reduce the problem domain by first distinguishing the motion status of the tagged-objects, i.e., “stationary” or “moving”, and then tracking the moving objects with the state-of-the-art localization schemes, which significantly reduces the efforts of tracking all the objects. Toward this end, we propose a moving tag detection mechanism, which achieves the time efficiency by exploiting the useless collision signal in RFID systems. In particular, we extract two kinds of physical-layer features (namely, phase profile and backscatter link frequency ) from the collision signal received by the USRP to distinguish tags at different positions. We further develop the Graph Matching (GM) method and Coherent Phase Variance (CPV) method to detect the moving tagged-objects. Experiment results show that our approach can accurately detect the moving objects while reducing 80 percent inventory time compared with the state-of-art solutions.

Vertex nomination via seeded graph matching

AbstractConsider two networks on overlapping, nonidentical vertex sets. Given vertices of interest (VOIs) in the first network, we seek to identify the corresponding vertices, if any exist, in the second network. While in moderately sized networks graph matching methods can be applied directly to recover the missing correspondences, herein we present a principled methodology appropriate for situations in which the networks are too large/noisy for brute‐force graph matching. Our methodology identifies vertices in a local neighborhood of the VOIs in the first network that have verifiable corresponding vertices in the second network. Leveraging these known correspondences, referred to as seeds, we match the induced subgraphs in each network generated by the neighborhoods of these verified seeds, and rank the vertices of the second network in terms of the most likely matches to the original VOIs. We demonstrate the applicability of our methodology through simulations and real data examples.

Dynamic Hybrid Graph Matching for Unsupervised Video-based Person Re-identification

Taking videos as nodes in a graph, graph matching is an effective technique for unsupervised video-based person re-identification (re-ID). However, most of existing methods are sensitive to noisy training data and mainly only focus on visual content relations between query and gallery videos, which may introduce large amount of false positives. To enhance the robustness to training data and alleviate the visual ambiguity, a Dynamic Hybrid Graph Matching (DHGM) method is proposed, which jointly considers both content and context information for person re-ID in an iterative manner. The content relations between video nodes are obtained by metric learning, based on which the context relation is acquired by encoding the bidirectional feature of each probe node relative to its graph neighbors. The model is iteratively updated during the process of graph construction for promoted distance measurement and further better matching performance. Experimental results on the PRID 2011 and iLIDS-VID datasets demonstrate the superiority of the DHGM.

Graph matching based on local and global information of the graph nodes

Graph matching is an essential NP-problem in computer vision and pattern recognition. In this paper, we propose an approximate graph matching method. This method formulates the problem of computing the correspondences between two graphs as a problem of selecting nodes on an association graph. The nodes of the association graph represent candidate correspondences between the two original graphs. Our method first constructs an affinity matrix based on both the global and local information of the original graphs’ nodes. Each element of this matrix is used to measure the mutual consistency of a pair of nodes within the association graph. Our method then applies the reweighted random walks technique that preserves the one-to-one matching constraint to simulate random walks on the association graph and to iteratively compute a quasi-stationary distribution. To discretize this distribution, our method finally applies the Hungarian algorithm and obtains an approximate matching between the original two graphs. Experimental results demonstrate the effectiveness of our method for graph matching and the ability of our method for being robust to outlier and deformation noise.

Drug-drug interaction prediction based on co-medication patterns and graph matching

Background: The problem of predicting whether a drug combination of arbitrary orders is likely to induce adverse drug reactions is considered in this manuscript. Methods: Novel kernels over drug combinations of arbitrary orders are developed within support vector machines for the prediction. Graph matching methods are used in the novel kernels to measure the similarities among drug combinations, in which drug co-medication patterns are leveraged to measure single drug similarities. Results: The experimental results on a real-world dataset demonstrated that the new kernels achieve an area under the curve (AUC) value 0.912 for the prediction problem. Conclusions: The new methods with drug co-medication based single drug similarities can accurately predict whether a drug combination is likely to induce adverse drug reactions of interest. Keywords: drug-drug interaction prediction; drug combination similarity; co-medication; graph matching

Drug-drug interaction prediction based on co-medication patterns and graph matching

Background: The problem of predicting whether a drug combination of arbitrary orders is likely to induce adverse drug reactions is considered in this manuscript. Methods: Novel kernels over drug combinations of arbitrary orders are developed within support vector machines for the prediction. Graph matching methods are used in the novel kernels to measure the similarities among drug combinations, in which drug co-medication patterns are leveraged to measure single drug similarities. Results: The experimental results on a real-world dataset demonstrated that the new kernels achieve an area under the curve (AUC) value 0.912 for the prediction problem. Conclusions: The new methods with drug co-medication based single drug similarities can accurately predict whether a drug combination is likely to induce adverse drug reactions of interest. Keywords: drug-drug interaction prediction; drug combination similarity; co-medication; graph matching

Research on Service Discovery Methods Based on Knowledge Graph

Service discovery is one of the main goals of a service-oriented architecture, helping to discover complex applications. Knowledge graph is a cross-cutting research hotspot in the fields of computer, knowledge engineering and information science. This paper makes in-depth research on the key issues of knowledge graph matching for service-oriented discovery. Firstly, construct a knowledge graph for service discovery, and then design a template to match the knowledge graph. By designing a matching template, the service discovery mechanism is transformed into a knowledge graph knowledge query. Finally, the validity of the knowledge graph matching method proposed in this paper is verified by experiments.