A method for detecting objects in images based on neural networks on graphs and a small number of training examples

Abstract

In the presented work, the object of research is computer vision systems. The subject of the study is a method for detecting objects in images based on neural networks on graphs and a small number of training examples. Such aspects of the topic as the use of a structural representation of the scene to improve the accuracy of object detection are discussed in detail. It is proposed to share information about the structure of the scene based on neural networks on graphs and training from "multiple shots" to increase the accuracy of object detection. Relationships between classes are established using external semantic links. To do this, a knowledge graph is pre-created. The method contains two stages. At the first stage, object detection is performed based on training with "multiple shots". At the second stage, the detection accuracy is improved using a neural network on graphs. The basis of the developed method is the use of convolution based on spectral graph theory. Each vertex represents a category in the knowledge graph, and the edge weight of the graph is calculated based on conditional probability. Based on the convolution, information from neighboring vertices and edges is combined to update the vertex values. The scientific novelty of the developed method lies in the joint use of convolutional networks on graphs and training from "multiple shots" to increase the accuracy of object detection. A special contribution of the author to the research of the topic is the use of a convolutional network based on a knowledge graph to improve the results of the object detection method using a small number of training examples. The method was studied on test sets of images from the field of computer vision. Using the PASCAL VOC and MS COCO datasets, it is demonstrated that the proposed method increases the accuracy of object detection by analyzing structural relationships. The average accuracy of object detection using the developed method increases by 1-5% compared to the "multiple shots" training method without using a structural representation.