Effective Content-based Retrieval Research Articles

Multi-view expressive graph neural networks for 3D CAD model classification

The creation of effective content-based retrieval systems for the 3D models of engineering components created by Mechanical Computer Aided Design (MCAD) systems has been a subject of academic investigation since the 1990 s. Recently some of the most promising results have been reported by researchers using Deep Neural Nets (DNN) to classify industrial parts represented by collections of images generated from different viewpoints. Known as Multi-View Convolutional Neural Network (MVCNN) these systems have extended the architectures developed for 2D images analysis to handle 3D data by using a series of pictures rendered from different viewpoints. In 2016 Monti et al. used Graph Neural Networks (GNN) to classify the superpixel images for the first time and reported results which suggested that the approach could produce good classification accuracy (an observation confirmed by the application of GNNs to 2D image datasets of common objects such as MNIST and Cifar). To investigate the potential for GNNs to reason about spatial adjacency relationships this paper reports, for the first time, the application of GNNs to classifications of 3D MCAD models of mechanical components. Viewing GNN as a convolution operator, parallel GNN can be applied to multiple views of a 3D shape in the same way as CNN is structured in a MVCNN. After outlining the architecture implemented to support MVGNNs the impact of various hyperparameters on the expressive power are explored. When optimised these hyperparameters (located in both intra-layer and inter-layer structures) combined with the multi-view architecture produced a classification accuracy superior to previous methods. The results also suggest that GNNs have the potential to produce fast, accurate classification systems for 3D MCAD data using significantly smaller datasets than those used for transfer learning in MVCNNs.

Improving 3D similarity search by enhancing and combining 3D descriptors

Effective content-based retrieval in 3D model databases is an important problem that has attracted much research attention over the last years. Many individual methods proposed to date rely on calculating global 3D model descriptors based on image, surface, volumetric, or structural model properties. Descriptors such as these are then input for determining the degree of similarity between models. Traditionally, the ability of individual descriptors to perform effective 3D search is decided by benchmarking. However, in practice the data set on which 3D retrieval is to be applied may differ from the characteristics of the respective benchmark. Therefore, statically determining the descriptor to use based on a fixed benchmark may lead to suboptimal results. We propose a generic strategy to improve the retrieval effectiveness in 3D retrieval systems consisting of multiple model descriptors. The specific contribution of this paper is two-fold. First, we propose to adaptively combine multiple descriptors by forming weighted descriptor combinations, where the weight of each descriptor is decided at query time. Second, we enhance the set of global model descriptors to be combined by including partial descriptors of the same kind in the combinations. Partial descriptors are obtained by applying a given descriptor extractor on the set of parts of a model, obtained by a simple model partitioning scheme. Thereby, more model information is exposed to the 3D descriptors, leading to a more complete object description. We give a systematic discussion of the descriptor combination space involving static and query-adaptive weighting schemes, and based on descriptors of different type and focus (model global vs. partial). The combination of both global and partial model descriptors is shown to deliver improved retrieval precision, compared to policies using single descriptors or fixed-weight combinations. The resulting scheme is generic and can accommodate a large class of global 3D model descriptors.

Computer-Aided Detection of Diagnostic and Therapeutic Operations in Colonoscopy Videos

Colonoscopy is an endoscopic technique that allows a physician to inspect the inside of the human colon and to perform--if deemed necessary-at the same time a number of diagnostic and therapeutic operations. In order to see the inside of the colon, a video signal of the internal mucosa of the colon is generated by a tiny video camera at the tip of the endoscope and displayed on a monitor for real-time analysis by the physician. We have captured and stored these videos in digital format and call these colonoscopy videos. Based on new algorithms for instrument detection and shot segmentation, we introduce new spatio-temporal analysis techniques to automatically identify an operation shot--a segment of visual data in a colonoscopy video that corresponds to a diagnostic or therapeutic operation. Our experiments on real colonoscopy videos demonstrate the effectiveness of the proposed approach. The proposed techniques and software are useful for 1) postprocedure review for causes of complications due to diagnostic or therapeutic operations; 2) establishment of an effective content-based retrieval system to facilitate endoscopic research and education; 3) development of a systematic approach to assess and improve the procedural skills of endoscopists.

An image retrieval system based on identifying objects of interest

This paper presents an effective content-based retrieval mechanism based on previous work in the area of image processing and segmentation. We use the Haar transform to isolate the object of interest (OOI) in the image and then remove the effect of r

An image retrieval system based on identifying objects of interest

This paper presents an effective content-based retrieval mechanism based on previous work in the area of image processing and segmentation. We use the Haar transform to isolate the object of intere...