3D Object Recognition Algorithm Research Articles

Remote Big Data Management and Visual Imagery Tools, Multisensor Fusion and Dynamic Routing Technologies, and 3D Space Mapping and Object Recognition Algorithms on Blockchain-based Metaverse Platforms

Remote Big Data Management and Visual Imagery Tools, Multisensor Fusion and Dynamic Routing Technologies, and 3D Space Mapping and Object Recognition Algorithms on Blockchain-based Metaverse Platforms

DGCB-Net: Dynamic Graph Convolutional Broad Network for 3D Object Recognition in Point Cloud

3D (3-Dimensional) object recognition is a hot research topic that benefits environment perception, disease diagnosis, and the mobile robot industry. Point clouds collected by range sensors are a popular data structure to represent a 3D object model. This paper proposed a 3D object recognition method named Dynamic Graph Convolutional Broad Network (DGCB-Net) to realize feature extraction and 3D object recognition from the point cloud. DGCB-Net adopts edge convolutional layers constructed by weight-shared multiple-layer perceptrons (MLPs) to extract local features from the point cloud graph structure automatically. Features obtained from all edge convolutional layers are concatenated together to form a feature aggregation. Unlike stacking many layers in-depth, our DGCB-Net employs a broad architecture to extend point cloud feature aggregation flatly. The broad architecture is structured utilizing a flat combining architecture with multiple feature layers and enhancement layers. Both feature layers and enhancement layers concatenate together to further enrich the features’ information of the point cloud. All features work on the object recognition results thus that our DGCB-Net show better recognition performance than other 3D object recognition algorithms on ModelNet10/40 and our scanning point cloud dataset.

A fast 3D object recognition algorithm using plane-constrained point pair features

The point pair feature (PPF) algorithm is one of the best-performing 3D object recognition algorithms. However, the high dimensionality of its search space is a disadvantage of this algorithm. This high dimensionality means the feature matching process contains a large number of uninformative features, which reduces recognition speed. To solve this problem and improve the object recognition speed, this paper proposes a fast 3D object recognition algorithm based on the plane-constrained point pair features. By utilizing the property of the coplanar point pair features and the characteristics of the object placement plane, the proposed algorithm extracts the object placement plane through convex hull area calculation, eliminates irrelevant point pair features, and then performs object recognition with the reduced point pair feature descriptors for the feature matching. Experimental results demonstrate that the proposed algorithm significantly reduces the number of feature descriptors and accelerates the recognition speed of 3D objects in a complex background. Compared to the original point pair feature algorithm, the proposed method can achieve better performance and efficiency for 3D object recognition.

Search inliers based on redundant geometric constraints

This paper presents an efficient correspondence grouping algorithm to search inliers from an initial set of feature matches. The novelty lies in the proposal of a scoring technique for measuring the reliability of a triple combination (three pairs of matches) based on redundant geometric constraints. According to the proposed scoring method, several top-ranking triple combinations are selected for estimating the transformation hypotheses between two 3D shapes. For each transformation hypothesis, a correspondence from a selected correspondence set should cast a vote whether it is satisfying the geometric constraint with it. Finally, the transformation hypothesis with the most votes is considered as the best transformation and the correspondences from the initial correspondence set agreeing with the best transformation are grouped as inliers. We performed both comparative experiments and real application experiments to evaluate the performance of our proposed method on five popular datasets. The experimental results show the superior performance of our method with respect to different levels of noise, point density variation, partial overlap, clutter and occlusion. In addition, our proposed method can boost the performance of a feature-based 3D object recognition algorithm, giving an increase in both high recognition rate and computational efficiency.

3D object recognition based on pairwise Multi-view Convolutional Neural Networks

With the development of 3D sensors, it will be much easier for us to obtain 3D models, which is prevailing in our future daily life, but up to now, although many 3D object recognition algorithms have been proposed, there are some limitations, including the lack of training samples, hand-crafted feature representation, feature extraction and recognition separately. In this work, we propose a novel pairwise Multi-View Convolutional Neural Network for 3D Object Recognition (PMV-CNN for short), where automatic feature extraction and object recognition are put into a unify CNN architecture. Moreover, since the pairwise network architecture is utilized in PMV-CNN, thus, the requirement of the number of training samples in the original dataset is not severe. In addition, the latent complementary relationships from different views can be highly explored by view pooling. Large scale experiments demonstrate that the pairwise architecture is very useful when the number of labeled training samples is very small. Moreover, it also makes more robust feature extraction. Furthermore, since the end-to-end network architecture is employed in PMV-CNN, thus, the extracted feature is very suitable for 3D object recognition, whose performance is much better than that of hand-crafted features. In a word, the performance of our proposed method outperforms state-of-the-art methods.

Context-aware 3D object anchoring for mobile robots

A world model representing the elements in a robot’s environment needs to maintain a correspondence between the objects being observed and their internal representations, which is known as the anchoring problem. Anchoring is a key aspect for an intelligent robot operation, since it enables high-level functions such as task planning and execution. This work presents an anchoring system that continually integrates new observations from a 3D object recognition algorithm into a probabilistic world model. Our system takes advantage of the contextual relations inherent to human-made spaces in order to improve the classification results of the baseline object recognition system. To achieve that, the system builds a graph-based world model containing the objects in the scene (both in the current and previously perceived observations), which is exploited by a Probabilistic Graphical Model (PGM) in order to leverage contextual information during recognition. The world model also enables the system to exploit information about objects beyond the current field of view of the robot sensors. Most importantly, this is done in an online fashion, overcoming both the disadvantages of single-shot recognition systems (e.g., limited sensor aperture) and offline recognition systems that require prior registration of all frames of a scene (e.g., dynamic scenes, unsuitability for plan-based robot control). We also propose a novel way to include the outcome of local object recognition methods in the PGM, which results in a decrease in the usually high model learning complexity and an increase in the system performance. The system performance has been assessed with a dataset collected by a mobile robot from restaurant-like settings, obtaining positive results for both its data association and object recognition capabilities. The system has been successfully used in the RACE robotic architecture.

Simultaneous 3D Object Recognition and Pose Estimation Based on RGB-D Images

Object recognition and pose estimation are essential functions in applications of computer vision, and they also are fundamental modules in robotic vision systems. In recent years, RGB-D cameras become more and more popular, and the 3D object recognition technology has got more and more attention. In this paper, a novel design of simultaneous 3D object recognition and pose estimation algorithm is proposed based on RGB-D images. The proposed system converts the input RGB-D image to colored point cloud data and extracts features of the scene from the colored point cloud. Then, the existing color signature of histograms of orientations (CSHOT) description algorithm is employed to build descriptors of the detected features based on local texture and shape information. Given the extracted feature descriptors, a two-stage matching process is performed to find correspondences between the scene and a colored point cloud model of an object. Next, a Hough voting algorithm is used to filter out matching errors in the correspondence set and estimate the initial 3D pose of the object. Finally, the pose estimation stage employs RANdom SAmple Consensus (RANSAC) and hypothesis verification algorithms to refine the initial pose and filter out poor estimation results with error hypotheses. Experimental results show that the proposed system not only successfully recognizes the object in a complex scene but also accurately estimates the 3D pose information of the object with respect to the camera.

3D object recognition using scale-invariant features

As 3D scanning technology develops, it becomes easier to acquire various 3D surface data; thus, there is a growing need for 3D data registration and recognition technology. Many existing studies use local descriptors using local surface patches, and most of them use a fixed support radius, so they cannot cope perfectly when the model and scene have different scales. In this study, we propose a perfectly scale-invariant feature selection algorithm by extending the 2D SIFT algorithm (Lowe in Int J Comput Vis 60(2):91–110, 2004) to a 3D mesh. The feature selection method proposed in this study can obtain highly repeatable feature points and support radii regardless of mesh scale. The selected features can effectively describe the local information by the new shape descriptor proposed in this study. Unlike existing shape descriptors, it is possible to perform scale-invariant 3D object recognition and achieve a high recognition rate when combined with the feature point selection algorithm proposed in this study by using the gradients of the scalar functions defined on the 3D surface. We also reduced the searching space and lowered the false positive rate by suggesting a new RANSAC-based transformation hypotheses generation algorithm. Our 3D object recognition algorithm achieves recognition rates of 100 and 98.5%, respectively, when tested on the U3OR and CFVD datasets, exceeding the results of previous studies.

Multi-sensor 3D object dataset for object recognition with full pose estimation

In this work, we propose a new dataset for 3D object recognition using the new high-resolution Kinect V2 sensor and some other popular low-cost devices like PrimeSense Carmine. Since most already existing datasets for 3D object recognition lack some features such as 3D pose information about objects in the scene, per pixel segmentation or level of occlusion, we propose a new one combining all this information in a single dataset that can be used to validate existing and new 3D object recognition algorithms. Moreover, with the advent of the new Kinect V2 sensor we are able to provide high-resolution data for RGB and depth information using a single sensor, whereas other datasets had to combine multiple sensors. In addition, we will also provide semiautomatic segmentation and semantic labels about the different parts of the objects so that the dataset could be used for testing robot grasping and scene labeling systems as well as for object recognition.

Recognizing objects in 3D point clouds with multi-scale local features.

Recognizing 3D objects from point clouds in the presence of significant clutter and occlusion is a highly challenging task. In this paper, we present a coarse-to-fine 3D object recognition algorithm. During the phase of offline training, each model is represented with a set of multi-scale local surface features. During the phase of online recognition, a set of keypoints are first detected from each scene. The local surfaces around these keypoints are further encoded with multi-scale feature descriptors. These scene features are then matched against all model features to generate recognition hypotheses, which include model hypotheses and pose hypotheses. Finally, these hypotheses are verified to produce recognition results. The proposed algorithm was tested on two standard datasets, with rigorous comparisons to the state-of-the-art algorithms. Experimental results show that our algorithm was fully automatic and highly effective. It was also very robust to occlusion and clutter. It achieved the best recognition performance on all of these datasets, showing its superiority compared to existing algorithms.

A novel local surface feature for 3D object recognition under clutter and occlusion

This paper presents a highly distinctive local surface feature called the TriSI feature for recognizing 3D objects in the presence of clutter and occlusion. For a feature point, we first construct a unique and repeatable Local Reference Frame (LRF) using the implicit geometrical information of neighboring triangular faces. We then generate three signatures from the three orthogonal coordinate axes of the LRF. These signatures are concatenated and then compressed into a TriSI feature. Finally, we propose an effective 3D object recognition algorithm based on hierarchical feature matching. We tested our TriSI feature on two popular datasets. Rigorous experimental results show that the TriSI feature was highly descriptive and outperformed existing algorithms under all levels of Gaussian noise, Laplacian noise, shot noise, varying mesh resolutions, occlusion, and clutter. Moreover, we tested our TriSI-based 3D object recognition algorithm on four standard datasets. The experimental results show that our algorithm achieved the best overall recognition results on these datasets.

A distributed computational cognitive model for object recognition

Based on cognitive functionalities in human vision processing, we propose a computational cognitive model for object recognition with detailed algorithmic descriptions. The contribution of this paper is of two folds. Firstly, we present a systematic review on psychological and neurophysiological studies, which provide collective evidence for a distributed representation of 3D objects in the human brain. Secondly, we present a computational model which simulates the distributed mechanism of object vision pathway. Experimental results show that the presented computational cognitive model outperforms five representative 3D object recognition algorithms in computer science research.

Rotational Projection Statistics for 3D Local Surface Description and Object Recognition

Recognizing 3D objects in the presence of noise, varying mesh resolution, occlusion and clutter is a very challenging task. This paper presents a novel method named Rotational Projection Statistics (RoPS). It has three major modules: local reference frame (LRF) definition, RoPS feature description and 3D object recognition. We propose a novel technique to define the LRF by calculating the scatter matrix of all points lying on the local surface. RoPS feature descriptors are obtained by rotationally projecting the neighboring points of a feature point onto 2D planes and calculating a set of statistics (including low-order central moments and entropy) of the distribution of these projected points. Using the proposed LRF and RoPS descriptor, we present a hierarchical 3D object recognition algorithm. The performance of the proposed LRF, RoPS descriptor and object recognition algorithm was rigorously tested on a number of popular and publicly available datasets. Our proposed techniques exhibited superior performance compared to existing techniques. We also showed that our method is robust with respect to noise and varying mesh resolution. Our RoPS based algorithm achieved recognition rates of 100, 98.9, 95.4 and 96.0 % respectively when tested on the Bologna, UWA, Queen’s and Ca’ Foscari Venezia Datasets.

On-machine 3D vision system for machining setup modeling

In computer numerical control machine tools, using machining simulation to prevent collision becomes more and more popular due to its efficiency and low cost. However, if the entire digital model of the machining setup does not exist, the simulation is not applicable. As a result, the operator has to manually check the numerical control program, which is a time-consuming and error-prone process. In this paper, an on-machine vision system is presented to quickly construct the digital model based on the actual machining setup. The total construction for a complex setup can be done within a few minutes. Several key technologies have been developed. First, a 2D edge feature detection algorithm has been designed which will extract the edges of the object of interest by processing both the real and virtual images. Second, a stereo vision system is developed which will obtain the three-dimensional (3D) edge data of the object of interest. A new algorithm is presented to solve correspondence, which is the key problem of the stereovision system. Furthermore, the 3D object recognition algorithm is developed to let the system intelligently search for the matched solid model in the database and import it into the virtual environment with an accurate pose. Finally, experiments are carried out to test the developed system.

3D object recognition based on curvature information of planar views

In this paper, a 3D object recognition algorithm is proposed. Objects are recognized by studying planar images corresponding to a sequence of views. Planar shape contours are represented by their adaptively calculated curvature functions, which are decomposed in the Fourier domain as a linear combination of a set of representative shapes. Finally, sequences of views are identified by means of Hidden Markov Models. The proposed system has been tested for artificial and real objects. Distorted and noisy versions of the objects were correctly clustered together.

Model-Based Object Recognition Using Geometric Invariants of Points and Lines

In this paper, we derive new geometric invariants for structured 3D points and lines from single image under projective transform, and we propose a novel model-based 3D object recognition algorithm using them. Based on the matrix representation of the transformation between space features (points and lines) and the corresponding projected image features, new geometric invariants are derived via the determinant ratio technique. First, an invariant for six points on two adjacent planes is derived, which is shown to be equivalent to Zhu's result [1], but in simpler formulation. Then, two new geometric invariants for structured lines are investigated: one for five lines on two adjacent planes and the other for six lines on four planes. By using the derived invariants, a novel 3D object recognition algorithm is developed, in which a hashing technique with thresholds and multiple invariants for a model are employed to overcome the over-invariant and false alarm problems. Simulation results on real images show that the derived invariants remain stable even in a noisy environment, and the proposed 3D object recognition algorithm is quite robust and accurate.

Probabilistic 3D Object Recognition

A probabilistic 3D object recognition algorithm is presented. In order to guide the recognition process the probability that match hypotheses between image features and model features are correct is computed. A model is developed which uses the probabilistic peaking effect of measured angles and ratios of lengths by tracing iso-angle and iso-ratio curves on the viewing sphere. The model also accounts for various types of uncertainty in the input such as incomplete and inexact edge detection. For each match hypothesis the pose of the object and the pose uncertainty which is due to the uncertainty in vertex position are recovered. This is used to find sets of hypotheses which reinforce each other by matching features of the same object with compatible uncertainty regions. A probabilistic expression is used to rank these hypothesis sets. The hypothesis sets with the highest rank are output. The algorithm has been fully implemented, and tested on real images.

On view likelihood and stability

We define two measures on views: view likelihood and view stability. View likelihood measures the probability that a certain view of a given 3D object is observed; it may be used to identify typical, or characteristic views. View stability measures how little the-image changes as the viewpoint is slightly perturbed; it may be used to identify generic views. Both definitions are shown to be identical up to the prior probability of camera orientations, and determined by the 2D metric used to compare images. We analytically derive the stability and likelihood measures for two feature-based 2D metrics, where the most stable and most likely view is shown to be the flattest view of the 3D shape. Incorporating view likelihood or stability in 3D object recognition and 3D reconstruction increases the chance of robust performance. In particular, we propose to use these measures to enhance 3D object recognition and 3D reconstruction algorithms, by adding a second step where the most likely solution is selected among all feasible solutions. These applications are demonstrated using simulated and real images.

Generating an interpretation tree from a CAD model for 3D-object recognition in bin-picking tasks

This article describes a method to generate 3D-object recognition algorithms from a geometrical model for bin-picking tasks. Given a 3D solid model of an object, we first generate apparent shapes of an object under various viewer directions. Those apparent shapes are then classified into groups (representative attitudes) based on dominant visible faces and other features. Based on the grouping, recognition algorithms are generated in the form of an interpretation tree. The interpretation tree consists of two parts: the first part for classifying a target region in an image into one of the shape groups, and the second part for determining the precise attitude of the object within that group. We have developed a set of rules to find out what appropriate features are to be used in what order to generate an efficient and reliable interpretation tree. Features used in the interpretation tree include inertia of a region, relationship to the neighboring regions, position and orientation of edges, and extended Gaussian images. This method has been applied in a task for bin-picking objects that include both planar and cylindrical surfaces. As sensory data, we have used surface orientations from photometric stereo, depth from binocular stereo using oriented-region matching, and edges from an intensity image.