3D Recognition Research Articles

Long-Distance Multi-Vehicle Detection at Night Based on Gm-APD Lidar

Long-distance multi-vehicle detection at night is critical in military operations. Due to insufficient light at night, the visual features of vehicles are difficult to distinguish, and many missed detections occur. This paper proposes a two-level detection method for long-distance nighttime multi-vehicles based on Gm-APD lidar intensity images and point cloud data. The method is divided into two levels. The first level is 2D detection, which enhances the local contrast of the intensity image and improves the brightness of weak and small objects. With the confidence threshold set, the detection result greater than the threshold is reserved as a reliable object, and the detection result less than the threshold is a suspicious object. In the second level of 3D recognition, the suspicious object area from the first level is converted into the corresponding point cloud classification judgment, and the object detection score is obtained through comprehensive judgment. Finally, the object results of the two-level recognition are merged into the final detection result. Experimental results show that the method achieves a detection accuracy of 96.38% and can effectively improve the detection accuracy of multiple vehicles at night, which is better than the current state-of-the-art detection methods.

3D point cloud recognition of substation equipment based on plane detection

Substation equipment identification is a key step in the process of intelligent substation designing. The target recognition of 3D point cloud of substation equipment firstly uses a 3D laser scanner to obtain the 3D point cloud data that can express the surface shape and spatial position of a substation device, then identify the device information by matching it with template point cloud data. Because the point cloud data obtained by 3D laser scanner is affected by equipment attitude and substation environment, there is redundant, missing, and shadow phenomena in the data, resulting in a low algorithm recognition rate. This paper proposes a method combining plane detection and point cloud registration algorithm, achieving substation equipment identification better result. The main steps of the method include: point cloud preprocessing, plane detecting, preliminary filtering based on Umeyama registration method, and Iterative Closest Point (ICP) algorithm identifying. Plane detection is used for the pre-processed point cloud to obtain planar features, then samples and the planar feature templates in template library are preliminarily selected by Umeyama registration method, and substation equipment is finally identified by Iterative Closest Point algorithm. Experiments show that the recognition rate of devices point cloud reaches 92.2% and the recognition time is shorter, which proves that this recognition algorithm has a good recognition effect.

Two-photon interference LiDAR imaging.

Optical Coherence Tomography (OCT) is a key 3D imaging technology that provides micron scale depth resolution for bio-imaging. This resolution substantially surpasses what it typically achieved in Light Detection and Ranging (LiDAR) which is often limited to the millimetre scale due to the impulse response of the detection electronics. However, the lack of coherence in LiDAR scenes, arising from mechanical motion for example, make OCT practically infeasible. Here we present a quantum interference inspired approach to LiDAR which achieves OCT depth resolutions without the need for high levels of stability. We demonstrate depth imaging capabilities with an effective impulse response of 70 μm, thereby allowing ranging and multiple reflections to be discerned with much higher resolution than conventional LiDAR approaches. This enhanced resolution opens up avenues for LiDAR in 3D facial recognition, and small feature detection/tracking as well as enhancing the capabilities of more complex time-of-flight methods such as imaging through obscurants and non-line-of-sight imaging.

Erratum to: 3D Recognition: State of the Art and Trends

Erratum to: 3D Recognition: State of the Art and Trends

Exploring object and multi-target instrument tracking for AR-guided interventions

Abstract The rapid development of available hard- and software for computer-assisted or augmented reality (AR) guided interventions creates a need for fast and inexpensive prototyping environments. However, intraoperative tracking systems in particular represent a high cost threshold. Therefore, this work presents a low-cost tracking method based on a conventional RGB camera. Here, a combined approach of multiple image targets and 3D object target recognition is implemented. The system is evaluated with a systematic accuracy assessment analyzing a total of 385 3D positions. On average, a deviation of 15,69+-9,95 mm was measured. In addition, a prototypical AR-based needle navigation visualization was developed using Microsoft HoloLens 2. This system’s feasibility and usability was evaluated positively in a pilot study (n=3).

Space Modeling Design Art of Suspense Movies Based on Machine Vision

After suspense movie space has a substantial role, it plays an irreplaceable role in movie narration. Different spatial forms can give us different feelings and different spatial feelings and can cooperate with different stories to create a different atmosphere. In this paper, a new 3D human behavior recognition algorithm in MV (machine vision) perception is proposed. The new algorithm is achieved by collecting the human behavior images and detecting the contour areas of the images. By reducing the projection area of the spatial positioning error area of unknown nodes on the plane, the positioning accuracy of unknown nodes in each axis direction is improved. The necessary brightness correction and color saturation compensation are carried out in color space. The results show that the average recognition rate of human behavior in MV perception is 95%, which can effectively improve the recognition accuracy. The enhanced color image is not only clearer but also more vivid and bright, which achieves the expected purpose.

A Transformer-Based Capsule Network for 3D Part-Whole Relationship Learning.

Learning the relationship between the part and whole of an object, such as humans recognizing objects, is a challenging task. In this paper, we specifically design a novel neural network to explore the local-to-global cognition of 3D models and the aggregation of structural contextual features in 3D space, inspired by the recent success of Transformer in natural language processing (NLP) and impressive strides in image analysis tasks such as image classification and object detection. We build a 3D shape Transformer based on local shape representation, which provides relation learning between local patches on 3D mesh models. Similar to token (word) states in NLP, we propose local shape tokens to encode local geometric information. On this basis, we design a shape-Transformer-based capsule routing algorithm. By applying an iterative capsule routing algorithm, local shape information can be further aggregated into high-level capsules containing deeper contextual information so as to realize the cognition from the local to the whole. We performed classification tasks on the deformable 3D object data sets SHREC10 and SHREC15 and the large data set ModelNet40, and obtained profound results, which shows that our model has excellent performance in complex 3D model recognition and big data feature learning.

The 3D Deburring Processing Trajectory Recognition Method and Its Application Base on Random Sample Consensus

As of 2022, most automatic deburring trajectories are still generated using offline programming methods. The trajectories generated using these methods are often suboptimal, which limits the precision of the robotic arms used to perform automatic deburring and, in turn, results in workpiece dimensional errors. Therefore, despite advances in automated deburring trajectory generation, deburring is still mostly performed manually. However, manual deburring is a time-consuming, labor-intensive, and expensive process that results in small profit margins for organizational equipment manufacturers (OEMs). To address these problems and the obstacles to the implementation of automated deburring in the robotics industry, the present study developed an online automated deburring trajectory generation method that uses 2D contouring information obtained from linear contour scanning sensors, a CAD model, and curve fitting to detect burrs and generate appropriate trajectories. The method overcomes many of the limitations of common deburring methods, especially by enabling real-time trajectory tracking. When the method was tested using bicycle forks, work that originally took three to four people 8–12-h to complete was completed by one person in 30 min, and the production cost was reduced by 70%.

Multifeature Fusion Human Pose Tracking Algorithm Based on Motion Image Analysis

Human pose and motion detection is an important area of computer vision research, covering the interplay of different fields such as image processing, pattern recognition, and artificial intelligence. Due to the complexity of human motion, existing 3D recognition and pose detection methods based on low-quality depth images are not very accurate and reliable. Due to the time-sensitive nature of features, a single feature cannot adapt to the dynamic changes of the scene, so it is difficult for the target tracking algorithm based on a single feature to achieve robust tracking results. If multiple features are fused and applied in the tracking algorithm, the complementarity between different features can be used to better adapt to the scene changes and achieve robust tracking results. In order to solve the problem of human pose and human motion recognition in low-quality depth images, this paper uses the Kinect somatosensory camera to obtain 20 human skeleton joint points through the Kinect skeleton tracking technology. This paper studies the typical human posture and interactive action recognition technology in daily life. On the basis of understanding the characteristics of skeletal data, this paper proposes a distance feature and angle feature model combined with human body structure. Through the experimental results, it is found that the distance feature and the angle feature value are basically not affected by the distance change. When the subjects turned 45° to the left and 45° to the right, the distance characteristics changed, which was different from the characteristic data when the subjects turned around.

VFMVAC: View-filtering-based multi-view aggregating convolution for 3D shape recognition and retrieval

Multi-view based 3D shape recognition methods have achieved state-of-the-art performance in 3D shape recognition and retrieval. The main focus of multi-view based approaches is determining how to fuse multi-view features into a compact, descriptive, and robust 3D shape descriptor that can then be utilized for 3D shape recognition and retrieval. This paper proposes a novel multi-view aggregating framework, view-filtering-based multi-view aggregating convolution (VFMVAC) to learn global shape descriptors for 3D shape recognition. The proposed VFMVAC applies a voting-based view filtering strategy to select representative views, also introduces a novel multi-view aggregating module to integrate multi-view features; this substantially improves the descriptiveness of the descriptors, and therefore improves the performance of 3D shape recognition and retrieval. Specifically, all views are fed into a voting-based view filtering module to select the top-k representative views. Subsequently, the features of the top-k views are fed into the multi-view aggregating module, which first conducts cross-view channel shuffle for achieving cross-view information flowing, and the resulted reshaped features are then fed into the aggregating convolution module for feature fusion. Experiments on benchmark datasets demonstrate that the proposed VFMVAC is effective and outperforms several recent techniques with respect to the classification and retrieval performance, robustness and efficiency.

Touchless palmprint recognition based on 3D Gabor template and block feature refinement

With the growing demand for hand hygiene, touchless palmprint recognition has made great strides recently, providing an effective solution for personal identification. Despite many efforts devoted to this area, the discriminative ability of touchless palmprint is still uncertain, especially for large-scale datasets. To address this problem, in this study, a large-scale touchless palmprint dataset was built containing 2334 palms from 1167 individuals. To the best of our knowledge, this is currently the largest touchless palmprint image benchmark regarding the number of individuals and palms. In addition, a novel deep learning framework was proposed for touchless palmprint recognition, referred to as 3D convolution palmprint recognition network (3DCPN), which leverages 3D convolutions to dynamically integrate multiple Gabor features. In 3DCPN, a novel variant of the Gabor filter is embedded into the first layer to enhance curve feature extraction. Using a well-designed ensemble scheme, low-level 3D features are convolved to extract high-level features. Finally, a region-based loss function is implemented to strengthen the discriminative ability of both global and local descriptors. Extensive experiments were conducted on the newly built dataset and other popular databases to demonstrate the superiority of our method. The results show that the proposed 3DCPN achieves state-of-the-art or comparable performance.

A Generic View Planning System Based on Formal Expression of Perception Tasks.

View planning (VP) is a technique that guides the adjustment of the sensor’s postures in multi-view perception tasks. It converts the perception process into active perception, which improves the intelligence and reduces the resource consumption of the robot. We propose a generic VP system for multiple kinds of visual perception. The VP system is built on the basis of the formal description of the visual task, and the next best view is calculated by the system. When dealing with a given visual task, we can simply update its description as the input of the VP system, and obtain the defined best view in real time. Formal description of the perception task includes the task’s status, the objects’ prior information library, the visual representation status and the optimization goal. The task’s status and the visual representation status are updated when data are received at a new view. If the task’s status has not reached its goal, candidate views are sorted based on the updated visual representation status, and the next best view that can minimize the entropy of the model space is chosen as the output of the VP system. Experiments of view planning for 3D recognition and reconstruction tasks are conducted, and the result shows that our algorithm has good performance on different tasks.

Score-Level Fusion of 3D Face and 3D Ear for Multimodal Biometric Human Recognition.

A novel multimodal biometric system is proposed using three-dimensional (3D) face and ear for human recognition. The proposed model overcomes the drawbacks of unimodal biometric systems and solves the 2D biometric problems such as occlusion and illumination. In the proposed model, initially, the principal component analysis (PCA) is utilized for 3D face recognition. Thereafter, the iterative closest point (ICP) is utilized for 3D ear recognition. Finally, the 3D face is fused with a 3D ear using score-level fusion. The simulations are performed on the Face Recognition Grand Challenge database and the University of Notre Dame Collection F database for 3D face and 3D ear datasets, respectively. Experimental results reveal that the proposed model achieves an accuracy of 99.25% using the proposed score-level fusion. Comparative analyses show that the proposed method performs better than other state-of-the-art biometric algorithms in terms of accuracy.

3D Recognition: State of the Art and Trends

3D Recognition: State of the Art and Trends

MinkLoc3D-SI: 3D LiDAR Place Recognition With Sparse Convolutions, Spherical Coordinates, and Intensity

The 3D LiDAR place recognition aims to estimate a coarse localization in a previously seen environment based on a single scan from a rotating 3D LiDAR sensor. The existing solutions to this problem include hand-crafted point cloud descriptors (e.g., ScanContext, M2DP, LiDAR IRIS) and deep learning-based solutions (e.g., PointNetVLAD, PCAN, LPD-Net, DAGC, MinkLoc3D), which are often only evaluated on accumulated 2D scans from the Oxford RobotCar dataset. We introduce MinkLoc3D-SI, a sparse convolution-based solution that utilizes spherical coordinates of 3D points and processes the intensity of 3D LiDAR measurements, improving the performance when a single 3D LiDAR scan is used. Our method integrates the improvements typical for hand-crafted descriptors (like ScanContext) with the most efficient 3D sparse convolutions (MinkLoc3D). Our experiments show improved results on single scans from 3D LiDARs (USyd Campus dataset) and great generalization ability (KITTI dataset). Using intensity information on accumulated 2D scans (RobotCar Intensity dataset) improves the performance, even though spherical representation doesn’t produce a noticeable improvement. As a result, MinkLoc3D-SI is suited for single scans obtained from a 3D LiDAR, making it applicable in autonomous vehicles.

SC-LPR: Spatiotemporal context based LiDAR place recognition

• A spatiotemporal and semantic information fusion based LiDAR place recognition method improves recognition accuracy. • A novel feature encoding module is proposed to effectively aggregate spatiotemporal features. • Spatiotemporal information is fused without geometrical transformations between LiDAR scans. • A superior feature similarity evaluation network is proposed to predict similarity score. 3D LiDAR-based place recognition remains challenging due to ambiguities and dynamics of scenes. Place representation can be learned effectively by deep learning, but perceptual confusion is inevitable based on single-scan. To alleviate the problem, we propose a new place recognition method named SC-LPR. In this method, spatiotemporal contextual information from LiDAR scans is used to increase the capacity of features’ representation. Firstly, a semantic graph is constructed to represent the topological geometric map of each LiDAR scan. Then, an end-to-end network is designed to predict similarity, in which GRU-EdgeConv++ is proposed to learn discriminative spatiotemporal feature representation and a novel C&TNT to predict a score. We evaluate our approach on the KITTI odometry benchmark. The experimental results show that our method can effectively fuse spatiotemporal information for place recognition. Compared with the state-of-the-art methods, our approach is superior on the KITTI dataset and can achieve competitive performance. To benefit the community by serving as a benchmark for place recognition, the code of our method will be made open-source on Github 1 1 https://github.com/Daideyun/SC-LPR .

Beteng Cepuri as Ethnomathematics of Elementary Schools

Ethnomathematical objects can be found in artifacts in the form of historical buildings. One of the historical buildings in Kotagede, Yogyakarta, Indoensia, is the Cepuri Beteng. The purpose of this research is an exploration any mathematical concepts contained in Beteng Cepuri that can be used as a source of learning mathematics as an effort to develop ethnomathematics for learning elementary mathematics. The researchers invited the 5th graders of an elementary school in Kotagede, nearby the Cepuri Beteng, to explore and understand mathematical concepts. The results showed that there was a relationship between the construction of the fort building and the concept of building space in several parts of the Cepuri Beteng. The sections are in the form of blocks, and rectangles. Taking the students to visualize the artifacts may assist them to build 3D shape recognition. It can be described that ethnomathematics is able to provide a very good interest in learning followed by concept acquisition.

A union of deep learning and swarm-based optimization for 3D human action recognition

Human Action Recognition (HAR) is a popular area of research in computer vision due to its wide range of applications such as surveillance, health care, and gaming, etc. Action recognition based on 3D skeleton data allows simplistic, cost-efficient models to be formed making it a widely used method. In this work, we propose DSwarm-Net, a framework that employs deep learning and swarm intelligence-based metaheuristic for HAR that uses 3D skeleton data for action classification. We extract four different types of features from the skeletal data namely: Distance, Distance Velocity, Angle, and Angle Velocity, which capture complementary information from the skeleton joints for encoding them into images. Encoding the skeleton data features into images is an alternative to the traditional video-processing approach and it helps in making the classification task less complex. The Distance and Distance Velocity encoded images have been stacked depth-wise and fed into a Convolutional Neural Network model which is a modified version of Inception-ResNet. Similarly, the Angle and Angle Velocity encoded images have been stacked depth-wise and fed into the same network. After training these models, deep features have been extracted from the pre-final layer of the networks, and the obtained feature representation is optimized by a nature-inspired metaheuristic, called Ant Lion Optimizer, to eliminate the non-informative or misleading features and to reduce the dimensionality of the feature set. DSwarm-Net has been evaluated on three publicly available HAR datasets, namely UTD-MHAD, HDM05, and NTU RGB+D 60 achieving competitive results, thus confirming the superiority of the proposed model compared to state-of-the-art models.

MolNet‐3D: Deep Learning of Molecular Representations and Properties from 3D Topography

AbstractA new paradigm combining quantum‐mechanical calculations with machine learning (ML) to rationally design compounds with specific properties from extremely large chemical space is emerging and developing at a rapid pace. In this context, appropriately describing molecules and efficiently extracting patterns from electronic‐structure calculations are the core challenges for the success of the quantum‐mechanics‐based ML approaches. Here, MolNet‐3D is introduced, a strong deep learning architecture capable of mapping from a flexible and universal 3D topography descriptor to quantum‐mechanical observables of molecules of arbitrary shape. The model can learn an invariant representation without the need for the transformation of atom coordinates into interatomic distances, thus preserving the intrinsic 3D topography information of molecules. The capabilities of MolNet‐3D are shown by accurately predicting the various density functional theory calculated properties for molecules, including energetic, electronic, and thermodynamic properties. Compared with the previously proposed ML methods in the MoleculeNet benchmarks, our model generally offers the best performance in those quantum‐mechanical tasks, elucidating the importance of intrinsic topography information in molecular representation learning. This work may provide new insight into the construction of molecular ML models from 3D topography recognition perspectives.

Three-dimensional localization of gas leakage using acoustic holography

Gas leakage localization is essential to prevent accidents from gas transportation or storage failures. There have been extensive studies on one-dimensional (1D) and two-dimensional (2D) gas leakage localization problem. This paper is the first to report the demonstration of a three-dimensional (3D) localization of gas leaks using far-field acoustic holography. This approach simultaneously provides 2D sound field imaging and depth position information regarding the leakage source. A virtual phased array composed of only two sensors collects the needed data, resulting in improved accuracy, enhanced feasibility and reduced cost. Various parameters, including the number of sensors, the spacing of array elements and the distance to the leakage source, are analyzed by a simulation of leakage sound field reconstruction. A series of experiments validated the effectiveness and accuracy of the proposed approach. Our results demonstrate that the proposed method achieves 3D localization and high-resolution imaging recognition of gas leaks.