Geometry-based Method Research Articles

Deep-learning-based point cloud completion methods: A review

Point cloud completion aims to utilize algorithms to repair missing parts in 3D data for high-quality point clouds. This technology is crucial for applications such as autonomous driving and urban planning. With deep learning’s progress, the robustness and accuracy of point cloud completion have improved significantly. However, the quality of completed point clouds requires further enhancement to satisfy practical requirements. In this study, we conducted an extensive survey of point cloud completion methods, with the following main objectives: (i) We classified point cloud completion methods into categories based on their principles, such as point-based, convolution-based, GAN-based, and geometry-based methods, and thoroughly investigated the advantages and limitations of each category. (ii) We collected publicly available datasets for point cloud completion algorithms and conducted experimental comparisons using various typical deep-learning networks to draw conclusions. (iii) With our research in this paper, we discuss future research trends in this rapidly evolving field.

Enhancing accuracy of surgical stylus-tip tracking: A comparative calibration study

Accurate determination of the position of surgical stylus-tip is crucial for informed decision-making in computer-assisted surgery. This study assesses the accuracy of various calibration methods for stylus-tip tracking. A novel template-based calibration method is proposed, offering a practical protocol for assessing surgical stylus-tip accuracy. Relative position and orientation errors are measured using two custom-made phantoms and compared with those from existing techniques. Experimental results shows that our proposed method achieves greater positional accuracy (mean: 1.73 mm; range: 0.53-3.42 mm) compared to the geometry-based method (mean: 3.78 mm; range: 2.52-5.05 mm) and the algebraic-based method (mean: 2.47 mm; range: 1.52-3.52 mm). In terms of orientation accuracy, our method also performs slightly better (mean: 0.95 deg; range: 0-3.41 deg) than the geometry-based method (mean: 1.03 deg; range: 0-3.36 deg) and the algebraic-based method (mean: 1.01 deg; range: 0-3.22 deg). The advantages of our method are highlighted, demonstrating its potential for application in the operating room.

Multi-Cluster Approaches to Radio Sensor Array Channel Modeling and Simulation.

In this paper, we explore the physical propagation environment of radio waves by describing it in terms of distant scattering clusters. Each cluster consists of numerous scattering objects that may exhibit certain statistical properties. By utilizing geometry-based methods, we can study the channel second-order statistics (CSOS), where each distant scattering cluster corresponds to a CSOS, contributes a portion to the Doppler spectrum, and is associated with a state-space multiple-input and multiple-output (MIMO) radio channel model. Consequently, the physical propagation environment of radio waves can be modeled by summing multiple state-space MIMO radio channel models. This approach offers three key advantages: simplicity, the ability to construct the entire Doppler power spectrum from multiple uncorrelated distant scattering clusters, and the capability to obtain the channels contributed by these clusters by summing the individual channels. This methodology enables the reconstruction of the radio wave propagation environment in a simulated manner and is crucial for developing massive MIMO channel models.

Automatic facial expression recognition under partial occlusion based on motion reconstruction using a denoising autoencoder

Automatic facial expression recognition (FER) plays a valuable role in various fields, including health, road safety, and marketing, where providing feedback on the user’s condition is crucial. While significant progress has been made in controlled environments (such as frontal, unconcluded, and well-lit conditions), recognizing facial expressions in unconstrained environments (natural settings) remains challenging. The presence of occlusions poses a particular difficulty as they obscure parts of the facial information captured in the image. To address this issue, researchers have proposed different solutions, broadly categorized into two approaches: those focusing on visible regions of the face and those attempting to reconstruct hidden parts. Currently, most solutions rely on texture or geometry-based methods, with only a few utilizing motion-based approaches. However, incorporating motion appears to be particularly promising in adapting to occlusions due to its unique characteristics, such as close-range propagation and local coherence. In this paper, our focus lies on leveraging motion to overcome the challenges posed by occlusions in FER tasks.

HPPLO-Net: Unsupervised LiDAR Odometry Using a Hierarchical Point-to-Plane Solver

High-precision LiDAR odometry (LO) plays an essential role in autonomous driving. Generally, due to inaccurate data associations and the existence of outliers, it is a challenging task to estimate ego-motions reliably and efficiently given sparse and complexly distributed point clouds in outdoor environments. In this paper, we propose an unsupervised method for LiDAR odometry, named HPPLO-Net, to predict the relative pose of a LiDAR sensor in a hierarchical way. Specifically, we achieve accurate 6-DoF (Degree of Freedom) pose estimation between the source and target point clouds using a differentiable Point-to-Plane solver with the assistance of scene flow. The novel Point-to-Plane solver consists of a multi-scale aggregation (MSA) normal estimation layer and a differentiable weighted Point-to-Plane SVD module. The MSA layer is introduced to find reliable normal vectors of the pseudo target point cloud by aggregating multi-scale contextual information. The differentiable weighted Point-to-Plane SVD is embedded in the network to solve the pose matrix and alleviate the problem of lacking accurate data association in two LiDAR scans, which exists in the point-to-point alternatives. To reduce the impact of noises and outliers, our method can learn and update the inlier mask and the (residual) flow uncertainty at each layer of the hierarchy. We demonstrate the effectiveness of our method on the KITTI Odometry Dataset, Ford Campus Vision and Lidar DataSet, and the Apollo-SouthBay Dataset. Our method has achieved superior performance than recent unsupervised learning-based methods and some traditional geometry-based methods and has also achieved promising performance close to A-LOAM with mapping optimization. The source code of our method is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/IMRL/HPPLO-Net</uri> .

Riemannian transfer learning based on log-Euclidean metric for EEG classification.

Brain computer interfaces (BCI), which establish a direct interaction between the brain and the external device bypassing peripheral nerves, is one of the hot research areas. How to effectively convert brain intentions into instructions for controlling external devices in real-time remains a key issue that needs to be addressed in brain computer interfaces. The Riemannian geometry-based methods have achieved competitive results in decoding EEG signals. However, current Riemannian classifiers tend to overlook changes in data distribution, resulting in degenerated classification performance in cross-session and/or cross subject scenarios. This paper proposes a brain signal decoding method based on Riemannian transfer learning, fully considering the drift of the data distribution. Two Riemannian transfer learning methods based log-Euclidean metric are developed, such that historical data (source domain) can be used to aid the training of the Riemannian decoder for the current task, or data from other subjects can be used to boost the training of the decoder for the target subject. The proposed methods were verified on BCI competition III, IIIa, and IV 2a datasets. Compared with the baseline that without transfer learning, the proposed algorithm demonstrates superior classification performance. In contrast to the Riemann transfer learning method based on the affine invariant Riemannian metric, the proposed method obtained comparable classification performance, but is much more computationally efficient. With the help of proposed transfer learning method, the Riemannian classifier obtained competitive performance to existing methods in the literature. More importantly, the transfer learning process is unsupervised and time-efficient, possessing potential for online learning scenarios.

Disentangled representation learning for collaborative filtering based on hyperbolic geometry

In the realm of recommender systems, the exploration of hyperbolic geometry-based embeddings for users and items has emerged as a promising avenue, particularly in the context of collaborative filtering through graph convolution networks. Despite the advancements in this domain, there are two significant questions have received limited attention: (i) Most hyperbolic geometry-based methods ignore the influence of different users’ intents on the preferences of historical behaviors; (ii) They usually learn high-dimensional embeddings akin to Euclidean geometry-based methods, without taking good advantage of the capacity of hyperbolic spaces. To tackle these limitations head-on, we propose a novel method called Disentangled Hyperbolic Collaborative Filtering (DHCF). DHCF learns multiple embeddings in different low-dimensional hyperbolic spaces, enabling the disentanglement of users’ intents and the separate modeling of user and item representations. Moreover, we design an intent-aware graph reconstruction module, which adaptively allocates intent-aware relation strengths to build a dynamic interaction graph. Additionally, this module reduces the risk of oversmoothing in low-dimension spaces, facilitating the stacking of multiple graph aggregation layers. To the best of our knowledge, our method achieves competitive performance compared to state-of-the-art approaches.

Slab arching degree identification and evaluation based on track dynamic inspection data

Due to structural deterioration and the external environment, the ballastless track is prone to slab arching during high-temperature periods. Arching will weaken the constraints between components, affect the smoothness of the line, and even endanger train safety in severe cases. Therefore, it is necessary to identify the arching in time and clarify the arching severity, which is of great significance for scientifically formulating maintenance plans and ensuring line stability. The existing research has established track geometry-based methods for arching identification, but they are difficult to recognize small arching and cannot quantitatively evaluate arching degree. To this end, this paper proposed a set of arching identification and evaluation methodology based on track dynamic inspection data. Combining the simulated and measured data, the time–frequency characteristics of car body vertical vibration caused by arching were finely extracted, and the arching-induced vibration index was established for identification. Then, a deep learning hybrid model for arching degree evaluation was proposed based on the analysis of the relationship between multiple feature indexes and arching gap height. The results showed that car body vibration could identify the arching defects with gap heights exceeding 1 mm, and the arching degree could be evaluated quantitatively based on multiple feature indexes. In practice, the proposed method can be used to detect arching of different degrees, assist maintenance decision-making, and help avoid the occurrence of serious failures.

Resilience assessment of mobile emergency generator-assisted distribution networks: A stochastic geometry approach

Escalation of extreme weather events represents substantial threat to power system infrastructure. Mobile emergency generators (MEGs) can form part of a flexible restoration strategy against such destructive events. However, with continued expansion of distribution networks, quantification of the impact of MEGs has become increasingly challenging owing to extreme-weather-event-induced uncertainties. In this paper, we propose a stochastic geometry-based method for assessing the impact of MEG deployment on distribution networks affected by extreme weather events through investigation of structural features. First, we propose a distance measure to represent the electrical connection between power grid components. Subsequently, we adopt the point process and Voronoi tessellation to describe the spatial distribution of power grid components and the service coverage provided by MEGs under different scenarios. Then, we propose a set of assessment metrics to evaluate the survivability of power grid components and the resilience of the entire distribution network under extreme weather events. Finally, we derive accurate analytical expressions for the distance distribution and resilience metrics, such as coverage probability and load shedding, enabling us to explore the relationship between MEG deployment decisions, structural features, and power grid resilience. The proposed method enables analytic assessment of the impact of MEG deployment on the resilience of distribution networks, and provides beneficial insights to help formulate efficient measures for enhancing resilience. Case studies demonstrated that the proposed method is accurate and efficient in dealing with network analysis and assessment problems for distribution networks under massive potential failure scenarios.

A multi-stage neural network approach for coronary 3D reconstruction from uncalibrated X-ray angiography images

We present a multi-stage neural network approach for 3D reconstruction of coronary artery trees from uncalibrated 2D X-ray angiography images. This method uses several binarized images from different angles to reconstruct a 3D coronary tree without any knowledge of image acquisition parameters. The method consists of a single backbone network and separate stages for vessel centerline and radius reconstruction. The output is an analytical matrix representation of the coronary tree suitable for downstream applications such as hemodynamic modeling of local vessel narrowing (i.e., stenosis). The network was trained using a dataset of synthetic coronary trees from a vessel generator informed by both clinical image data and literature values on coronary anatomy. Our multi-stage network achieved sub-pixel accuracy in reconstructing vessel radius (RMSE = 0.16 ± 0.07 mm) and stenosis radius (MAE = 0.27 ± 0.18 mm), the most important feature used to inform diagnostic decisions. The network also led to 52% and 38% reduction in vessel centerline reconstruction errors compared to a single-stage network and projective geometry-based methods, respectively. Our method demonstrated robustness to overcome challenges such as vessel foreshortening or overlap in the input images. This work is an important step towards automated analysis of anatomic and functional disease severity in the coronary arteries.

A geometry-based method for the rapid deployment of a UAV pseudolite navigation system in a target area

A geometry-based method for the rapid deployment of a UAV pseudolite navigation system in a target area

Single-image piecewise planar reconstruction of urban buildings based on geometric priors

Single-image 3D urban building reconstruction is an important and challenging topic in computer vision. However, it is a highly ill-posed problem due to the intrinsic geometrical ambiguity in a 3D space. To address the problem, this paper presents a novel single-image piecewise planar reconstruction method based on geometric priors. The proposed method utilizes geometric priors (e.g., plane orientation and intersection angles between planes) learned through convolutional neural networks to handle three challenging subtasks sequentially: (1) detecting planar regions with structure information (e.g., each component edge is associated with two planes intersecting at a specified angle); (2) inferring the planes corresponding to the planar regions progressively by taking full advantage of diverse constraints (e.g., structure and proximity constraints) based on geometric priors; and (3) globally optimizing the resulting planes under a unified framework that incorporates image cues and geometric priors. Experimental results on two public datasets released by CASIA and our own dataset demonstrate that the proposed method significantly outperforms five state-of-the-art methods (including two geometry-based methods and three learning-based methods) by an approximately 20 % margin in terms of accuracy. Code and data are available at https://github.com/three-dimensional-computer-vision/single-image-reconstruction.

Comparative study on real-time pose estimation of vision-based unmanned underwater vehicles

Background: Navigation and localization are key to the successful execution of autonomous unmanned underwater vehicles (UUVs) in marine environmental monitoring, underwater 3D mapping, and ocean resource surveys. The estimation of the position and the orientation of autonomous UUVs are a long-standing challenging and fundamental problem. As one of the underwater sensors, camera has always been the focus of attention due to its advantages of low cost and rich content information in low visibility waters, especially in the fields of visual perception of the underwater environment, target recognition and tracking. At present, the visual real-time pose estimation technology that can be used for UUVs is mainly divided into geometry-based visual positioning algorithms and deep learning-based visual positioning algorithms. Methods: In order to compare the performance of different positioning algorithms and strategies, this paper uses C++ and python, takes the ORB-SLAM3 algorithm and DF-VO algorithm as representatives to conduct a comparative experiment and analysis. Results: The geometry-based algorithm ORB-SLAM3 is less affected by illumination, performs more stably in different underwater environments, and has a shorter calculation time, but its robustness is poor in complex environments. The visual positioning algorithm DF-VO based on deep learning takes longer time to compute, and the positioning accuracy is more easily affected by illumination, especially in dark conditions. However, its robustness is better in unstructured environments such as large-scale image rotation and dynamic object interference. Conclusions: In general, the deep learning-based algorithm is more robust, but multiple deep learning networks make it need more time to compute. The geometry-based method costs less time and is more accurate in low-light and turbid underwater conditions. However, in real underwater situations, these two methods can be connected as binocular vision or methods of multi-sensor combined pose estimation.

RGB-D SLAM in indoor dynamic environments with two channels based on scenario classification

In visual simultaneous localization and mapping (SLAM) systems, the limitations of the assumption of scene rigidity are usually broken by using learning-based or geometry-based methods. However, learning-based methods usually have a high time cost, and geometry-based methods usually do not result in clean maps which are useful for advanced robotic applications. In this paper, an RGB-D SLAM in indoor dynamic environments with two channels that classifies frames as slightly and highly dynamic scenarios based on matching accuracy is proposed. And a geometric constraint based on Hamming distance is proposed to improve the effectiveness of matching accuracy as a basis for scenario classification. Dynamic features are detected by affine consistency constraint and semantic method. The semantic method is only used for highly dynamic scenarios to reduce the time cost of dynamic feature detection and provide a basis for mapping. Furthermore, an improved adaptive threshold algorithm is proposed to improve the robustness of feature matching. The proposed method is evaluated in the TUM RGB-D dataset and a real scenario. The experimental results demonstrate that the proposed method achieves highly accurate tracking with appreciable time cost in both slightly and highly indoor dynamic environments while obtaining effective maps.

STATE: Learning structure and texture representations for novel view synthesis

Novel viewpoint image synthesis is very challenging, especially from sparse views, due to large changes in viewpoint and occlusion. Existing image-based methods fail to generate reasonable results for invisible regions, while geometry-based methods have difficulties in synthesizing detailed textures. In this paper, we propose STATE, an end-to-end deep neural network, for sparse view synthesis by learning structure and texture representations. Structure is encoded as a hybrid feature field to predict reasonable structures for invisible regions while maintaining original structures for visible regions, and texture is encoded as a deformed feature map to preserve detailed textures. We propose a hierarchical fusion scheme with intra-branch and inter-branch aggregation, in which spatio-view attention allows multi-view fusion at the feature level to adaptively select important information by regressing pixel-wise or voxel-wise confidence maps. By decoding the aggregated features, STATE is able to generate realistic images with reasonable structures and detailed textures. Experimental results demonstrate that our method achieves qualitatively and quantitatively better results than state-of-the-art methods. Our method also enables texture and structure editing applications benefiting from implicit disentanglement of structure and texture. Our code is available at http://cic.tju.edu.cn/faculty/likun/projects/STATE.

EEG-based emergency braking intention detection during simulated driving

BackgroundCurrent research related to electroencephalogram (EEG)-based driver’s emergency braking intention detection focuses on recognizing emergency braking from normal driving, with little attention to differentiating emergency braking from normal braking. Moreover, the classification algorithms used are mainly traditional machine learning methods, and the inputs to the algorithms are manually extracted features.MethodsTo this end, a novel EEG-based driver’s emergency braking intention detection strategy is proposed in this paper. The experiment was conducted on a simulated driving platform with three different scenarios: normal driving, normal braking and emergency braking. We compared and analyzed the EEG feature maps of the two braking modes, and explored the use of traditional methods, Riemannian geometry-based methods, and deep learning-based methods to predict the emergency braking intention, all using the raw EEG signals rather than manually extracted features as input.ResultsWe recruited 10 subjects for the experiment and used the area under the receiver operating characteristic curve (AUC) and F1 score as evaluation metrics. The results showed that both the Riemannian geometry-based method and the deep learning-based method outperform the traditional method. At 200 ms before the start of real braking, the AUC and F1 score of the deep learning-based EEGNet algorithm were 0.94 and 0.65 for emergency braking vs. normal driving, and 0.91 and 0.85 for emergency braking vs. normal braking, respectively. The EEG feature maps also showed a significant difference between emergency braking and normal braking. Overall, based on EEG signals, it was feasible to detect emergency braking from normal driving and normal braking.ConclusionsThe study provides a user-centered framework for human–vehicle co-driving. If the driver's intention to brake in an emergency can be accurately identified, the vehicle's automatic braking system can be activated hundreds of milliseconds earlier than the driver's real braking action, potentially avoiding some serious collisions.

A model for permeability in fibre-reinforced plastics

Increased use of continuous fibre-reinforced plastics (CoFRP) is being seen in the automotive industry due to their high strength specific properties. The manufacturing process, however, is still expensive due to the number of critical steps and material costs. Cost reduction is being combated by computational modelling of the infiltration and curing processes to predict void formation and other potential defects. The accuracy of these simulations is highly dependent on capturing the presence of the carbon sheets in the mould due to the large differences in permeability between flows parallel and normal to the fibre tows. This work presents a geometry-based method for locally orienting the fibre and thickness direction for 2D extruded CoFRP components. The capabilities of these methods will be presented by comparing the fibre orientation prediction for two geometries (i.e., hat channel and double dome) using three different draping schemes (i.e., 0°, 45°, and 90°) against the results of a validated draping simulation developed in LS-DYNA.

An automated, geometry-based method for hippocampal shape and thickness analysis

The hippocampus is one of the most studied neuroanatomical structures due to its involvement in attention, learning, and memory as well as its atrophy in ageing, neurological, and psychiatric diseases. Hippocampal shape changes, however, are complex and cannot be fully characterized by a single summary metric such as hippocampal volume as determined from MR images. In this work, we propose an automated, geometry-based approach for the unfolding, point-wise correspondence, and local analysis of hippocampal shape features such as thickness and curvature. Starting from an automated segmentation of hippocampal subfields, we create a 3D tetrahedral mesh model as well as a 3D intrinsic coordinate system of the hippocampal body. From this coordinate system, we derive local curvature and thickness estimates as well as a 2D sheet for hippocampal unfolding. We evaluate the performance of our algorithm with a series of experiments to quantify neurodegenerative changes in Mild Cognitive Impairment and Alzheimer’s disease dementia. We find that hippocampal thickness estimates detect known differences between clinical groups and can determine the location of these effects on the hippocampal sheet. Further, thickness estimates improve classification of clinical groups and cognitively unimpaired controls when added as an additional predictor. Comparable results are obtained with different datasets and segmentation algorithms. Taken together, we replicate canonical findings on hippocampal volume/shape changes in dementia, extend them by gaining insight into their spatial localization on the hippocampal sheet, and provide additional, complementary information beyond traditional measures. We provide a new set of sensitive processing and analysis tools for the analysis of hippocampal geometry that allows comparisons across studies without relying on image registration or requiring manual intervention.

Inferring slip-surface geometry and volume of creeping landslides based on InSAR: A case study in Jinsha River basin

The slip-surface geometry and volume of landslides are fundamental for landslide modeling and mechanism interpretation. The movement of a landslide, which is generally controlled by the slip-surface geometry, can be obtained from interferometric synthetic aperture radar (InSAR) measurements. However, there is a lack of a general approach for inferring the slip-surface geometry and volume of landslides through the InSAR-derived deformation field. Here we developed a geometry-based method to determine the landslide slip-surface geometry and volume using InSAR measurements and applied it to the Jinsha River Basin, a landslide-prone area that poses a significant threat to residents and infrastructure. Through the InSAR-derived displacement, topography, and Google Earth images, 50 creeping landslides were identified in the study area. Based on the displacement field, the landslide slip-surface slope was inverted under the assumption that the landslide displacement was parallel to the slip-surface. Then the two-dimensional slip-surface depths and volumes of selected landslides were inferred using the slip-surface slope. Comparisons with the in-situ data suggest that the results obtained in this study are reliable. The mapped landslides in the study area have depths of approximately 16–160 m along the central axis and volumes ranging from 483,412 to 135,789,944 m3. The derived volume–area relationship and 2D slip-surface depth suggest that deep-seated landslide is a major landslide type in the study area. We conclude that our method can infer the slip-surface geometry of creeping landslides based on InSAR observation and our results have improved the understanding of the landslide mechanisms in the Jinsha River Basin, China.

StereoVO: Learning Stereo Visual Odometry Approach Based on Optical Flow and Depth Information

We present a novel stereo visual odometry (VO) model that utilizes both optical flow and depth information. While some existing monocular VO methods demonstrate superior performance, they require extra frames or information to initialize the model in order to obtain absolute scale, and they do not take into account moving objects. To address these issues, we have combined optical flow and depth information to estimate ego-motion and proposed a framework for stereo VO using deep neural networks. The model simultaneously generates optical flow and depth information outputs from sequential stereo RGB image pairs, which are then fed into the pose estimation network to achieve final motion estimation. Our experiments have demonstrated that our combination of optical flow and depth information improves the accuracy of camera pose estimation. Our method outperforms existing learning-based and monocular geometry-based methods on the KITTI odometry dataset. Furthermore, we have achieved real-time performance, making our method both effective and efficient.