Planar Scene Research Articles

Desensitized optimal trajectory for hopping rovers on small bodies

Future exploration tasks of small bodies will need to sample or visit multiple points on the target to obtain more scientific returns, requiring rovers to have the ability to hop on a small body surface. This paper proposes an approach to generate a desensitized optimal trajectory for hopping rovers, aiming at reducing the sensitivity of hopping trajectory in the presence of uncertainties. Firstly, considering parameter uncertainties and initial state errors, analytical expressions of optimal initial states are derived on a planar scene, based on ballistic dynamics. Then, similar methods are developed in both uphill and downhill cases of inclined scenes. Subsequently, the desensitization performance of long-distance hopping trajectory is analyzed under single-hop, identical, and non-identical N-hop strategies. To facilitate the application of the proposed analytical solution to the simulated surface environment of small bodies, a prediction-correction procedure is presented. Finally, Monte Carlo simulations are carried out to verify the effectiveness of the proposed methods. The results indicate that the sensitivity of the hopping trajectory to uncertainties can be effectively diminished by employing the desensitized optimal trajectory and multiple hopping strategy.

Quasi-Dense Matching for Oblique Stereo Images through Semantic Segmentation and Local Feature Enhancement

This paper proposes a quasi-dense feature matching algorithm that combines image semantic segmentation and local feature enhancement networks to address the problem of the poor matching of image features because of complex distortions, considerable occlusions, and a lack of texture on large oblique stereo images. First, a small amount of typical complex scene data are used to train the VGG16-UNet, followed by completing the semantic segmentation of multiplanar scenes across large oblique images. Subsequently, the prediction results of the segmentation are subjected to local adaptive optimization to obtain high-precision semantic segmentation results for each planar scene. Afterward, the LoFTR (Local Feature Matching with Transformers) strategy is used for scene matching, enabling enhanced matching for regions with poor local texture in the corresponding planes. The proposed method was tested on low-altitude large baseline stereo images of complex scenes and compared with five classical matching methods. Results reveal that the proposed method exhibits considerable advantages in terms of the number of correct matches, correct rate of matches, matching accuracy, and spatial distribution of corresponding points. Moreover, it is well-suitable for quasi-dense matching tasks of large baseline stereo images in complex scenes with considerable viewpoint variations.

Searching near and far: The attentional template incorporates viewing distance.

According to theories of visual search, observers generate a visual representation of the search target (the "attentional template") that guides spatial attention toward target-like visual input. In real-world vision, however, objects produce vastly different visual input depending on their location: your car produces a retinal image that is 10 times smaller when it is parked 50 compared to 5 m away. Across four experiments, we investigated whether the attentional template incorporates viewing distance when observers search for familiar object categories. On each trial, participants were precued to search for a car or person in the near or far plane of an outdoor scene. In "search trials," the scene reappeared and participants had to indicate whether the search target was present or absent. In intermixed "catch-trials," two silhouettes were briefly presented on either side of fixation (matching the shape and/or predicted size of the search target), one of which was followed by a probe-stimulus. We found that participants were more accurate at reporting the location (Experiments 1 and 2) and orientation (Experiment 3) of probe stimuli when they were presented at the location of size-matching silhouettes. Thus, attentional templates incorporate the predicted size of an object based on the current viewing distance. This was only the case, however, when silhouettes also matched the shape of the search target (Experiment 2). We conclude that attentional templates for finding objects in scenes are shaped by a combination of category-specific attributes (shape) and context-dependent expectations about the likely appearance (size) of these objects at the current viewing location. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

IPR-VINS: Real-time monocular visual-inertial SLAM with implicit plane optimization

SLAM typically involves updating numerous landmarks to reconstruct scenes rather than generating a topological depiction of the actual environment. Planar features, prevalent in structured environments, can contribute to the development of more nuanced yet less cumbersome maps. The incorporation of fusions between feature points and planar facets gives rise to new discrepancies that must be meticulously addressed. To prevent the uncontrolled expansion of planar boundaries, a robust technique for boundary extraction is offered. Scene planes are fused using the boundary function, ensuring stable feature extraction and planar association. To leverage limited computational resources effectively, a streamlined back-end framework is constructed. Finally, factors are fine-tuned at a consistent scale, which not only elevates locational accuracy but also diminishes computational complexity. To reduce systematic errors, plane prior information is assimilated into global optimization. Compared with VINS-Mono, the average Absolute Trajectory Error of the proposed method on EuRoC is reduced by 44%.

Research on 3D Animation Scene Plane Design Based on Deep Learning

Abstract At present, most graphic design methods of 3D animation scenes only get a small part of 3D animation scenes, and all of them are in a 3D coordinate system, with observers as the core, so it is difficult to express the depth information of 3D animation scenes. This project intends to study a plane design method for 3D animation scenes based on deep learning. CNN (Convolutional Neural Network) is used to build a multi-view 3D animation scene generation network, and 3D geometry and structure of objects are reconstructed through multiple or a group of images. On this basis, the feature extraction method of 3D animation scenes is studied, and the collaborative learning model of multiple networks is established to improve the modeling accuracy of 3D animation scenes. The experimental results show that the network model is superior to the method based on multi-view and 0-1 voxel in detecting retrieval performance, and the accuracy rate can reach 91.725%. The multi-view 3D animation scene generation method in this paper has achieved better results than the current advanced methods, which proves that the multi-view feature fusion network proposed in this paper is a more reasonable method to fuse multi-view image features.

Certifiable algorithms for the two-view planar triangulation problem

Planar scenes predominate in man-made environments, e.g.interior or facades of buildings and in ground images from aerial vehicles. Points lying on those surfaces can be reconstructed from their observations in two images. However, generic reconstruction algorithms output 3D points not lying on the plane, thus obtaining inaccurate reconstructions. The problem also turns to be non-convex with many local minima, hence hindering the performance of iterative method. Therefore, being able to obtain and certify the optimal solution to this problem is of special relevant for these applications. In this paper we first propose a fast and certifiable algorithm that both estimates and certifies the optimal solution to the triangulation problem. From this formulation, we also present an optimality certificate that tells us whether a given solution (obtained by any solver) is the global optimum. Last, from this certificate we derive a sufficient (but not necessary) optimality condition that allows us to certify optimality in less than one microsecond. We test the proposed algorithms on extensive experiments on both synthetic and real data. Code is made available at https://github.com/mergarsal.

Vehicle trajectory data extraction from the horizontal curves of mountainous roads

ABSTRACT Trajectory data is essential for understanding the driver-vehicle-road interaction which is crucial for safety and operational assessment. The video image-processing technique is useful to collect such data on the entire traffic stream. Existing techniques assume that the road is planar which limits the application of the video-based traffic data collection to the plain terrains. With the objective to collect trajectory data from mountainous terrain, the present study proposes an approach to convert the pixel coordinates into real-world coordinates. The hypothesis was that a non-planar scene could be converted into a sequence of piece-wise planar scenes with a separate projection matrix for each planar region. The analysis shows that the proposed method could effectively transform the pixel coordinates into the real-world coordinates. Comparison of the estimated and observed speed/path profiles indicate the adequacy of the proposed method in collecting video-based trajectory data from mountainous terrain.

A spatial AR system for wide-area axis-aligned metric augmentation of planar scenes

Augmented reality (AR) promises to enable use cases in industrial settings that include the embedding of assembly instructions directly into the scene, potentially reducing or altogether obviating the need for workers to refer to such instructions in paper form or on a statically situated screen. Spatial AR, in turn, is a form of AR whereby the augmentation of the scene is carried out using a projector, with the advantage of rendering the augmentation visible to all onlookers simultaneously without calling for any to hold a handheld device such as a tablet or for each to wear some form of head-mounted display. In carrying out spatial AR, however, care must be taken to appropriately warp the image to be projected such that it, when projected, appear free of projective distortions to the viewer. For planar scene geometry (such as a floor, wall, or table), a manual process called keystone correction can be used to carry out an appropriate corrective image warp, a process that can be cumbersome. Another drawback of conventional spatial AR relying only on a single projector is that it is capable of augmenting only the portion of the scene within the projector’s static field of view, thereby hindering its applicability to use cases calling for augmentation of wide areas such as a factory floorspace. We propose a spatial AR system for wide-area metric augmentation of planar scene surfaces that produces the effect of keystone correction analytically as a function of the relative geometry of the projector and scene plane, using a projector equipped with a steerable mirror to direct the projection across varying target locations and a camera facing the scene plane in support of calibrating the system. Our system renders the placement of augmentations in the scene more intuitive than manual keystone correction in two ways. First, (i) the horizontal and vertical axes of the desired augmentations are set in accordance with the horizontal and vertical image axes of the camera, thereby making setting those axes as simple as appropriately rotating the camera relative to the scene plane, and thereby providing a consistent axial reference frame across all target locations. Second, (ii) the desired dimensions of the projected augmentations are specified in metric terms, thereby providing for consistent scaling, likewise across all target locations.

Depth and Perspective Perception of Flat Images in Static and Dynamic Visual Scenes

The paper shows that a sense of depth can arise from two-dimensional (2D) scenes without the presence of a stereoscopic depth signal. Experimental information was obtained on three-dimensional (3D) visual perception of 2D static and dynamic scenes. The technique is based on fixing the conditions of eye movement during the perception of two-dimensional stimulus scenes. To obtain registration of the depth perception effects, they used volume and spatial perspective of 2D images (3D phenomenon), and a binocular eye tracker. The 3D phenomenon is identified using 3D raster images. It is assumed that the comparison of eye movements during a 3D raster image viewing allows you to identify uniquely the effects of the 3D phenomenon of stimulus planar scenes displayed on the monitor screen. The first part of the work shows the conditions for the emergence of a 3D phenomenon on two plots of dynamic and static scenes. The second part demonstrates the three-dimensional attributes of dynamic scenes with the highlighting of various video components. We emphasize that dynamic and static scenes are obtained directly from TV programs. The proposed graphical and mathematical method of analysis made it possible to show qualitatively the perception of the 3D phenomenon by KFU students and revealed the features of volume observation for planar images without the occurrence of binocular disparity.

Homography estimation from a single-point correspondence using template matching and particle swarm optimization.

Existing feature-based methods for homography estimation require several point correspondences in two images of a planar scene captured from different perspectives. These methods are sensitive to outliers, and their effectiveness depends strongly on the number and accuracy of the specified points. This work presents an iterative method for homography estimation that requires only a single-point correspondence. The homography parameters are estimated by solving a search problem using particle swarm optimization, by maximizing a match score between a projective transformed fragment of the input image using the estimated homography and a matched filter constructed from the reference image, while minimizing the reprojection error. The proposed method can estimate accurately a homography from a single-point correspondence, in contrast to existing methods, which require at least four points. The effectiveness of the proposed method is tested and discussed in terms of objective measures by processing several synthetic and experimental projective transformed images.

Dense monocular Simultaneous Localization and Mapping by direct surfel optimization

This work presents a novel approach for monocular dense Simultaneous Localization and Mapping. The surface to be estimated is represented as a piecewise planar surface, defined as a group of surfels each having as parameters its position and normal. These parameters are then directly estimated from the raw camera pixels measurements, by a Gauss-Newton iterative process. The representation of the surface as a group of surfels has several advantages. It allows the recovery of robust and accurate pixel depths, without the need to use a computationally demanding depth regularization schema. This has the further advantage of avoiding the use of a physically unlikely surface smoothness prior. New surfels can be correctly initialized from the information present in nearby surfels, avoiding also the need to use an expensive initialization routine commonly needed in Gauss-Newton methods. The method was written in the GLSL shading language, allowing the usage of GPU thus achieving real-time. The method was tested against several datasets, showing both its depth and normal estimation correctness, and its scene reconstruction quality. The results presented here showcase the usefulness of the more physically grounded piecewise planar scene depth prior, instead of the more commonly pixel depth independence and smoothness prior.

Studies on Three-Dimensional (3D) Accuracy Optimization and Repeatability of UAV in Complex Pit-Rim Landforms As Assisted by Oblique Imaging and RTK Positioning.

Unmanned Aerial Vehicles (UAVs) are a novel technology for landform investigations, monitoring, as well as evolution analyses of long−term repeated observation. However, impacted by the sophisticated topographic environment, fluctuating terrain and incomplete field observations, significant differences have been found between 3D measurement accuracy and the Digital Surface Model (DSM). In this study, the DJI Phantom 4 RTK UAV was adopted to capture images of complex pit-rim landforms with significant elevation undulations. A repeated observation data acquisition scheme was proposed for a small amount of oblique-view imaging, while an ortho-view observation was conducted. Subsequently, the 3D scenes and DSMs were formed by employing Structure from Motion (SfM) and Multi-View Stereo (MVS) algorithms. Moreover, a comparison and 3D measurement accuracy analysis were conducted based on the internal and external precision by exploiting checkpoint and DSM of Difference (DoD) error analysis methods. As indicated by the results, the 3D scene plane for two imaging types could reach an accuracy of centimeters, whereas the elevation accuracy of the orthophoto dataset alone could only reach the decimeters (0.3049 m). However, only 6.30% of the total image number of oblique images was required to improve the elevation accuracy by one order of magnitude (0.0942 m). (2) An insignificant variation in internal accuracy was reported in oblique imaging-assisted datasets. In particular, SfM-MVS technology exhibited high reproducibility for repeated observations. By changing the number and position of oblique images, the external precision was able to increase effectively, the elevation error distribution was improved to become more concentrated and stable. Accordingly, a repeated observation method only including a few oblique images has been proposed and demonstrated in this study, which could optimize the elevation and improve the accuracy. The research results could provide practical and effective technology reference strategies for geomorphological surveys and repeated observation analyses in sophisticated mountain environments.

ESPEE: Event-Based Sensor Pose Estimation Using an Extended Kalman Filter.

Event-based vision sensors show great promise for use in embedded applications requiring low-latency passive sensing at a low computational cost. In this paper, we present an event-based algorithm that relies on an Extended Kalman Filter for 6-Degree of Freedom sensor pose estimation. The algorithm updates the sensor pose event-by-event with low latency (worst case of less than 2 μs on an FPGA). Using a single handheld sensor, we test the algorithm on multiple recordings, ranging from a high contrast printed planar scene to a more natural scene consisting of objects viewed from above. The pose is accurately estimated under rapid motions, up to 2.7 m/s. Thereafter, an extension to multiple sensors is described and tested, highlighting the improved performance of such a setup, as well as the integration with an off-the-shelf mapping algorithm to allow point cloud updates with a 3D scene and enhance the potential applications of this visual odometry solution.

Deep Learning-Based Incorporation of Planar Constraints for Robust Stereo Depth Estimation in Autonomous Vehicle Applications

In autonomous vehicles, depth information for the environment surrounding the vehicle is commonly extracted using time-of-flight (ToF) sensors such as LiDARs and RADARs. Those sensors have some limitations that may potentially degrade the quality and utility of the depth information to a substantial extent. An alternative solution is depth estimation from stereo pairs. However, stereo matching and depth estimation often fails at ill-posed regions including areas with repetitive patterns or textureless surfaces which are commonly found on planar surfaces. This paper focuses on designing an efficient framework for stereo depth estimation, using deep learning technique, that is robust against the mentioned ill-posed regions. With the observation that disparities of all pixels belonging to planar areas (scene plane) viewed by two rectified stereo images can be described using affine transformations, our proposed method predicts pixel-wise affine transformation parameters based on the depth information encoded in the aggregated cost volume. We also introduce a propagation term which enforces all pixels belonging to the same scene plane to be transformed using the same parameters. Disparity can then be computed by multiplying the predicted affine parameters with the corresponding pixel locations. The proposed method was evaluated on several benchmark datasets. We are able to obtain competitive results and at the same time reducing the processing time of common convolution neural network (CNN) in stereo matching by 50%. Analysis of the findings shows that our method can produce reliable results at the ill-posed regions which are challenging to the current state-of-the-arts methods.

Pose Estimation for Ground Robots: On Manifold Representation, Integration, Reparameterization, and Optimization

In this article, we focus on pose estimation dedicated to nonholonomic ground robots with low-cost sensors, by probabilistically fusing measurements from wheel odometers and an exteroceptive sensor. For ground robots, wheel odometers are widely used in pose estimation tasks, especially in applications in planar scenes. However, since wheel odometer only provides two-dimensional (2D) motion measurements, it is extremely challenging to use that for accurate full 6-D pose (3-D position and 3-D orientation) estimation. Traditional methods for 6-D pose estimation with wheel odometers either approximate motion profiles at the cost of accuracy reduction, or rely on other sensors, e.g., inertial measurement unit, to provide complementary measurements. By contrast, we propose a novel motion-manifold-based method for pose estimation of ground robots, which enables to utilize wheel odometers for high-precision 6-D pose estimation. Specifically, the proposed method, first, formulates the motion manifold of ground robots by a parametric representation, second, performs manifold-based 6-D integration with wheel odometer measurements only, and third, reparameterizes manifold representation periodically for error reduction. To demonstrate the effectiveness and applicability of the proposed algorithmic module, we integrate that into a sliding-window pose estimator by using measurements from wheel odometers and a monocular camera. Extensive simulated and real-world experiments are conducted for evaluation, and the proposed algorithm is shown to outperform competing the state-of-the-art algorithms by a significant margin in pose estimation accuracy, especially when deployed in complex, large-scale real-world environments.

An Efficient Iterated EKF-Based Direct Visual-Inertial Odometry for MAVs Using a Single Plane Primitive

This letter proposes an efficient visual-inertial estimator for aerial robots. The main contribution lies in the direct use of intensity measurements of the latest image frames and key frame via both continuous and regular homographic relations under an assumption of a single planar scene. The filter-based method provides comprehensive estimates of position and attitude with notable efficiency and robustness. The flight evaluation indicates that the incorporation of keyframes significantly minimizes the estimation drift while retaining accurate estimates of flight velocity. Compared to state-of-the-art methods, the proposed work provides rivalling performance at substantially lower computational cost by taking the advantage of the assumed single planar structure.

Minimal Solvers for Rectifying From Radially-Distorted Conjugate Translations.

This paper introduces minimal solvers that jointly solve for radial lens undistortion and affine-rectification using local features extracted from the image of coplanar translated and reflected scene texture, which is common in man-made environments. The proposed solvers accommodate different types of local features and sampling strategies, and three of the proposed variants require just one feature correspondence. State-of-the-art techniques from algebraic geometry are used to simplify the formulation of the solvers. The generated solvers are stable, small and fast. Synthetic and real-image experiments show that the proposed solvers have superior robustness to noise compared to the state of the art. The solvers are integrated with an automated system for rectifying imaged scene planes from coplanar repeated texture. Accurate rectifications on challenging imagery taken with narrow to wide field-of-view lenses demonstrate the applicability of the proposed solvers.

Direct Visual-Inertial Ego-Motion Estimation Via Iterated Extended Kalman Filter

This letter proposes a reactive navigation strategy for recovering the altitude, translational velocity and orientation of Micro Aerial Vehicles. The main contribution lies in the direct and tight fusion of Inertial Measurement Unit (IMU) measurements with monocular feedback under an assumption of a single planar scene. An Iterated Extended Kalman Filter (IEKF) scheme is employed. The state prediction makes use of IMU readings while the state update relies directly on photometric feedback as measurements. Unlike feature-based methods, the photometric difference for the innovation term renders an inherent and robust data association process in a single step. The proposed approach is validated using real-world datasets. The results show that the proposed method offers better robustness, accuracy, and efficiency than a feature-based approach. Further investigation suggests that the accuracy of the flight velocity estimates from the proposed approach is comparable to those of two state-of-the-art Visual Inertial Systems (VINS) while the proposed framework is $\approx15-30$ times faster thanks to the omission of reconstruction and mapping.

Automatic Dense Annotation for Monocular 3D Scene Understanding

Deep neural networks have revolutionized many areas of computer vision, but they require notoriously large amounts of labeled training data. For tasks such as semantic segmentation and monocular 3d scene layout estimation, collecting high-quality training data is extremely laborious because dense, pixel-level ground truth is required and must be annotated by hand. In this paper, we present two techniques for significantly reducing the manual annotation effort involved in collecting large training datasets. The tools are designed to allow rapid annotation of entire videos collected by RGBD cameras, thus generating thousands of ground-truth frames to use for training. First, we propose a fully-automatic approach to produce dense pixel-level semantic segmentation maps. The technique uses noisy evidence from pre-trained object detectors and scene layout estimators and incorporates spatial and temporal context in a conditional random field formulation. Second, we propose a semi-automatic technique for dense annotation of 3d geometry, and in particular, the 3d poses of planes in indoor scenes. This technique requires a human to quickly annotate just a handful of keyframes per video, and then uses the camera poses and geometric reasoning to propagate these labels through an entire video sequence. Experimental results indicate that the technique could be used as an alternative or complementary source of training data, allowing large-scale data to be collected with minimal human effort.

Reconstructing piecewise planar scenes with multi-view regularization

Reconstruction of man-made scenes from multi-view images is an important problem in computer vision and computer graphics. Observing that man-made scenes are usually composed of planar surfaces, we encode plane shape prior in reconstructing man-made scenes. Recent approaches for single-view reconstruction employ multi-branch neural networks to simultaneously segment planes and recover 3D plane parameters. However, the scale of available annotated data heavily limits the generalizability and accuracy of these supervised methods. In this paper, we propose multi-view regularization to enhance the capability of piecewise planar reconstruction during the training phase, without demanding extra annotated data. Our multi-view regularization enables the consistency among multiple views by making the feature embedding more robust against view change and lighting variations. Thus, the neural network trained by multi-view regularization performs better on a wide range of views and lightings in the test phase. Based on more consistent prediction results, we merge the recovered models from multiple views to reconstruct scenes. Our approach achieves state-of-the-art reconstruction performance compared to previous approaches on the public ScanNet dataset.