Dense Map Research Articles

Rao-Blackwellized particle smoothing for simultaneous localization and mapping

Abstract Simultaneous localization and mapping (SLAM) is the task of building a map representation of an unknown environment while at the same time using it for positioning. A probabilistic interpretation of the SLAM task allows for incorporating prior knowledge and for operation under uncertainty. Contrary to the common practice of computing point estimates of the system states, we capture the full posterior density through approximate Bayesian inference. This dynamic learning task falls under state estimation, where the state-of-the-art is in sequential Monte Carlo methods that tackle the forward filtering problem. In this paper, we introduce a framework for probabilistic SLAM using particle smoothing that does not only incorporate observed data in current state estimates, but it also backtracks the updated knowledge to correct for past drift and ambiguities in both the map and in the states. Our solution can efficiently handle both dense and sparse map representations by Rao-Blackwellization of conditionally linear and conditionally linearized models. We show through simulations and real-world experiments how the principles apply to radio (Bluetooth low-energy/Wi-Fi), magnetic field, and visual SLAM. The proposed solution is general, efficient, and works well under confounding noise.

RDFC-GAN: RGB-Depth Fusion CycleGAN for Indoor Depth Completion.

Raw depth images captured in indoor scenarios frequently exhibit extensive missing values due to the inherent limitations of the sensors and environments. For example, transparent materials frequently elude detection by depth sensors; surfaces may introduce measurement inaccuracies due to their polished textures, extended distances, and oblique incidence angles from the sensor. The presence of incomplete depth maps imposes significant challenges for subsequent vision applications, prompting the development of numerous depth completion techniques to mitigate this problem. Numerous methods excel at reconstructing dense depth maps from sparse samples, but they often falter when faced with extensive contiguous regions of missing depth values, a prevalent and critical challenge in indoor environments. To overcome these challenges, we design a novel two-branch end-to-end fusion network named RDFC-GAN, which takes a pair of RGB and incomplete depth images as input to predict a dense and completed depth map. The first branch employs an encoder-decoder structure, by adhering to the Manhattan world assumption and utilizing normal maps from RGB-D information as guidance, to regress the local dense depth values from the raw depth map. The other branch applies an RGB-depth fusion CycleGAN, adept at translating RGB imagery into detailed, textured depth maps while ensuring high fidelity through cycle consistency. We fuse the two branches via adaptive fusion modules named W-AdaIN and train the model with the help of pseudo depth maps. Comprehensive evaluations on NYU-Depth V2 and SUN RGB-D datasets show that our method significantly enhances depth completion performance particularly in realistic indoor settings.

UAV-Based High-Throughput Phenotyping to Segment Individual Apple Tree Row Based on Geometrical Features of Poles and Colored Point Cloud

Highlights Integrating RGB values and 3D coordinates of fruit trees provided phenotype-related data. The use of point clouds projection in tree rows segmentation has been confirmed. Using geometric features increased the accuracy of removing poles in an orchard. Segmenting fruit trees after removing poles on a single tree row was more accurate than traditional segmentation method. UAV equipped with LiDAR and a camera is promising for orchard high-throughput phenotyping. Abstract. High-throughput phenotyping (HTP) of fruit trees is important for providing crop geometrical information to evaluate their high yield genotypes. Unmanned aerial vehicle (UAV) is suitable for HTP by obtaining remote sensing data of large modern apple orchards, where each tree row needs to be segmented before segmenting a single tree. This study aims to develop a method for segmenting each row without noise (ERWON) of apple trees based on integrating RGB values and three-dimensional coordinates by UAV. A robust, real-time, RGB-colored, and LiDAR-inertial-visual tightly-coupled state estimation network was used to form a dense map of the orchard, which provided datasets of colored point clouds. Supporting poles were removed from the point clouds based on the consistent number of half upper parts and lower parts. Random sampling and an effective local feature aggregator were trained to segment ERWON after pole segmentation. Results showed that a precision of 0.971, a recall of 0.984, and an intersection-over-union of 0.817 for ERWON segmentation were achieved. This method proposed a potential solution for addressing the challenge of accurately and efficiently segmenting ERWON in large orchards. It is expected to be helpful for obtaining general parameters, such as geometric, morphological, and textural characteristics, as well as more specific parameters relevant to a particular phenotyping task. Keywords: Apple trees, Detection, Point cloud, RGB-colored, Segmentation.

H3-Mapping: Quasi-Heterogeneous Feature Grids for Real-Time Dense Mapping Using Hierarchical Hybrid Representation

H3-Mapping: Quasi-Heterogeneous Feature Grids for Real-Time Dense Mapping Using Hierarchical Hybrid Representation

Highly Dynamic Visual SLAM Dense Map Construction Based on Indoor Environments

Highly Dynamic Visual SLAM Dense Map Construction Based on Indoor Environments

Metric3D v2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimation.

We introduce Metric3D v2, a geometric foundation model for zero-shot metric depth and surface normal estimation from a single image, which is crucial for metric 3D recovery. While depth and normal are geometrically related and highly complimentary, they present distinct challenges. State-of-the-art (SoTA) monocular depth methods achieve zero-shot generalization by learning affine-invariant depths, which cannot recover real-world metrics. Meanwhile, SoTA normal estimation methods have limited zero-shot performance due to the lack of large-scale labeled data. To tackle these issues, we propose solutions for both metric depth estimation and surface normal estimation. For metric depth estimation, we show that the key to a zero-shot single-view model lies in resolving the metric ambiguity from various camera models and large-scale data training. We propose a canonical camera space transformation module, which explicitly addresses the ambiguity problem and can be effortlessly plugged into existing monocular models. For surface normal estimation, we propose a joint depth-normal optimization module to distill diverse data knowledge from metric depth, enabling normal estimators to learn beyond normal labels. Equipped with these modules, our depth-normal models can be stably trained with over 16 million of images from thousands of camera models with different-type annotations, resulting in zero-shot generalization to in-the-wild images with unseen camera settings. Our method currently ranks the 1st on various zero-shot and non-zero-shot benchmarks for metric depth, affine-invariant-depth as well as surface-normal prediction, shown in Fig.1. Notably, we surpassed the ultra-recent MarigoldDepth and DepthAnything on various depth benchmarks including NYUv2 and KITTI. Our method enables the accurate recovery of metric 3D structures on randomly collected internet images, paving the way for plausible single-image metrology. The potential benefits extend to downstream tasks, which can be significantly improved by simply plugging in our model. For example, our model relieves the scale drift issues of monocular-SLAM (Fig.3), leading to high-quality metric scale dense mapping. These applications highlight the versatility of Metric3D v2 models as geometric foundation models. Our project page is at https://JUGGHM.github.io/Metric3Dv2.

CONTINUOUS 3D-LABEL SEMI-GLOBAL MATCHING FOR SATELLITE STEREO

Abstract. We present Continuous 3D-Label Semi-Global Matching (CoSGM), a new dense stereo matching algorithm for satellite stereo. CoSGM overparameterizes the disparity map as an array of local planes, so it can regularize the first-order smoothness of the estimated disparity. Furthermore, the algorithm can produce a dense normal map beside the disparity map. We show experimentally that CoSGM could achieve denser depth maps with comparable accuracy to traditional semi-global matching algorithms.

MATCHING FILTER-BASED VSLAM OPTIMIZATION IN INDOOR ENVIRONMENTS

Abstract. An important factor that reduced the accuracy of motion trajectories in existing VSLAM (Visual Simultaneous Localization and Mapping) systems is the poor estimation of the position pose of the vision odometer. The existing methods generate many incorrect matches during the feature matching process, resulting in low computational accuracy of rotations and translations between cameras, which further leads to a reduction in the robustness of the overall system. In addition, the sparse feature point maps do not provide a detailed description of the surrounding environment, which makes it difficult for the devices equipped with VSLAM systems to perform advanced tasks such as navigation, path planning and human-computer interaction. To address the accuracy problem, we select the set of matches from existing feature matching algorithms based on the motion consistency constraint and use a random sampling consistency algorithm to obtain the best quality matches from the selected samples for computing the geometric transformation model and estimating the current pose. To address the problem of sparse map points, we use the depth information from the RGB-D or Stereo camera to build a dense map module to ensure that information about the surrounding environment is recorded as a point cloud, which provides data support for the implementation of advanced tasks of the device.

Dense orchard landscape mapping based on image merging with skeleton prediction

To address the difficulty of individual vine-plant segmentation in large orchards, this study proposes a segmentation model for individual canopies that combines the characteristics of tree deciduousness and natural growth patterns. This enables the landscape mapping of orchards under intensive planting conditions. In this study, winter and summer unmanned aerial vehicle (UAV) remote-sensing images of kiwifruit orchards were used for canopy segmentation. It involved: 1) target location identification of tree trunks in winter images and alignment of winter and summer image information; 2) semantic segmentation of branches in winter images; 3) screening of individual skeletons and prediction of branch growth; 4) removal of non-canopy parts of summer images using digital image processing to obtain canopy leaf segmentation results; and 5) segmentation of the extent of the individual plant canopies based on the predicted results of the individual plants. The branch precision rate for the final kiwifruit canopy at the plant level was 99.25% and the recall rate was 95.96%. The average recall rate of the canopy precision segmentation was 82.79%, the accuracy rate of which was 73.31%. This model does not involve plant height information and can implement instance segmentation of deciduous trees of the same species, with similar characteristics, and dense planting. The proposed solution addresses the problem of large-scale standardized planting of the same species, especially those that do not have obvious height or color differences and cannot be segmented using a digital elevation model.

Two-stage cross-fusion network for stereo event-based depth estimation

Drawing inspiration from biological retina, event camera uses dynamic vision sensor to asynchronously capture pixel intensity changes. Event camera provides a series of advantages, including high dynamic range, high temporal resolution, lack of motion blur, and low power consumption, which make it suitable candidate for depth estimation, specially in challenging scenarios with fast motion and low-lighting conditions. However, due to the sparse nature of event data, it is troublesome to estimate dense disparity map from them. Although it is plausible to estimate disparity map from edges of a structure from where most events are captured, estimating dense disparity is challenging in the areas where events rarely occur. In this study, we develop a stereo matching network for dense disparity estimation, which effectively fuses reconstructed image features with event to estimate disparity. To this end, we propose a two-stage cross-fusion network (TCFNet) to estimate disparity by fusing event and image features with two cascaded fusion mechanisms. First, we propose cross-SPADE fusion, based on the spatially-adaptive denormalization mechanism (SPADE), which modulates event features with image features, and image features with event features. In the second stage, the modulated features are further fused using the cross-gating mechanism, which uses the multi-axis gated MLP (gMLP). Experiments on both real and synthetic datasets indicate that the proposed two-stage cross-fusion network outperforms the state-of-the-art event-based stereo matching networks empirically and visually.

Population genomics provides insights into the genetic diversity and adaptation of the Pieris rapae in China.

The cabbage white butterfly (Pieris rapae), a major agricultural pest, has become one of the most abundant and destructive butterflies in the world. It is widely distributed in a large variety of climates and terrains of China due to its strong adaptability. To gain insight into the population genetic characteristics of P. rapae in China, we resequenced the genome of 51 individuals from 19 areas throughout China. Using population genomics approaches, a dense variant map of P. rapae was observed, indicating a high level of polymorphism that could result in adaptation to a changing environment. The feature of the genetic structure suggested considerable genetic admixture in different geographical groups. Additionally, our analyses suggest that physical barriers may have played a more important role than geographic distance in driving genetic differentiation. Population history showed the effective population size of P. rapae was greatly affected by global temperature changes, with mild periods (i.e., temperatures warmer than those during glaciation but not excessively hot) leading to an increase in population size. Furthermore, by comparing populations from south and north China, we have identified selected genes related to sensing temperature, growth, neuromodulation and immune response, which may reveal the genetic basis of adaptation to different environments. Our study is the first to illustrate the genetic signatures of P. rapae in China at the population genomic level, providing fundamental knowledge of the genetic diversity and adaptation of P. rapae.

PanoVLM: Low-Cost and accurate panoramic vision and LiDAR fused mapping

Cameras and LiDARs are currently two types of sensors commonly used for 3D mapping. Vision-based methods are susceptible to textureless regions and lighting, and LiDAR-based methods easily degenerate in scenes with insignificant structural features. Most current fusion-based methods require strict synchronization between the camera and LiDAR and need auxiliary sensors, such as IMU. All of these lead to an increase in device cost and complexity. To address that, in this paper, we propose a low-cost mapping pipeline called PanoVLM that only requires a panoramic camera and a LiDAR without strict synchronization. First, camera poses are estimated by a LiDAR-assisted global Structure-from-Motion, and LiDAR poses are derived with the initial camera-LiDAR relative pose. Then, line-to-line and point-to-plane associations are established between LiDAR point clouds, which are used to further refine LiDAR poses and remove motion distortion. With the initial sensor poses, line-to-line correspondences are established between images and LiDARs to refine their poses jointly. The final step, joint panoramic Multi-View Stereo, estimates the depth map for each panoramic image and fuses them into a complete dense 3D map. Experimental results show that PanoVLM can work on various scenarios and outperforms state-of-the-art (SOTA) vision-based and LiDAR-based methods. Compared with the current SOTA LiDAR-based techniques, namely LOAM, LeGO-LOAM, and F-LOAM, PanoVLM manifests a reduction in the absolute rotation error and absolute translation error by 20% and 35%, respectively. Our code and dataset are available at https://github.com/3dv-casia/PanoVLM.

Low voltage and low wave speed are rarely present outside the left atrium-pulmonary veins junction in paroxysmal atrial fibrillation but frequently present in persistent forms

Abstract Background and aim Multielectrode high-density catheters have enabled acquisition of comprehensive and dense maps of electrogram’s amplitude and timing. However, automated activation and voltage mapping are flawed by catheter orientation in relation to the wavefront activation. Omnipolar mapping technology (OT) uses both unipolar and bipolar signals to obtain OT signals and increases the accuracy of automatic point acquisition allowing for a high- density map in a quick and efficient way. The aim of this study was to automatically assess with OT, the left atrium (LA) voltage and wave speed propagation map in sinus rhythm (SR) according to the type of atrial fibrillation (AF). Methods We studied 14 consecutive patients referred for ablation of AF, either paroxysmal (PAF) (n=6) or persistent (Pers-AF) (n=8) using catheters with OT. Patients with Pers-AF were cardioverted, and a SR map was obtained before ablation in all patients and voltage and propagation wave speed maps were obtained. The cut-off for low voltage areas (LVAs) was less than 0.1 mV and for low wave speed (LWS) was less than 0.4 mm/ms. Results were compared according to the type of AF. Results The results are depicted in the Table. The median duration of AF in Pers-AF was 18 (5-24) months. Patients with Pers-AF had larger LA volumes and displayed presence of LVAs and LWS areas in 100% of cases in comparison with 17% and 33% of cases respectively for PAF patients (p=0.003 and p=0.015). Also, the total LVAs and LWS areas in cm2 were significantly higher, (p=0.001 and p=0.005)). The LVAs and the LWS areas were more frequently located outside the PV-LA junction in Pers-AF than in PAF, respectively 75% vs 0% and 88% vs 0%, p=0.010 and p=0.005 (Figure). Conclusions Patients with PAF never presented LVAs or LWS outside the PV-LA junction. On the contrary, patients with Pers-AF had frequently anatomical and electrophysiological abnormalities in the LA body. These findings may suggest the need for a wider ablation strategy in Pers-AF.Normal voltage in purple ; LWS in white

City-scale continual neural semantic mapping with three-layer sampling and panoptic representation

Neural implicit representations are drawing much attention from the robotics community recently, as they are expressive, continuous and compact. However, city-scale continual implicit dense mapping based on sparse LiDAR input is still an under-explored challenge. To this end, we successfully build a city-scale continual neural mapping system with a panoptic representation that consists of environment-level and instance-level modeling. Given a stream of sparse LiDAR point cloud, it maintains a model that maps 3D coordinates to signed distance field (SDF) values. To address the difficulty of representing geometric information at different levels in city-scale space, we propose a tailored three-layer sampling strategy to sample the global, local and near-surface domains dynamically. Meanwhile, to realize high fidelity mapping of instance under incomplete observation, category-specific prior is introduced to better model the geometric details. We evaluate on the public SemanticKITTI [1] dataset and demonstrate the significance of the newly proposed three-layer sampling strategy and panoptic representation, using both quantitative and qualitative results. Codes and model will be publicly available.

Bird's-Eye-View Semantic Segmentation With Two-Stream Compact Depth Transformation and Feature Rectification

Bird's-eye-view (BEV) perception has gained popularity since it provides a 3D world representation with scale consistency. Although existing camera-based solutions achieve excellent performance, the BEV positions related to features are still less accurate. In this paper, a BEV semantic segmentation framework with two-stream compact depth transformation and feature rectification is proposed. To balance the conflict that the feature maps ensemble tends to use two temporal frames with long interval, while shorter temporal frames are more beneficial to depth prediction, a two-stream compact depth transformation is designed. Between original temporal frames, we introduce an intermediate frame to decouple the joint depth estimation of original frames. The local representations of the intermediate frame are respectively matched with each original temporal frame to achieve stereo depth predictions, where compact cost volumes are built to significantly reduce memory usage with high discriminability in depth-dimension. Further, virtual camera intrinsic parameters are derived to realize adaption of compact cost volume to various 2D data augmentation and improve generalization. On this basis, BEV feature maps are obtained via feature transformation. With the influence of depth distribution errors to BEV feature map, a feature rectified segmentation network is proposed to dynamically adjust the position offsets of input features via deformable convolution and semantic information-guided feature learning. As a result, a dense and accurate BEV semantic map is obtained. In addition, a self-supervised depth estimation teacher is adopted to provide extra supervision for depth prediction of our segmentation framework. The effectiveness of the proposed method is verified on public datasets.

Assessing the impact of blockchain technology on the overall performance of sustainable supply chains: an analytical perspective.

Supply chain control and sustainability can be significantly improved using distributed ledger technologies such as blockchain. The blockchain has the potential to facilitate responsible sourcing appropriately, compliance with weather requirements, and sustainable delivery chains. The purpose of this study is to address the hassle of managing conservatism when approaching era adoption and to explore the performance enhancements in blockchain-generated implementations. To achieve this goal, we introduce a scientific approach aimed at studying the outcomes of various factors in the adoption process in the blockchain era and verifying their impact on the overall performance of the delivery chain. Furthermore, a team of multidisciplinary professionals will establish causal relationships among these elements through a consensus-based approach. Ultimately, fuzzy reasoning tools can be used to determine the relative weights between identified factors and delivery chain performance goals. We will assemble causal representations of diagnoses using a dense scientific map model and dynamically generate scenarios for each. The study demonstrates that the integration of blockchain power generation can significantly improve the effectiveness of mineral supply chains. It uses smart contracts to promote environmental sustainability, traceability, and transparency.

A Lightweight Framework For Chromatin Loop Detection at the Single-Cell Level.

Single-cell Hi-C (scHi-C) has made it possible to analyze chromatin organization at the single-cell level. However, scHi-C experiments generate inherently sparse data, which poses a challenge for loop calling methods. The existing approach performs significance tests across the imputed dense contact maps, leading to substantial computational overhead and loss of information at the single-cell level. To overcome this limitation, a lightweight framework called scGSLoop is proposed, which sets a new paradigm for scHi-C loop calling by adapting the training and inferencing strategies of graph-based deep learning to leverage the sequence features and 1D positional information of genomic loci. With this framework, sparsity is no longer a challenge, but rather an advantage that the model leverages to achieve unprecedented computational efficiency. Compared to existing methods, scGSLoop makes more accurate predictions and is able to identify more loops that have the potential to play regulatory roles in genome functioning. Moreover, scGSLoop preserves single-cell information by identifying a distinct group of loops for each individual cell, which not only enables an understanding of the variability of chromatin looping states between cells, but also allows scGSLoop to be extended for the investigation of multi-connected hubs and their underlying mechanisms.

Compressed sensing on displacement signals measured with optical coherence tomography.

Optical coherence tomography (OCT) is capable of angstrom-scale vibrometry of particular interest to researchers of auditory mechanics. We develop a method for compressed sensing vibrometry using OCT that significantly reduces acquisition time for dense motion maps. Our method, based on total generalized variation with uniform subsampling, can reduce the number of samples needed to measure motion maps by a factor of ten with less than 5% normalized mean square error when tested on a diverse set of in vivo measurements from the gerbil cochlea. This opens up the possibility for more complex in vivo experiments for cochlear mechanics.

Toward real-time and accurate dense 3D mapping of crop fields for combine harvesters using a stereo camera.

Fast and accurate 3D scene perception is a crucial prerequisite for the autonomous navigation and harvesting of combine harvesters. However, crop field scenarios pose severe challenges for vision-based perception systems due to repetitive scenes, illumination changes and real-time constraints on embedded computing platforms. In this paper, we propose a feature-based, two-stage approach for real-time dense 3D mapping for combine harvesters. In the first stage, our approach constructs a sparse 3D map using reliable feature matching, which provides prior knowledge about the environment. In the second stage, our method formulates per-pixel disparity calculation as probabilistic inference. The key to our approach is the ability to compute dense 3D maps by combining Bayesian estimation with efficient and discriminative point cues from images, exhibiting tolerance against visual measurement uncertainties due to repetitive textures and uneven lighting in crop fields. We validate the performance of the proposed method using real crop field data, and the results demonstrate that our dense 3D maps provide detailed spatial metric information while maintaining a balance between accuracy and efficiency. This makes our approach highly valuable for online perception in combine harvesters operating with resource-limited systems.

Received signal strength reconstruction using pix2pix generative adversarial network

AbstractPositioning technology based on received signal strength (RSS) fingerprints has emerged as a vital research topic in indoor localization due to its high accuracy and low cost. Increasing the density of offline fingerprints collected can effectively improve localization accuracy in the online stage, but it will result in astronomical collection costs. RSS reconstruction using generative models is an efficient solution to dramatically reduce fingerprint collection costs while maintaining excellent accuracy. In this paper, the authors propose a pix2pix generative adversarial network based RSS reconstruction model, which can generate a dense RSS fingerprint map based on only the location information of the wireless access point. The accuracy and efficiency of the proposed model are verified in an indoor environment.