Large Scenes Research Articles

Optimizing Ray Tracing of Trimmed NURBS Surfaces on the GPU

AbstractThe representation of geometric models by trimmed NURBS surfaces has become a standard in the CAD industry. In CAD applications, the rendering of surfaces is usually solved by tessellation followed up by z‐buffer rendering. Ray tracing of NURBS surfaces has not been widely used in industry due to its computational complexity that hinders achieving real‐time performance in practice. We propose novel methods achieving faster point location search needed by trimming in the context of ray tracing trimmed NURBS surfaces. The proposed 2D data structure based on kd‐trees allows for faster ray tracing while it requires less memory for its representation and less preprocessing time than previously published methods. Further, we show the current state of the art for ray tracing trimmed NURBS surfaces on a GPU. With careful design and implementation, the number of rays cast on a GPU may reach real‐time performance in the order of tens to hundreds of million rays per second for moderately to large complex scenes containing hundreds of thousands of NURBS surfaces and trimming curves.

Compressive Sensing SAR Imaging Algorithm for LFMCW Systems

Linear frequency-modulated (LFM) continuous-wave (CW) radar is usually the first choice in synthetic aperture radar (SAR) imaging missions due to its relatively low cost and hardware simplicity. However, the use of continuous-wave unnecessarily introduces the problem of intrapulse motion. Furthermore, a large amount of data generated by the CW imaging process may overburden the onboard communication system with its high streaming data rate. On the other hand, a large amount of data may, indeed, not be necessary for high-resolution SAR imaging. In this article, we took full consideration of the intrapulse motion in LFMCW radar systems and modeled the phase preserving extended frequency scaling algorithm (EFSA) reconstruction process with far fewer data samples as compressive sensing problem and used subgradient descent algorithm with optimal step size to realize compressive sensing reconstruction. Our compressive sensing problem is different from the traditional direct inversion-based problem in that our reconstructed results contain both mainlobe and sidelobes. Comparisons were made with mainstream iterative soft thresholding-based reconstruction algorithm to demonstrate its capability to reconstruct large imaging scenes with high resolution. Both simulations and experiments with measured data have verified our proposed algorithm.

Signature Codes in Visible Light Positioning

With the explosive development of the Internet of Things (IoT), massive equipments are involved in the indoor environment with the demand of precise localization to provide intelligent interconnection. The visible light positioning (VLP) system, with the light emitting diode (LED) and the photo-diode (PD) or image sensor as the transceiver, has attracted a lot of attention due to low cost, high accuracy, energy efficiency, electromagnetic interference free, safety, and so on. In this article, we address the architecture and multiplexing methods of the VLP systems, together with their influence in positioning. From the engineering's point of view, we introduce a novel signature code with reduced complexity and computation. Performing the coding and decoding scheme, the received signal strengths (RSSs) from different LEDs can be decomposed. Based on these RSSs, an adaptive positioning mechanism is first considered following the analysis of merits of the common positioning algorithms, which is verified with optimal positioning performance. It can be concluded that the proposed VLP system has great potential to fit well for a large scene with massive IoT devices.

Informed Sampling Exploration Path Planner for 3D Reconstruction of Large Scenes

As vision-based navigation of small aircraft has been demonstrated to reach relative maturity, research into effective path-planning algorithms to complete the loop of autonomous navigation has been booming. Although the literature has seen some impressive works in this area, efficient path-planning that can be used in tasks such as inspection and coverage is still an open problem. In this spirit, this letter presents an online path-planning algorithm for fast exploration and 3D reconstruction of a previously unknown area of interest. Micro Aerial Vehicles (MAVs) are an ideal candidate for this task due to their maneuverability, but their limited computational power and endurance require efficient planning strategies. Popular sampling-based methods randomly sample the MAV's configuration space and evaluate viewpoints according to their expected information gain. Most often, however, valuable resources are spent on information gain calculations of unpromising viewpoints. This letter proposes a novel informed sampling approach that leverages surface frontiers to sample viewpoints only where high information gain is expected, leading to faster exploration. We study the impact of informed sampling in a wide range of photo-realistic scenes, and we show that our approach outperforms state-of-the-art exploration path planners in terms of both speed and reconstruction quality.

A deep learning semantic template matching framework for remote sensing image registration

We propose a deep learning framework by the probability of the predicting semantic spatial position distribution for remote sensing image registration. Traditional matching methods optimize similarity metrics with pixel-by-pixel searching, which is time consuming and sensitive to radiometric differences. Driven by learning-based methods, we take the reference and template images as inputs and map them to the semantic distribution position of the corresponding reference image. We apply the affine invariant to obtain a correspondence between the overlap of the barycenter position of the semantic template and the center pixel point, which transforms the semantic boundary alignment into a point-to-point matching problem. Additionally, two loss functions are proposed, one for optimizing the subpixel matching position and the other for determining the semantic spatial probability distribution of the matching template. In this work, we explore the influence of the template radius size, the filling form of training labels, and the weighted combination of loss function on the matching accuracy. Our experiments show that the trained model is robust to template deformation while still operating orders of magnitude faster. Furthermore, our proposed method implements high matching accuracy in four large scene images with radiometric differences. This ensures the improved speed of remote sensing image analysis and pipeline processing while facilitating novel directions in learning-based registration. Our code is freely available at https://github.com/liliangzhi110/semantictemplatematching.

Service Robots in Wuhan Cabin Hospitals

medical service robot, human-robot interaction, COVID-19, sensor fusion The cabin hospitals used for COVID-19 victims have an infectious environment and thus require highly autonomous systems to provide various services. This article presents a multirobot architecture designed for cabin hospitals with special focus on the sensors used by service robots. The robots' functions include autonomous disinfection, delivery, cleaning, temperature measurement, physical interaction, and conversation assistance. The robots collaborate with each other by using multiple sensors in large and complicated scenes and evolve together. The robots are controlled by cloud controllers considering a human-centered multi-robot collaboration mechanism. This article presents the details of these robots, including the hardware and software architecture of the robots with sensors, the communication modules, and the cloudbased central controllers. It also presents several applications of the robots and discusses open problems. © 2021 M Y U Scientific Publishing Division. All rights reserved.

CEB-Collaboratively Multi-Granularity Interest Scheduling Algorithm for Loading Large WebBIM Scene

<p indent=0mm>With the rise of 5G and VR, the demand on visualizing large-scale BIM scene on mobile web browsers is increasing rapidly. However, due to the weakness of browser rendering and networking delay, it is still rather challenging to access huge Web3D scenes smoothly. Firstly, edge computing is introduced into WebBIM online visualization, and a cloud-edge-browser (CEB) collaboratively multi-granularity WebBIM framework is designed. Secondly, the interest degree is defined for each of BIM components and the multi-granularity WebBIM transmission scheduling is proposed based on interest degree. Then, the optimal viewpoint selection for initial loading is presented in terms of globality of viewing scope, light weightness and interest degree. Finally, the experimental platform is deployed using Alibaba Cloud service, and some large-scale BIM scenes are used for practical transmission testing. Experimental results show that the proposed algorithm can effectively reduce the data amount of loading WebBIM scenes. The initial loading speed is also improved significantly, and there is no obvious delay to occur when users move huge WebBIM scenes through freely.

An Indoor Integrated Localization Algorithm Based on UWB/BLE

In recent years, some excellent technologies have emerged in the field of indoor localization, but they are not perfect. For example, The Ultra-WideBand(UWB) hardware is expensive, The application of UWB localization in narrow scenes will increase the number of anchors and bring a lot of expenses. Bluetooth-LowEnergy(BLE) localization has a short working distance, so it is not suitable for deployment in wide indoor scenes. In view of the above problems, A integrated localization algorithm based on UWB and BLE is proposed, which is used to achieve full coverage of wide and narrow scenes in indoor localization. The algorithm is based on the extended Kalman filter, and the observation update is accomplished by TDOA and RSSI, so as to realize the integrated localization. Experiments on the algorithm show that UWB localization can achieve an accuracy of 0.2m in large indoor scenes, and BLE localization can achieve an accuracy of 1.6m in small indoor scenes. The localization switching between wide and narrow scenes is smooth and stable with good robustness, which proves that the algorithm can accomplish full scene coverage of indoor localization.

Construction of Virtual Simulation Experimental Teaching System for the Whole Process of Hydroelectric Power Generation

By using the virtual simulation technology and taking the generation, utilization and optimization process of hydroelectric power as the frame, a comprehensive and overall experimental teaching system for the whole process of hydroelectric power generation with the whole basin, large scene and multiple levels is established. This system consists of two modules: hydroelectric power science, water resources and water information. By building a digital model of the entire process of hydroelectric power generation, students can systematically master basic professional knowledge. By optimizing the experimental teaching system, students’ innovative thinking ability and professional practice ability are cultivated.

R-CenterNet+: Anchor-Free Detector for Ship Detection in SAR Images.

In recent years, the rapid development of Deep Learning (DL) has provided a new method for ship detection in Synthetic Aperture Radar (SAR) images. However, there are still four challenges in this task. (1) The ship targets in SAR images are very sparse. A large number of unnecessary anchor boxes may be generated on the feature map when using traditional anchor-based detection models, which could greatly increase the amount of computation and make it difficult to achieve real-time rapid detection. (2) The size of the ship targets in SAR images is relatively small. Most of the detection methods have poor performance on small ships in large scenes. (3) The terrestrial background in SAR images is very complicated. Ship targets are susceptible to interference from complex backgrounds, and there are serious false detections and missed detections. (4) The ship targets in SAR images are characterized by a large aspect ratio, arbitrary direction and dense arrangement. Traditional horizontal box detection can cause non-target areas to interfere with the extraction of ship features, and it is difficult to accurately express the length, width and axial information of ship targets. To solve these problems, we propose an effective lightweight anchor-free detector called R-Centernet+ in the paper. Its features are as follows: the Convolutional Block Attention Module (CBAM) is introduced to the backbone network to improve the focusing ability on small ships; the Foreground Enhance Module (FEM) is used to introduce foreground information to reduce the interference of the complex background; the detection head that can output the ship angle map is designed to realize the rotation detection of ship targets. To verify the validity of the proposed model in this paper, experiments are performed on two public SAR image datasets, i.e., SAR Ship Detection Dataset (SSDD) and AIR-SARShip. The results show that the proposed R-Centernet+ detector can detect both inshore and offshore ships with higher accuracy than traditional models with an average precision of 95.11% on SSDD and 84.89% on AIR-SARShip, and the detection speed is quite fast with 33 frames per second.

Sentinel-1&amp;2 Multitemporal Water Surface Detection Accuracies, Evaluated at Regional and Reservoirs Level

Water stock monitoring is a major issue for society on a local and global scale. Sentinel-1&amp;2 satellites provide frequent acquisitions to track water surface dynamics, proxy variables to enable water surface volume monitoring. How do we combine such observations along time for each sensor? What advantages and disadvantages of single-date, monthly or time-windowed estimations? In this context, we analysed the impact of merging information through different types and lengths of time-windows. Satellite observations were processed separately on optical (Sentinel-2) and radar (Sentinel-1) water detectors at 10 m resolution. The analysis has been applied at two scales. First, validating with 26 large scenes (110 × 110 km) in different climatic zones in France, time-windows yielded an improvement on radar detection (F1-score improved from 0.72 to 0.8 for 30 days on average logic) while optical performances remained stable (F1-score 0.89). Second, validating reservoir area estimations with 29 instrumented reservoirs (20–1250 ha), time-windows presented in all cases an improvement on both optical and radar error for any window length (5–30 days). The mean relative absolute error in optical area detection improved from 16.9% on single measurements to 12.9% using 15 days time-windows, and from 22.15% to 15.1% in radar detection). Regarding reservoir filling rates, we identified an increased negative bias for both sensors when the reservoir is nearly full. This work helped to compare accuracies of separate optical and radar capabilities, where optical statistically outperforms radar at both local and large scale to the detriment of less frequent measurements. Furthermore, we propose a geomorphological indicator of reservoirs to predict the quality of radar area monitoring (R2 = 0.58). In conclusion, we suggest the use of time-windows on operational water mapping or reservoir monitoring systems, using 10–20 days time-windows with average logic, providing more frequent and faster information to water managers in periods of crisis (e.g., water shortage) compared to monthly estimations.

Deep social grouping network for large scenes with multiple subjects

In computer vision, more attention has been paid to group analysis, and the group detection in images becomes a key technology of human analysis on groups. The existing social grouping methods only focus on small scenes with fixed number of persons and cannot deal with large scene images in the real world. This paper proposes the first fine-grained social grouping framework for gigapixel large scene images based on deep learning, which consists of a graph-guided global-to-local partition strategy and a deep grouping network that learns an implicit respresentation for social pairs. The framework has achieved accurate grouping on large scene images. Our method is also applicable to small scene images, and has outperformed the existing methods. The relevant code and the training dataset will be released soon.

ITÖ algorithm with local search for large scale multiple balanced traveling salesmen problem

In the fields such as computer science, intelligent transport systems and logistics scheduling, many problems can be reflected as the variants of traveling salesman problem (TSP). In recent years, although there are many achievements and developments on the study of intelligent optimization algorithms for TSP and its variants, the challenge and problem are still existed: while the scale of problem is large, there is the dimension disaster problem; the optimization algorithms for TSP variants are easy to fall into the local optimum. Multiple balanced traveling salesmen problem (MBTSP) is a variant of TSP, it can be used in the fields such as optimizing multiple gas turbine engines. Since MBTSP is NP-hard problem, many intelligent optimization algorithms, such as genetic algorithm and ITÖ algorithm, have been used to solve it. However, while the scale of MBTSP is large, the traditional algorithms are easy to fall into local optimum. Aiming at the problem, this paper extends the scale of MBTSP to large scale scenes, and proposes a novel ITÖ algorithm (NITÖ) based on crossover operator and local search for large scale MBTSP. For NITÖ algorithm, the drift operator and volatility operator of ITÖ algorithm are redesigned, which are carried out by improved crossover operator and local search. The extensive experiments show that NITÖ can demonstrate better solution quality than the compared state-of-the-art algorithms.

Soil moisture and vegetation optical depth retrievals over heterogeneous scenes using LEWIS L-band radiometer

The Soil Moisture and Ocean Salinity (SMOS) satellite has been providing global data for more than 11 years. The algorithm to retrieve surface soil moisture (SM) and vegetation optical depth (VOD) from Brightness Temperatures (TB) has constantly evolved as our understanding of the instrument and of the responses to various ecosystems increased. The evolution of the radiative transfer modelling used in the inversion process is in part based upon ground measurements done on experimental sites. We present the particular case of an L-band radiometer placed on top of a 800- meter-high cliff. This design allows us to study large and various scenes when compared to classical experiments where a radiometer is placed 10–20 m above a surface. With our set up, the footprints cover areas larger than 300 m in diameter, covering thus several land use types and the footprint can be directed towards different dominant land classes (forest, crops, urban etc…). This paper presents the whole set up consisting of in situ measurements (soil moisture and surface temperatures) and L-band radiometry acquired by the LEWIS instrument at high incidence angles. We perform SM/VOD retrievals from brightness temperatures using various configurations which are compared to the in situ observations. We test in particular a 2-VOD retrieval approach that consists in deriving one SM for the entire scene but two VOD, one for agricultural/low vegetation and a VOD for the forest parts, which improves the performance of the SM/VOD retrievals.

A Novel Method for Fast Positioning of Non-Standardized Ground Control Points in Drone Images

Positioning the pixels of ground control points (GCPs) in drone images is an issue of great concern in the field of drone photogrammetry. The current mainstream automatic approaches are based on standardized markers, such as circular coded targets and point coded targets. There is no denying that introducing standardized markers improves the efficiency of positioning GCP pixels. However, the low flexibility leads to some drawbacks, such as the heavy logistical input in placing and maintaining GCP markers. Especially as drone photogrammetry steps into the era of large scenes, the logistical input in maintaining GCP markers becomes much more costly. This paper proposes a novel positioning method applicable for non-standardized GCPs. Firstly, regions of interest (ROIs) are extracted from drone images with stereovision technologies. Secondly, the quality of ROIs is evaluated using image entropy, and then the outliers are filtered by an adjusted boxplot. Thirdly, pixels of interest are searched with a corner detector, and the precise imagery coordinates are obtained by subpixel optimization. Finally, the verification was carried out in an urban scene, and the results show that this method has good applicability to the GCPs on road traffic signs, and the accuracy rate is over 95%.

Editable free-viewpoint video using a layered neural representation

Generating free-viewpoint videos is critical for immersive VR/AR experience, but recent neural advances still lack the editing ability to manipulate the visual perception for large dynamic scenes. To fill this gap, in this paper, we propose the first approach for editable free-viewpoint video generation for large-scale view-dependent dynamic scenes using only 16 cameras. The core of our approach is a new layered neural representation, where each dynamic entity, including the environment itself, is formulated into a spatio-temporal coherent neural layered radiance representation called ST-NeRF. Such a layered representation supports manipulations of the dynamic scene while still supporting a wide free viewing experience. In our ST-NeRF, we represent the dynamic entity/layer as a continuous function, which achieves the disentanglement of location, deformation as well as the appearance of the dynamic entity in a continuous and self-supervised manner. We propose a scene parsing 4D label map tracking to disentangle the spatial information explicitly and a continuous deform module to disentangle the temporal motion implicitly. An object-aware volume rendering scheme is further introduced for the re-assembling of all the neural layers. We adopt a novel layered loss and motion-aware ray sampling strategy to enable efficient training for a large dynamic scene with multiple performers, Our framework further enables a variety of editing functions, i.e., manipulating the scale and location, duplicating or retiming individual neural layers to create numerous visual effects while preserving high realism. Extensive experiments demonstrate the effectiveness of our approach to achieve high-quality, photo-realistic, and editable free-viewpoint video generation for dynamic scenes.

Fast diffraction pathfinding for dynamic sound propagation

In the context of geometric acoustic simulation, one of the more perceptually important yet difficult to simulate acoustic effects is diffraction, a phenomenon that allows sound to propagate around obstructions and corners. A significant bottleneck in real-time simulation of diffraction is the enumeration of high-order diffraction propagation paths in scenes with complex geometry (e.g. highly tessellated surfaces). To this end, we present a dynamic geometric diffraction approach that consists of an extensive mesh preprocessing pipeline and complementary runtime algorithm. The preprocessing module identifies a small subset of edges that are important for diffraction using a novel silhouette edge detection heuristic. It also extends these edges with planar diffraction geometry and precomputes a graph data structure encoding the visibility between the edges. The runtime module uses bidirectional path tracing against the diffraction geometry to probabilistically explore potential paths between sources and listeners, then evaluates the intensities for these paths using the Uniform Theory of Diffraction. It uses the edge visibility graph and the A* pathfinding algorithm to robustly and efficiently find additional high-order diffraction paths. We demonstrate how this technique can simulate 10th-order diffraction up to 568 times faster than the previous state of the art, and can efficiently handle large scenes with both high geometric complexity and high numbers of sources.

Onboard Real-Time Dense Reconstruction in Large Terrain Scene Using Embedded UAV Platform

Using unmanned aerial vehicles (UAVs) for remote sensing has the advantages of high flexibility, convenient operation, low cost, and wide application range. It fills the need for rapid acquisition of high-resolution aerial images in modern photogrammetry applications. Due to the insufficient parallaxes and the computation-intensive process, dense real-time reconstruction for large terrain scenes is a considerable challenge. To address these problems, we proposed a novel SLAM-based MVS (Multi-View-Stereo) approach, which can incrementally generate a dense 3D (three-dimensional) model of the terrain by using the continuous image stream during the flight. The pipeline of the proposed methodology starts with pose estimation based on SLAM algorithm. The tracked frames were then selected by a novel scene-adaptive keyframe selection method to construct a sliding window frame-set. This was followed by depth estimation using a flexible search domain approach, which can improve accuracy without increasing the iterate time or memory consumption. The whole system proposed in this study was implemented on the embedded GPU based on an UAV platform. We proposed a highly parallel and memory-efficient CUDA-based depth computing architecture, enabling the system to achieve good real-time performance. The evaluation experiments were carried out in both simulation and real-world environments. A virtual large terrain scene was built using the Gazebo simulator. The simulated UAV equipped with an RGB-D camera was used to obtain synthetic evaluation datasets, which were divided by flight altitudes (800-, 1000-, 1200 m) and terrain height difference (100-, 200-, 300 m). In addition, the system has been extensively tested on various types of real scenes. Comparison with commercial 3D reconstruction software is carried out to evaluate the precision in real-world data. According to the results on the synthetic datasets, over 93.462% of the estimation with absolute error distance of less then 0.9%. In the real-world dataset captured at 800 m flight height, more than 81.27% of our estimated point cloud are less then 5 m difference with the results of Photoscan. All evaluation experiments show that the proposed approach outperforms the state-of-the-art ones in terms of accuracy and efficiency.

The Forensic Utility of Photogrammetry in Surface Scene Documentation

Outdoor crime scene documentation needs to be accurate and precise to preserve evidence. Photogrammetry is a potential option. Structure from Motion (SfM) processes photographs into 3D models. As commercial software does not disclose this process, this documentation technique could be legally inappropriate. A potential solution to this problem is open-source software. A series of mock outdoor crime scenes were documented using SfM and total station mapping. Ten large surface scatter scenes containing plastic human remains and personal objects were laid out in 10 × 10 m units in a New England forested environment. The small surface scatter scenes consisted of a pig (Sus scrofa) mandible placed in different environments. The resulting models were built using PhotoScan by AgiSoft and MicMac by IGN. Accuracy was measured by the amount of variance in fixed-datum measurements, whereas visual qualitywas determined by comparison.&#x0D; The average total variance in fixed-datum lengths for six of the ten scenes was below 0.635 cm. The maximum differences in measurement between the total station and software measurements were 0.0917 m (PhotoScan) and 0.178 m (MicMac). Comparative histograms had low standard deviations and mean distances between points. Conditions such as light, ground foliage and topography affect model quality. This research shows that SfM has the potential to be a rapid, accurate and low-cost resource, but there are limitations that must be considered.

Structure-aware indoor scene reconstruction via two levels of abstraction

In this paper, we propose a novel approach that reconstructs the indoor scene in a structure-aware manner and produces two meshes with different levels of abstraction. To be precise, we start from the raw triangular mesh of indoor scene and decompose it into two parts: structure and non-structure objects. On the one hand, structure objects are defined as significant permanent parts in the indoor environment such as floors, ceilings and walls. In the proposed algorithm, structure objects are abstracted by planar primitives and assembled into a polygonal structure mesh. This step produces a compact structure-aware watertight model that decreases the complexity of original mesh by three orders of magnitude. On the other hand, non-structure objects are movable objects in the indoor environment such as furniture and interior decoration. Meshes of these objects are repaired and simplified according to their relationship with respect to structure primitives. Finally, the union of all the non-structure meshes and structure mesh comprises the scene mesh. Note that structure mesh and scene mesh preserve various levels of abstraction and can be used for different applications according to user preference. Our experiments on both LIDAR and RGBD data scanned from simple to large scale indoor scenes indicate that the proposed framework generates structure-aware results while being robust and scalable. It is also compared qualitatively and quantitatively against popular mesh approximation, floorplan generation and piecewise-planar surface reconstruction methods to demonstrate its performance.