Scene Model Research Articles

Time-optimal trajectory planning for autonomous parking of tractor-semitrailer vehicle based on piecewise Gauss pseudospectral method

For the autonomous parking problem of the tractor-semitrailer vehicle, a piecewise Gauss pseudospectral method is proposed as a trajectory planning scheme in this paper. Firstly, the trajectory planning problem is transformed into an optimal control problem by establishing a vehicle kinematics model for the low-velocity scene, the dynamic constraints including the physical constraints, the collision avoidance constraints, and the endpoint constraints. The optimal control problem are discretized into a nonlinear programming problem by piecewise Gauss pseudospectral method according to the movement area during parking process. The shortest time to complete parking is taken as the objective function. The Gauss pseudospectral method is further adopted to solve the nonlinear programming problem in the two parking stages. Three parallel parking conditions with different lengths of parking spaces and three vertical parking conditions with different widths of parking spaces are set for simulation verification based on Matlab/Simulink. The simulation results show that the proposed method can uniformly and effectively solve the trajectory planning problem of the tractor-semitrailer vehicle. Compared with the traditional pseudospectral method, a shorter parking time and a significantly higher convergence rate can be got with the method, which further indicates that the proposed method has good effectiveness and robustness.

Simulating snow-covered forest bidirectional reflectance by extending hybrid geometric optical–radiative transfer model

Accurate simulations of the bidirectional reflectance distribution function (BRDF) of snow-covered forests are crucial for retrieving snow and canopy properties, which affects the analysis of net radiation, radiative forcing, and climate change. The anisotropic reflectance of snow in conifer forests is greatly influenced by the high reflectance of snow and the complex travel pathways of radiation in 3D canopy structures, which complicates BRDF simulations. Although bidirectional reflection models have been widely established for snow-free forests, few studies have attempted to simulate the BRDF in snow-covered forests. Therefore, we propose a new snow-covered forest bidirectional reflectance (SFBR) model by coupling geometric optical and radiative transfer models, which includes a snow anisotropic reflectance model (ART), needle leaf optical properties model (LIBERTY), canopy radiative transfer model (4SAIL), and geometric optical model (GOST2). The SFBR model mainly focuses on conifer forests with snow backgrounds; it can simulate anisotropic reflectance spectra from 400 nm to 2500 nm and is the only model to fully consider discontinuous canopy distribution, topographic effects, and snow anisotropy. The SFBR model was validated by simulating the reflectance over snow-covered forests, the results of which showed good consistency with simulations of 3D scene model and remote sensing observations. The proposed SFBR model has the potential to quantify radiation interactions between the snow background and forest overstory, evaluate the sensitivity of canopy reflectance to snow and forest properties, and retrieve snow-covered forest parameters from satellite-observed reflectance data.

Perspectives on point cloud-based 3D scene modeling and XR presentation within the cloud-edge-client architecture

With the support of edge computing, the synergy and collaboration among central cloud, edge cloud, and terminal devices form an integrated computing ecosystem known as the cloud-edge-client architecture. This integration unlocks the value of data and computational power, presenting significant opportunities for large-scale 3D scene modeling and XR presentation. In this paper, we explore the perspectives and highlight new challenges in 3D scene modeling and XR presentation based on point cloud within the cloud-edge-client integrated architecture. We also propose a novel cloud-edge-client integrated technology framework and a demonstration of municipal governance application to address these challenges.

A novel outdoor thermal comfort simulation model for heritage environments (OTC-SM-HE): Verify the effectiveness in Gulangyu, China

The study of outdoor thermal comfort (OTC) for heritage environments can offer essential guidance for the articulation of multiple values of cultural heritage and the sustainable development of traditional settlements. However, previous study methods often fail to capture the complexity and diversity of heritage environments completely. This study proposes a novel outdoor thermal comfort simulation model for heritage environments (OTC-SM-HE) based on 3D real scene modeling technology and universal thermal climate index (UTCI). The effectiveness of the model is verified by comparing it with field test data in Gulangyu Island. The results show that: 1) UTCI are calculated by OTC-SM-HE and field testing showed high proximity started at 14:00 or 15:00 in the same space on different dates, respectively. 2) The mean error is 2.02 °C of UTCI in the same period and space, and the mean d-index of validation of models is 0.735.3) The similarity of trend duration is 85% in field test time. Thus, OTC-SM-HE can be shown to be effective in terms of response spatial characteristics, numerical values and variation trends. The establishment of OTC-SM-HE aims to simulate and evaluate the effects of meteorological conditions, shading facilities, and greening measures on the OTC of visitors and residents in different heritage environments. It provides a scientific basis and optimization recommendations for the sustainable management of heritage environments.

Object detection in traffic videos: an optimized approach using super-resolution and maximal clique algorithm

Detection of small objects is one of the main challenges to be improved in deep learning, mainly due to the small number of pixels and scene’s context, leading to a loss in performance. In this paper, we present an optimized approach based on deep object detection models that allow the detection of a higher number of elements and improve the score obtained for their class inference. The main advantage of the presented methodology is that it is not necessary to modify the internal structure of the selected convolutional neural network model or re-training for a specific scene. Our proposal is based on detecting initial regions to generate several sub-images using super-resolution (SR) techniques, increasing the number of pixels of the elements, and re-infer over these areas using the same pre-trained model. A reduced set of windows is calculated in the super-resolved image by analyzing a computed graph that describes the distances among the preliminary object detections. This analysis is done by finding maximal cliques on it. This way, the number of windows to be examined is diminished, significantly speeding up the detection process. This framework has been successfully tested on real traffic sequences obtained from the U.S. Department of Transportation. An increase of up to 44.6% is achieved, going from an average detection rate for the EfficientDet D4 model of 14.5% compared to 59.1% using the methodology presented for the first sequence. Qualitative experiments have also been performed over the Cityscapes and VisDrone datasets.

Robot trajectory planning for autonomous 3D reconstruction of cockpit in aircraft final assembly testing

The trend towards automation and intelligence in aircraft final assembly testing has led to a new demand for autonomous perception of unknown cockpit operation scenes in robotic collaborative airborne system testing. To address this demand, a robotic automated 3D reconstruction cell which enables to autonomously plan the robot end-camera's trajectory is developed for image acquisition and 3D modeling of the cockpit operation scene. A continuous viewpoint path planning algorithm is proposed that incorporates both 3D reconstruction quality and robot path quality into optimization process. Smoothness metrics for viewpoint position paths and orientation paths are introduced together for the first time in 3D reconstruction. To ensure safe and effective movement, two spatial constraints, Domain of View Admissible Position (DVAP) and Domain of View Admissible Orientation (DVAO), are implemented to account for robot reachability and collision avoidance. By using diffeomorphism mapping, the orientation path is transformed into 3D, consistent with the position path. Both orientation and position paths can be optimized in a unified framework to maximize the gain of reconstruction quality and path smoothness within DVAP and DVAO. The reconstruction cell is capable of automatic data acquisition and fine scene modeling, using the generated robot C-space trajectory. Simulation and physical scene experiments have confirmed the effectiveness of the proposed method to achieve high-precision 3D reconstruction while optimizing robot motion quality.

3D Scene Modeling Method and Feasibility Analysis of River Water-Land Integration

Aiming at the problem of rapid construction of a river three-dimensional 3D scene, this article integrates remote sensing, 3D modeling, and CityEngine technology to construct a 3D scene model reconstruction method of river water-land integration. The method includes intelligent extraction of underwater topography, refined modeling of hydraulic structures, and construction of a water-land integrated real scene model. Based on this method, the high-fidelity land-underwater seamless digital terrain and the water-land 3D real scene models can be formed. Through experiments, the feasibility and limitations of this method are verified. It can effectively extract the shallow underwater terrain of inland rivers, and the overall accuracy of the study area is less than 2 m. The performance of the seamless fusion 3D terrain is better than the public digital elevation model data set. In the inland basin of Class I to II water quality, it can meet the needs of intelligent perception of a river- and lake-integrated 3D scene model.

Monocular Depth Estimation for 3D Map Construction at Underground Parking Structures

Converting the actual scenes into three-dimensional models has inevitably become one of the fundamental requirements in autonomous driving. At present, the main obstacle to large-scale deployment is the high-cost lidar for environment sensing. Monocular depth estimation aims to predict the scene depth and construct a 3D map via merely a monocular camera. In this paper, we add geometric consistency constraints to address the non-Lambertian surface problems in depth estimation. We also utilize the imaging principles and conversion rules to produce a 3D scene model from multiple images. We built a prototype and conduct extensive experiments in a corridor and an underground parking structure, and the results show the effectiveness for indoor location-based services.

Fuzzy-based indoor scene modeling with differentiated examples

Well-designed indoor scenes incorporate interior design knowledge, which has been an essential prior for most indoor scene modeling methods. However, the layout qualities of indoor scene datasets are often uneven, and most existing data-driven methods do not differentiate indoor scene examples in terms of quality. In this work, we aim to explore an approach that leverages datasets with differentiated indoor scene examples for indoor scene modeling. Our solution conducts subjective evaluations on lightweight datasets having various room configurations and furniture layouts, via pairwise comparisons based on fuzzy set theory. We also develop a system to use such examples to guide indoor scene modeling using user-specified objects. Specifically, we focus on object groups associated with certain human activities, and define room features to encode the relations between the position and direction of an object group and the room configuration. To perform indoor scene modeling, given an empty room, our system first assesses it in terms of the user-specified object groups, and then places associated objects in the room guided by the assessment results. A series of experimental results and comparisons to state-of-the-art indoor scene synthesis methods are presented to validate the usefulness and effectiveness of our approach.

Multi-factor Stock Forecasting Model Based on Independence Criterion Weighted Bagging Algorithm

Stock price prediction and the selection of important factors are a hot issue for statisticians. Machine learning methods need to ensure the prediction accuracy of the model in high-dimensional data scenes on the one hand, and the interpretability of the output of the prediction model on the other. If a single model is used for forecasting, it is easy to cause the problems of deviation and variance imbalance of the forecasting model (over-fitting and under-fitting). Based on this, we propose an interpretable forecasting model weighted bagging based on independence criterion. On the one hand, the proposed method solves the problem that a single model is easy to fall into over-fitting and under-fitting by using the idea of model averaging. On the other hand, when the sub-model is an interpretable machine learning method in Bagging framework, the proposed method can also output the weighted feature importance of each factor. In addition, the choice of independence criterion and sub-model is free. The actual data analysis shows that the proposed method has smaller prediction error compared with the comparison methods such as LASSO, ridge regression, random forest and XGBoost. The factor selection model under the minimum prediction error criterion will be more interpretable.

Underwater 3D Reconstruction from Video or Still Imagery: Matisse and 3DMetrics Processing and Exploitation Software

This paper addresses the lack of “push-button” software for optical marine imaging, which currently limits the use of photogrammetric approaches by a wider community. It presents and reviews an open source software, Matisse, for creating textured 3D models of complex underwater scenes from video or still images. This software, developed for non-experts, enables routine and efficient processing of underwater images into 3D models that facilitate the exploitation and analysis of underwater imagery. When vehicle navigation data are available, Matisse allows for seamless integration of such data to produce 3D reconstructions that are georeferenced and properly scaled. The software includes pre-processing tools to extract images from videos and to make corrections for color and uneven lighting. Four datasets of different 3D scenes are provided for demonstration. They include both input images and navigation and associated 3D models generated with Matisse. The datasets, captured under different survey geometries, lead to 3D models of different sizes and demonstrate the capabilities of the software. The software suite also includes a 3D scene analysis tool, 3DMetrics, which can be used to visualize 3D scenes, incorporate elevation terrain models (e.g., from high-resolution bathymetry data) and manage, extract, and export quantitative measurements for the 3D data analysis. Both software packages are publicly available.

Optimizing Fiducial Marker Placement for Improved Visual Localization

Adding fiducial markers to a scene is a well-known strategy for making visual localization algorithms more robust. Traditionally, these marker locations are selected by humans who are familiar with visual localization techniques. This paper explores the problem of automatic marker placement within a scene. Specifically, given a predetermined set of markers and a scene model, we compute optimized marker positions within the scene that can improve accuracy in visual localization. Our main contribution is a novel framework for modeling camera localizability that incorporates both natural scene features and artificial fiducial markers added to the scene. We present optimized marker placement (OMP), a greedy algorithm that is based on the camera localizability framework. We have also designed a simulation framework for testing marker placement algorithms on 3D models and images generated from synthetic scenes. We have evaluated OMP within this testbed and demonstrate an improvement in the localization rate by up to 20 percent on four different scenes.

Line Clipping in 3D: Overview, Techniques and Algorithms

Clipping algorithms essentially compute the intersection of the clipping object and the subject, so to go from two to three dimensions we replace the two-dimensional clipping object by the three-dimensional one (the view frustum). In three-dimensional graphics, the terminology of clipping can be used to describe many related features. Typically, “clipping” refers to operations in the plane that work with rectangular shapes, and “culling” refers to more general methods to selectively process scene model elements. The aim of this article is to survey important techniques and algorithms for line clipping in 3D, but it also includes some of the latest research performed by the authors.

A new method for constructing roads map in forest area using UAV images

High definition map is a map used for automatic driving. Its absolute precision is sub meter level, that is, the error between the position of an element on the map and the position of the element in the real world is less than 100 cm. In addition, compared with ordinary maps, high definition maps contain more geographic information, such as road width, slope, curvature, etc. At present, high definition roads map construction is largely for the main traffic road. However, there are few studies on the construction of high definition forest roads map for forestry production and management. Thus, it is imperative and helpful to explore an efficient and low-cost method for constructing high definition forest roads map that can provide accurate basic map services for forestry management activities. In this study, firstly, the DJI Phantom 4 RTK UAV was used for aerial photography to obtain the forest oblique images. Then, combined with close range photogrammetry, the high-resolution images under the forest are acquired and generated into dense point cloud images. Next, the two images are fused using iterative closest point (ICP) algorithm combined with manual registration; Finally, the fine-grained 3D scene model is generated by using the Context Capture 3D modeling software, based on which a high definition forest roads map was constructed using human-computer interaction model. Taking Menggongshan Forest Farm as an example, the construction of the fine 3D scene model was carried out to solve the problem of loopholes and ambiguities of the road model under forest, based on which the high definition forest roads map was constructed. To evaluate the accuracy of the forest roads map, the accuracy check was conducted. The results showed that the point position on the forest roads map, compared to the actual point, had a medium, minimum and maximum error of 0.076 m, 0.020 m and 0.170 m in plane, respectively, a medium, minimum and maximum error of 0.122 m, 0.005 m and -0.214 m in elevation, respectively. The accuracy of the high definition forest roads map meets the need of forestry production and management activities. It is feasible to adopt the method of UAV oblique photogrammetry combined with close-range photogrammetry, fine 3D scene model construction and human-computer interactive for constructing high definition forest road map. And this method can provide a low-cost and high-efficiency way for constructing high definition forest roads map, and also can better provide basic map data for forest management activities.

Sparsity-based efficient simulation of cluster targets electromagnetic scattering

An efficient and real-time simulation method is proposed for the dynamic electromagnetic characteristics of cluster targets to meet the requirements of engineering practical applications. First, the coordinate transformation method is used to establish a geometric model of the observation scene, which is described by the azimuth angles and elevation angles of the radar in the target reference frame and the attitude angles of the target in the radar reference frame. Then, an approach for dynamic electromagnetic scattering simulation is proposed. Finally, a fast-computing method based on sparsity in the time domain, space domain, and frequency domain is proposed. The method analyzes the sparsity-based dynamic scattering characteristic of the typical cluster targets. The error between the sparsity-based method and the benchmark is small, proving the effectiveness of the proposed method.

UformPose: A U-Shaped Hierarchical Multi-Scale Keypoint-Aware Framework for Human Pose Estimation

Human pose estimation is a fundamental yet challenging task in computer vision. However, difficult scenarios such as invisible keypoints, occlusions and small-scale persons are still not well-handed. In this paper, we present a novel pose estimation framework named UformPose which targets to relieve these issues. UformPose has two core designs: Shared Feature Pyramid Stem (SFPS) and U-shaped hierarchical Multi-scale Keypoint-aware Attention Module (U-MKAM). SFPS is a feature pyramid stem with shared mechanism to learn stronger low-level features at the initial stage, and the shared mechanism can facilitate cross-resolution commonality learning. Our U-MKAM attempts to generate high-quality high-resolution representations by integrating all levels of feature representation of the backbone layer by layer. More importantly, we utilize the flexibility of attention operations for keypoint-aware modeling, which explicitly captures and trades-offs the dependencies between keypoints. We empirically demonstrate the effectiveness of our framework through the competitive pose estimation results on the COCO dataset. Extensive experiments and visual analysis on CrowdPose demonstrate the robustness of our model in crowd scenes.

Analysis and Detection of Transmission Lines Based on Twin Reality

Now life has been transformed and upgraded to the direction of intelligence, digitalization and informatization along with the advancement of scientific and technological information. The traditional detection of transmission lines is completed by the professional maintenance personnel of the power grid through manual detection of lines. This method is inefficient and has a certain threat to the life safety of detection personnel. The development of deep learning and computational vision supplies a fresh idea for detection of transmission lines. Therefore, this paper analyzes the traditional Faster R-CNN based on twin reality scene modeling technology. Then aiming at the shortcomings of traditional algorithms, the Cascade R-CNN algorithm is proposed to complete the detection of transmission lines based on twin reality system. It compares the accuracy and other indicators of Cascade R-CNN and Faster R-CNN algorithms. The effectiveness of two algorithms for analysis and detection of transmission lines is verified through experiments. Finally, the results indicates that the system using Cascade R-CNN has higher prediction accuracy, and has better practicability for analysis and detection of transmission lines.

Scene graph descriptors for visual place classification from noisy scene data

In visual robot place recognition (VPR), a scene graph is a rich scene model that can describe the complex contexts in a scene such as the relationships between various types of visual contents including appearance, space, and semantics. However, training an efficient scene graph classifier is not straightforward. Existing approaches typically rely on exhaustive matching between query and database graphs and are not scalable to large-size VPR problems. Our research is motivated by a recent development of the graph convolutional neural network (GCN) as an efficient and discriminative classifier for graph data, and it aims to explore the potential of the GCN as a scene graph classifier. However, unlike several existing GCN applications, no valid scene graph descriptor for a GCN classifier on noisy scene data exists. To address this issue, herein, we propose to train the GCN model in a teacher-to-student knowledge transfer scheme by employing an existing state-of-the-art single-view VPR system as the teacher model. The proposed approach is implemented within a practical VPR framework by combining the best of the following three independent fields: multimodal information retrieval, rank matching, and similarity-based pattern recognition. Experiments using the public NCLT dataset validate the effectiveness of the proposed approach.

The Influence of UAV Altitudes and Flight Techniques in 3D Reconstruction Mapping

Occasionally, investigating an accident is time-consuming, further compounding traffic congestion. This study aims to reconstruct a 3D model of an accident scene using an unmanned aerial vehicle (UAV). This study tested several flight parameters to check the accuracy and differences compared to site measurement data. The flight parameters selected were POIs and waypoint techniques. These designs can produce a good 3D model to achieve our objectives. This study tested all parameters for accuracy based on the root mean square error (RMSE) value by comparing the UAV data and site measurement data. This study tested this objective using five types of processing and different types of flight parameters (including RMSE) to determine the accuracy of the outcomes. The POI technique achieved an optimal result with centimeter-level accuracy. Furthermore, using UAVs can speed up decision-making, especially in data acquisition, and offer reliable accuracy for specific applications. This study is useful for accident investigation teams to expedite their data collection process.

Application of Laser Ranging in Path Planning of Mobile Medical Robot

Compared with traditional industrial robots, mobile robots with autonomous sensing, decision-making and execution functions have broad application prospects. With the continuous development of computer science and sensor technology, the research of mobile medical robot is also developing in the direction of intelligence and autonomy. The mobile medical robot senses the environmental information and its own state through sensors, and realizes the autonomous movement facing the target in the environment with obstacles, thus completing the medical functions specified by the designer. When applied to some practical scenes, the environment faced by mobile medical robots is often full of complexity and unknowns. Accurate 3D scene model is beneficial for robots to obtain accurate pose estimation by data matching and other methods, and can generate collision-free navigation paths based on environmental information. The mobile medical robot can move autonomously safely and effectively only if it can accurately and automatically position itself, the obstacle information in its working environment and its movement state.