Large Scenes Research Articles

Efficient Single Correspondence Voting for Point Cloud Registration.

3D point cloud registration is a crucial task in a variety of fields, including remote sensing mapping, computer vision, virtual reality, and autonomous driving. However, this task is still challenging due to the challenges of noise, non-uniformity, partial overlap, and repeated local features in large scene point clouds. In this paper, we propose an efficient single correspondence voting method for large scene point cloud registration. Specifically, we first propose an efficient hypothetical transformation prediction method called SCVC, which determines the 5 degrees of freedom of the transformation through one correspondence, and then uses Hough voting to determine the last degree of freedom. This algorithm can significantly improve the accuracy of registration in both indoor and outdoor scenes. On the other hand, we propose a more robust transformation verification function called VDIR, which can obtain the optimal registration result of two raw point clouds. Finally, we conduct a series of experiments that demonstrate that our method achieves state-of-the-art performance on four real-world datasets: 3DMatch, 3DLoMatch, KITTI, and WHU-TLS. Our code is available at https://github.com/xingxuejun1989/SCVC.

RORNet: Partial-to-Partial Registration Network With Reliable Overlapping Representations.

Three-dimensional point cloud registration is an important field in computer vision. Recently, due to the increasingly complex scenes and incomplete observations, many partial-overlap registration methods based on overlap estimation have been proposed. These methods heavily rely on the extracted overlapping regions with their performances greatly degraded when the overlapping region extraction underperforms. To solve this problem, we propose a partial-to-partial registration network (RORNet) to find reliable overlapping representations from the partially overlapping point clouds and use these representations for registration. The idea is to select a small number of key points called reliable overlapping representations from the estimated overlapping points, reducing the side effect of overlap estimation errors on registration. Although it may filter out some inliers, the inclusion of outliers has a much bigger influence than the omission of inliers on the registration task. The RORNet is composed of overlapping points' estimation module and representations' generation module. Different from the previous methods of direct registration after extraction of overlapping areas, RORNet adds the step of extracting reliable representations before registration, where the proposed similarity matrix downsampling method is used to filter out the points with low similarity and retain reliable representations, and thus reduce the side effects of overlap estimation errors on the registration. Besides, compared with previous similarity-based and score-based overlap estimation methods, we use the dual-branch structure to combine the benefits of both, which is less sensitive to noise. We perform overlap estimation experiments and registration experiments on the ModelNet40 dataset, outdoor large scene dataset KITTI, and natural data Stanford Bunny dataset. The experimental results demonstrate that our method is superior to other partial registration methods. Our code is available at https://github.com/superYuezhang/RORNet.

SASFF: A Video Synthesis Algorithm for Unstructured Array Cameras Based on Symmetric Auto-Encoding and Scale Feature Fusion.

For the synthesis of ultra-large scene and ultra-high resolution videos, in order to obtain high-quality large-scene videos, high-quality video stitching and fusion are achieved through multi-scale unstructured array cameras. This paper proposes a network model image feature point extraction algorithm based on symmetric auto-encoding and scale feature fusion. By using the principle of symmetric auto-encoding, the hierarchical restoration of image feature location information is incorporated into the corresponding scale feature, along with deep separable convolution image feature extraction, which not only improves the performance of feature point detection but also significantly reduces the computational complexity of the network model. Based on the calculated high-precision feature point pairing information, a new image localization method is proposed based on area ratio and homography matrix scaling, which improves the speed and accuracy of the array camera image scale alignment and positioning, realizes high-definition perception of local details in large scenes, and obtains clearer synthesis effects of large scenes and high-quality stitched images. The experimental results show that the feature point extraction algorithm proposed in this paper has been experimentally compared with four typical algorithms using the HPatches dataset. The performance of feature point detection has been improved by an average of 4.9%, the performance of homography estimation has been improved by an average of 2.5%, the amount of computation has been reduced by 18%, the number of network model parameters has been reduced by 47%, and the synthesis of billion-pixel videos has been achieved, demonstrating practicality and robustness.

Instability of energy spectrum disturbance for ship turbulent wakes: SAR imaging simulation and analysis

We present a ship turbulent wake synthetic aperture radar (SAR) simulator (STW-SARSim) producing two-dimensional true-scale turbulent wakes in SAR images that extend for kilometers. In this paper, we innovatively overcome the sea surface time-varying problem in solving the spectral energy balance equation by transforming the unsteady problem into a steady problem and considering the dark trailing centerline region of the turbulent wake in SAR images as a texture modulation effect of the Bragg wave for sea surface scattering. We also present SAR images simulated in three different bands: (i) Sentinel-1A C-band, (ii) TerraSAR X-band, and (iii) ALOS PALSAR L-band to validate the reliability of our proposed method and analyze the effects of wind speed, wind direction, and dissipation rates on the turbulent wake simulation. This paper provides new insights into the simulation of turbulent wakes in SAR images of large scenes inspired by recent research on detection algorithms. It sets the stage for future dataset generation for detection.

A large capacity expansion and robust recognition method of dot-dispersing coded targets with binary step-response serial encoding

In close-range photogrammetry, it is difficult to meet the measurement requirements of large scenes in actual engineering due to the limited capacity of coded targets. To expand the capacity of the coded target, we propose a binary step-response serial-coded target (BSSCT). The BSSCT introduces periodic binary wave information as an additional feature in the dot-dispersing coded target. Also, a robust recognition algorithm for the BSSCT is developed, the P2-Invariant, and the step period is used for decoding. The capacity of the coded target reaches 7 magnitudes without increasing the auxiliary points. Under different lighting conditions and viewing angles, the measurement experiments show that the BSSCT outperforms other state-of-the-art coded targets. Our BSSCT is a promising standard method for large-field system calibration and object measurement.

TARGET-GUIDED LEARNING FOR RARE CLASS SEGMENTATION IN LARGE-SCALE URBAN POINT CLOUDS

Abstract. In large-scale urban areas, the diversity of objects and the complexity of scenes pose challenges to semantic segmentation of point clouds. In particular, the data imbalance problem often results in poor performance for rare classes in large scenes. This paper proposes a rare class segmentation method based on the target-guided transformer network. In the network, all the feature extraction and segmentation procedures are realized by attention mechanisms. The self-attention blocks are embedded in U-Net-like structure to gradually integrate the features from local to global. Then, under the supervision of our target-guided block, the instance features of data-imbalanced rare classes are mapped onto the multi-scale features. At last, a multi-layer perceptron is utilized to convert the fused features to the segmentation logits for generating the semantic labels. Experiments using the Hessigheim High- Resolution 3D Point Cloud Benchmark indicated that our approach considerably outperforms the baseline network by up to 11.66% in terms of mean F1 score. In particular, the rare classes Vehicle and Chimney obtain outstanding F1-scores of 82.40% and 82.51%, respectively. Furthermore, our method achieves an overall accuracy of 87.63%, which increases by 1.09% compared to the baseline model.

INDOOR POSITIONING AND NAVIGATION BASED ON QR CODE MAP

Abstract. Solving the indoor positioning problem can effectively improve work and life efficiency. Outdoors, navigation and positioning mainly rely on satellites to provide global positioning and road network topology to constrain the direction of travel. Therefore, to achieve good indoor navigation and positioning, two things are needed: 1) continuous position acquisition and 2) an indoor road network. Since the indoor environment is relatively stable, the geodetic coordinates of key indoor nodes can be obtained in advance through measurement methods and control points can be set up. Inertial navigation is stable and autonomous, and when combined with control points, it can achieve stable position acquisition. QR codes are rich in information, low in cost, and easy to deploy. They can be used as nodes to construct an indoor road network or as control points. This study proposes an indoor road network map construction method that can achieve local accuracy and global consistency in large indoor scenes. In this indoor road network map, node information includes the position and posture of the QR code. When a machine passes through a node position, it can obtain its absolute position and direction in the geodetic coordinate system by taking a picture of the node’s QR code and combining it with IMU fusion positioning. Through map edge information, indoor navigation and path planning can be achieved.

Locally-Adaptive Level-of-Detail for Hardware-Accelerated Ray Tracing

We introduce an adaptive level-of-detail technique for ray tracing triangle meshes that aims to reduce the memory bandwidth used during ray traversal, which can be the bottleneck for rendering time with large scenes and the primary consumer of energy. We propose a specific data structure for hierarchically representing triangle meshes, allowing localized decisions for the desired mesh resolution per ray. Starting with the lowest-resolution triangle mesh level, higher-resolution levels are generated by tessellating each triangle into four via splitting its edges with arbitrarily-placed vertices. We fit the resulting mesh hierarchy into a specialized acceleration structure to perform on-the-fly tessellation level selection during ray traversal. Our structure reduces both storage cost and data movement during rendering, which are the main consumers of energy. It also allows continuous transitions between detail levels, while locally adjusting the mesh resolution per ray and preserving watertightness. We present how this structure can be used with both primary and secondary rays for reflections and shadows, which can intersect with different tessellation levels, providing consistent results. We also propose specific hardware units to cover the cost of additional compute needed for level-of-detail operations. We evaluate our method using a cycle-accurate simulation of a custom ray tracing hardware architecture. Our results show that, as compared to traditional bounding volume hierarchies, our method can provide more than an order of magnitude reduction in energy use and render time, given sufficient computational resources.

X3DFast model for classifying dairy cow behaviors based on a two-pathway architecture

Behavior is one of the important factors reflecting the health status of dairy cows, and when dairy cows encounter health problems, they exhibit different behavioral characteristics. Therefore, identifying dairy cow behavior not only helps in assessing their physiological health and disease treatment but also improves cow welfare, which is very important for the development of animal husbandry. The method of relying on human eyes to observe the behavior of dairy cows has problems such as high labor costs, high labor intensity, and high fatigue rates. Therefore, it is necessary to explore more effective technical means to identify cow behaviors more quickly and accurately and improve the intelligence level of dairy cow farming. Automatic recognition of dairy cow behavior has become a key technology for diagnosing dairy cow diseases, improving farm economic benefits and reducing animal elimination rates. Recently, deep learning for automated dairy cow behavior identification has become a research focus. However, in complex farming environments, dairy cow behaviors are characterized by multiscale features due to large scenes and long data collection distances. Traditional behavior recognition models cannot accurately recognize similar behavior features of dairy cows, such as those with similar visual characteristics, i.e., standing and walking. The behavior recognition method based on 3D convolution solves the problem of small visual feature differences in behavior recognition. However, due to the large number of model parameters, long inference time, and simple data background, it cannot meet the demand for real-time recognition of dairy cow behaviors in complex breeding environments. To address this, we developed an effective yet lightweight model for fast and accurate dairy cow behavior feature learning from video data. We focused on four common behaviors: standing, walking, lying, and mounting. We recorded videos of dairy cow behaviors at a dairy farm containing over one hundred cows using surveillance cameras. A robust model was built using a complex background dataset. We proposed a two-pathway X3DFast model based on spatiotemporal behavior features. The X3D and fast pathways were laterally connected to integrate spatial and temporal features. The X3D pathway extracted spatial features. The fast pathway with R(2 + 1)D convolution decomposed spatiotemporal features and transferred effective spatial features to the X3D pathway. An action model further enhanced X3D spatial modeling. Experiments showed that X3DFast achieved 98.49% top-1 accuracy, outperforming similar methods in identifying the four behaviors. The method we proposed can effectively identify similar dairy cow behaviors while improving inference speed, providing technical support for subsequent dairy cow behavior recognition and daily behavior statistics.

Modeling the Global Relationship via the Point Cloud Transformer for the Terrain Filtering of Airborne LiDAR Data

Due to the irregularity and complexity of ground and non-ground objects, filtering non-ground data from airborne LiDAR point clouds to create Digital Elevation Models (DEMs) remains a longstanding and unresolved challenge. Recent advancements in deep learning have offered effective solutions for understanding three-dimensional semantic scenes. However, existing studies lack the capability to model global semantic relationships and fail to integrate global and local semantic information effectively, which are crucial for the ground filtering of point cloud data, especially for larger objects. This study focuses on ground filtering challenges in large scenes and introduces an elevation offset-attention (E-OA) module, which considers global semantic features and integrates them into existing network frameworks. The performance of this module has been validated on three classic benchmark models (RandLA-Net, point transformer, and PointMeta-L). It was compared with two traditional filtering methods and the advanced CDFormer model. Additionally, the E-OA module was compared with three state-of-the-art attention frameworks. Experiments were conducted on two distinct data sources. The results show that our proposed E-OA module improves the filtering performance of all three benchmark models across both data sources, with a maximum improvement of 6.15%. The performance of models was enhanced with the E-OA module, consistently exceeding that of traditional methods and all competing attention frameworks. The proposed E-OA module can serve as a plug-and-play component, compatible with existing networks featuring local feature extraction capabilities.

Multi-branch stacking remote sensing image target detection based on YOLOv5

Optical remote sensing is crucial in land management, maritime safety, and rescue operations. Currently, high resolution target detection faces the problems including feature loss, false detection, and limited network robustness. To tackle the aforementioned issues, this study introduces a novel MBS-NET model, which is built upon the YOLOv5 structure. The proposed model presents a multi-branch stacking module structure to precisely capture deep target feature information. By introducing a dual-channel attention mechanism module (EGCA) at the model's neck, the proposed method ensures discriminative feature acquisition in crowded objects and vast spatial scenes. Prior to network training, introduce an improved data augmentation strategy to accommodate the multi-scale and directional variations in remote sensing objectives for model detection. Experimental results on the large-scale public DIOR dataset demonstrate that the MBS-NET model introduced in this paper displays exceptional performance and remarkable interpretability in remote sensing scenarios at a large scale. MBS-NET model outperforms YOLOv5 and YOLOv7 models by increasing the accuracy by 5% and 2% respectively. In addition, the recall rate and F1 Score index of MBS-NET model is superior to those of other methods, resulting in significant improvement of detection accuracy and robustness in large scenes.

The Role of the United Nations in Organizing the Work of the Metaverse

Objectives: The aim of this study is to review the role of the United Nations in organizing the work of Metaverse, as it represents the virtual reality revolution, which is part of a large scene of technological change, where it represents an indication of rapid technical development, and an important component of transformation from digital reality to augmented reality. The research highlighted the necessity of the United Nations' commitment to ensuring freedom of expression on Metaverse sites, in addition to its role in confronting the rhetoric of incitement to terrorism through Metaverse. Methodology: The study relied on the descriptive approach, as the study required the use of this approach to obtain real and adequate information about Metaverse, and the virtual technology revolution, and to study the role of the United Nations in organizing the work of Metaverse with the aim of reaching solutions to the problems raised by the research. At the end of the research, the study reached several results, the most prominent of which was the interest of the United Nations in ensuring the right to freedom of expression, and issuing many international agreements to protect and enhance freedom of contact, circulation and transferring information in the digital age, while continuing terrorist groups seeking to employ rapid technological and technical progress to expand the spread of incitement letters on terrorism, spread extremist ideas to serve their interests and agendas, and to exploit the spread of dealing through the Metaverse platform to expand the recruitment of youth through social media platforms, and to provide financial support required to implement its plans to spread panic, fear and spread chaos in society. Conclusion: The necessity of urging the concerted international efforts and encouraging cooperation among countries through the organs of the United Nations to maximize the optimal exploitation of the advantages of the Metaverse world, to enhance the efforts to counter terrorism and combat the spread of incitement speeches through social media, and to cause balance to ensure that the right of freedom of speech is not eroded which contributes to effective confrontation with the online challenges and the world of Metaverse.

Comparative Study of Sentinel-1-Focused and Simulated SAR Images Using LiDAR Point Cloud Modeling for Coastal Areas

Aiming at SAR imaging for large coastal scenes, a comprehensive comparative study is performed based on Sentinel-1 raw data, SAR imaging simulation, and Google Maps. A parallel Range-Doppler (RD) algorithm is developed and applied to focus Sentinel-1 raw data for large coastal scenes, and the focused SAR image is compared with the multi-look-processed SAR image using SNAP 9.0.0 software, as well as the corresponding areas of Google Maps. A scheme is proposed to convert the LiDAR point cloud data of the coast into a 3D coastal area digital elevation model (DEM), and a tailored 3D model suitable for RaySAR simulator is obtained after statistical outlier removal (SOR) denoising and down-sampling processing. Comparison results show good agreements, which verify the effectiveness of the parallel RD algorithm as well as the backward ray-tracing-based RaySAR simulator, which serves as a powerful SAR imaging tool due to its high efficiency and flexibility. The cosine similarity between the RaySAR-simulated SAR image and Google Maps achieves 0.93, while cosine similarity reaches 0.85 between Sentinel-1 SAR-focused images with our parallel RD algorithm and multi-look SAR image processed using SNAP software. This article can provide valuable assistance for SAR system performance evaluation, SAR imaging algorithm improvement, and remote sensing applications.

A Fusion-Based Dense Crowd Counting Method for Multi-Imaging Systems

Dense crowd counting has become an essential technology for urban security management. The traditional crowd counting methods mainly apply to the scene with a single view and obvious features but cannot solve the problem with a large area and fuzzy crowd features. Therefore, this paper proposes a crowd counting method based on high and low view information fusion (HLIF) for large and complex scenes. First, a neural network based on an attention mechanism (AMNet) is established to obtain a global density map from a high view and crowd counts from a low view. Then, the temporal correlation and spatial complementarity between cameras are used to calibrate the overlap areas of the two images. Finally, the total number of people is calculated by combining the low-view crowd counts and the high-view density map. Compared to single-view crowd counting methods, HLIF is experimentally more accurate and has been successfully applied in practice.

GCL-YOLO: A GhostConv-Based Lightweight YOLO Network for UAV Small Object Detection

Precise object detection for unmanned aerial vehicle (UAV) images is a prerequisite for many UAV image applications. Compared with natural scene images, UAV images often have many small objects with few image pixels. These small objects are often obscured, densely distributed, or in complex scenes, which causes great interference to object detection. Aiming to solve this problem, a GhostConv-based lightweight YOLO network (GCL-YOLO) is proposed. In the proposed network, a GhostConv-based backbone network with a few parameters was firstly built. Then, a new prediction head for UAV small objects was designed, and the original prediction head for large natural scene objects was removed. Finally, the focal-efficient intersection over union (Focal-EIOU) loss was used as the localization loss. The experimental results of the VisDrone-DET2021 dataset and the UAVDT dataset showed that, compared with the YOLOv5-S network, the mean average precision at IOU = 0.5 achieved by the proposed GCL-YOLO-S network was improved by 6.9% and 1.8%, respectively, while the parameter amount and the calculation amount were reduced by 76.7% and 32.3%, respectively. Compared with some excellent lightweight networks, the proposed network achieved the highest and second-highest detection accuracy on the two datasets with the smallest parameter amount and a medium calculation amount, respectively.

Minimum dose path planning method for virtual nuclear facilities based on navigation mesh

The operation, maintenance and decommissioning of nuclear facilities have radiation risks. When workers work in the radiation area, it is necessary to carry out radiation protection through path planning. In order to optimize the dose field path planning algorithm, a virtual verification platform for dose field path planning is established based on the Unreal Engine. The platform includes virtual environment, dose field, and personnel dose assessment modules, which can meet the needs of dose field path planning algorithm research and optimization. Considering the complexity and variability of the environment in the operation process, a new dose minimum path planning algorithm based on navigation mesh is designed. Based on this algorithm, the number of grids for path planning in complex or large scenes is significantly reduced, and the calculation time of the algorithm is reduced while ensuring the optimal dose. The simulation results show that the minimum dose path planning method based on navigation mesh can effectively reduce path planning time and ensure the safety of personnel during the decommissioning of nuclear facilities.

A model-driven approach for fast modeling of three-dimensional laser point cloud in large substation

Using point cloud to reconstruct the 3D model of a substation is crucial for smart grid operation. Its main objective is to swiftly capture equipment point cloud data and align each device’s model within the large and noisy point cloud scene of the substation. However, substation reconstruction needs improvement due to the low efficiency of traditional noise-resistant clustering methods and challenges in accurately classifying similar-looking electrical equipment. This paper proposes an automatic modeling framework for large-scale substation point cloud scenes. Firstly, we reduce the substation scene’s dimensionality to improve clustering efficiency and establish relationships between data dimensions using a re-clustering algorithm. Next, a neural network is developed to identify various device types within clusters, even with limited subdivisions. Finally, a model library is employed to register standard models onto the target device’s point cloud, obtaining device types and orientations. Real substation data processing demonstrates the ability to rapidly extract devices from complex and noisy point cloud scenes, effectively avoiding missegmentation issues. The automatic modeling approach achieves a precise substation calculation rate of 92.86%.

Shallow Marine High-Resolution Optical Mosaics Based on Underwater Scooter-Borne Camera.

Optical cameras equipped with an underwater scooter can perform efficient shallow marine mapping. In this paper, an underwater image stitching method is proposed for detailed large scene awareness based on a scooter-borne camera, including preprocessing, image registration and post-processing. An underwater image enhancement algorithm based on the inherent underwater optical attenuation characteristics and dark channel prior algorithm is presented to improve underwater feature matching. Furthermore, an optimal seam algorithm is utilized to generate a shape-preserving seam-line in the superpixel-restricted area. The experimental results show the effectiveness of the proposed method for different underwater environments and the ability to generate natural underwater mosaics with few artifacts or visible seams.

Keypoints-based Heterogeneous Graph Convolutional Networks for construction

Artificial intelligence algorithms employed for classifying excavator-related activities predominantly rely on sensors embedded within individual machinery or computer vision (CV) techniques encompassing a large scene. The existing CV-based methods are often difficult to tackle an image including multiple excavators and other cooperating machinery. This study presents a novel framework tailored to the classification of excavator activities, accounting for both the excavator itself and the dumpers collaborating with the excavator during operations. Distinct from most existing related studies, this method centres on the transformed heterogeneous graph data constructed using the keypoints of all cooperating machinery extracted from an image. The resulting model leverages the relationships between the mechanical components of an excavator in varying activation states and the associations between the excavator and the collaborating machinery. The framework commences with a novel definition of keypoints representing different machinery relevant to the targetted activities. A customised Machinery Keypoint R-CNN method is then developed to extract these keypoints, forming the basis of graph notes. By considering the type, attribute and edge of nodes, a Heterogeneous Graph Convolutional Network is finally utilised for activity recognition. The results suggest that the proposed framework can effectively predict earthwork activities (with an accuracy of up to 97.5%) when the image encompasses multiple excavators and cooperating machinery. This solution holds promising potential for the automated measurement and management of earthwork productivity within the construction industry. Code and data are available at: https://github.com/gillesflash/Keypoints-Based-Heterogeneous-Graph-Convolutional-Networks.git.

A GPU-friendly hybrid occlusion culling algorithm for large scenes

In this paper, we present a novel hybrid occlusion culling method for large scale scenes. The basic idea is to use an iterative hierarchical Z-buffer occlusion culling algorithm to execute a coarse-grained culling via compute shader, followed by a fine-grained culling via rasterization. We also propose a forward warping method to generate a low resolution approximated depth map to accelerate the culling process. Our solution requires only one indirect multidraw command and runs on GPU totally without any CPU read-back operations. Our experimental results indicate that our algorithm outperforms the existing solutions both in performance and culling rate.