3D Visual Information Research Articles

The Inversion Location of Microleakage Source and Diffusion Backward Tracing Method Based on 3D Visual Information Model of Salt-Cavern Gas Storage

Summary The safe and stable operation of underground gas storage makes a positive contribution to national energy security. To improve the efficiency of real-time monitoring and early warning of gas leakage accidents at the wellsite of salt-cavern gas storage, an inversion location of the gas microleakage source and a backward tracing method of leakage diffusion are proposed. Through building a 3D visual information model of the salt-cavern gas storage based on building information modeling (BIM) technology and combining it with the advantages of the gas sensors, the real-time inversion location of the microleakage source and diffusion backward tracing in the wellsite can be realized. First, multiple point-type laser methane sensors were used to monitor the leakage of some key zones in real time, and a pan/tilt/zoom (PTZ) scanning laser methane telemeter was used to realize the real-time multidimensional space monitoring on-site from horizontal 360° to vertical 180°. Second, a 3D integrated monitoring platform is established to analyze the gas leakage source by combining the monitoring data and improving the mesh interpolation points. Finally, the whole data and wellsite information are imported into the 3D integrated monitoring platform to realize the inversion location of the microleakage source and the real-time monitoring of the leakage and diffusion state in the whole region. The monitoring precision of gas leakage concentration can be up to 1.0 ppm, and the location distance is within 0.1 m based on the numerical simulation of the 3D model and actual detected data from sensors in the platform. Compared with the technical means of manual assisted leak location, this method effectively solves the difficult problem of leakage monitoring and location accurately in the gas storage and can timely reduce the expansion and impact of leakage accidents.

Impact of LiDAR point cloud compression on 3D object detection evaluated on the KITTI dataset

The rapid growth on the amount of generated 3D data, particularly in the form of Light Detection And Ranging (LiDAR) point clouds (PCs), poses very significant challenges in terms of data storage, transmission, and processing. Point cloud (PC) representation of 3D visual information has shown to be a very flexible format with many applications ranging from multimedia immersive communication to machine vision tasks in the robotics and autonomous driving domains. In this paper, we investigate the performance of four reference 3D object detection techniques, when the input PCs are compressed with varying levels of degradation. Compression is performed using two MPEG standard coders based on 2D projections and octree decomposition, as well as two coding methods based on Deep Learning (DL). For the DL coding methods, we used a Joint Photographic Experts Group (JPEG) reference PC coder, that we adapted to accept LiDAR PCs in both Cartesian and cylindrical coordinate systems. The detection performance of the four reference 3D object detection methods was evaluated using both pre-trained models and models specifically trained using degraded PCs reconstructed from compressed representations. It is shown that LiDAR PCs can be compressed down to 6 bits per point with no significant degradation on the object detection precision. Furthermore, employing specifically trained detection models improves the detection capabilities even at compression rates as low as 2 bits per point. These results show that LiDAR PCs can be coded to enable efficient storage and transmission, without significant object detection performance loss.

Telepresence augmentation for visual and haptic guided immersive teleoperation of industrial manipulator

Teleoperation under human guidance has become an effective solution to extend human’s reach in various environments. However, the teleoperation system still faces challenges of insufficient sense of both visual and haptic feedback from remote environments, which results in the inadequate guidance for the operator. In this paper, a visual/haptic integrated perception and reconstruction system (VHI-PRS) is developed to provide the operator with 3D visual information and effective haptic guidance. Specifically, a visual-based telepresence augmentation method is proposed to provide the operator with virtual-reality combined visual telepresence, where the real point cloud model is directly superimposed on virtual manipulator to avoid the time-consuming process of mesh model rendering. With the utilization of visual information, a haptic-based telepresence augmentation method is proposed to provide the operator with comprehensive force feedback, including the virtual guiding force, virtual repulsive force and real-time interactive force, which greatly helps reduce the workload of operator. Finally, a user study on grab-place task is carried out to verify the effectiveness of proposed system.

상수관망 관리를 위한 3D 공간정보 시스템 및 모델 프로토타입 개발

Recently, water supply problems and the advent of the 4th Industrial Revolution, smart water supply network management using digital twin technology is becoming needs. In this study, to develop a prototype of the model and system 3D visualization information system for water supply network management based on own technology without restrictions on licenses. Based on the data from the ME(2017), an inventory DB according to the Level classification was established, and for 3D visualization based on 2D information systems, 9-steps of implementation process and verification were conducted. Comparing and analyzing the values of latitude and longitude of polygons, it was confirmed that the reliability and accuracy of the two-dimensional information system implemented in this study were secured. Therefore, 3D visualization was performed using this, and the process of designing data linkage plans within the program and implementing 3D visualization was established. The results of this study are considered insignificant in the construction of a network management space server with high economic benefits and high reliability and accuracy of GIS.

A Novel No-Reference Quality Assessment Metric for Stereoscopic Images with Consideration of Comprehensive 3D Quality Information.

Recently, stereoscopic image quality assessment has attracted a lot attention. However, compared with 2D image quality assessment, it is much more difficult to assess the quality of stereoscopic images due to the lack of understanding of 3D visual perception. This paper proposes a novel no-reference quality assessment metric for stereoscopic images using natural scene statistics with consideration of both the quality of the cyclopean image and 3D visual perceptual information (binocular fusion and binocular rivalry). In the proposed method, not only is the quality of the cyclopean image considered, but binocular rivalry and other 3D visual intrinsic properties are also exploited. Specifically, in order to improve the objective quality of the cyclopean image, features of the cyclopean images in both the spatial domain and transformed domain are extracted based on the natural scene statistics (NSS) model. Furthermore, to better comprehend intrinsic properties of the stereoscopic image, in our method, the binocular rivalry effect and other 3D visual properties are also considered in the process of feature extraction. Following adaptive feature pruning using principle component analysis, improved metric accuracy can be found in our proposed method. The experimental results show that the proposed metric can achieve a good and consistent alignment with subjective assessment of stereoscopic images in comparison with existing methods, with the highest SROCC (0.952) and PLCC (0.962) scores being acquired on the LIVE 3D database Phase I.

Research on Information Visualization of Chinese Cultural Heritage Patterns in the Digital Perspective

Abstract In today’s highly developed information age, Chinese cultural heritage is gradually shrinking, and some of them are even in danger of disappearing. In this paper, the study of Chinese cultural heritage pattern information visualization from a digital perspective is based on the 3D modeling technology method, firstly, the acquisition of cultural heritage data is studied, and the principle and method of cloud data alignment are analyzed. The data of regular surface cultural heritage and heterogeneous complex surface cultural heritage patterns are processed respectively, and the basic theoretical basis of surface reconstruction is given, including data parameterization, surface interpolation method, and surface approximation method. Based on the comparison of the accuracy of three different methods for visualizing the information of Beijing cultural heritage patterns, the experimental analysis shows that the 3D model technology information visualization is highly accurate and stable, always maintaining an accuracy of 0.8, with low technical requirements, and the limitation is that it is not suitable for use in highly complex cultural heritage. The analysis shows that the information visualization of Chinese cultural heritage patterns is conducive to the protection of the dissemination and transmission of Chinese cultural heritage, and helps young people to establish correct socialist core values under the inculcation of Chinese cultural heritage.

Lip landmark-based audio-visual speech enhancement with multimodal feature fusion network

Traditional speech enhancement uses only audio signals to obtain clean speech by removing background noise. Audio-visual speech enhancement uses additional visual information to improve the intelligibility and perceptual quality of noisy speech, which can be applied to video conferencing. However, the original visual cues contain redundant information, which causes high latency and makes the model prone to overfitting. Meanwhile, related methods do not pay much attention to exploiting critical visual features and fusing multimodal audio-visual features. In this paper, we propose an efficient multimodal feature fusion network (MFF-Net) for audio-visual speech enhancement. Specifically, we are the first to use fine-grained 3D lip landmarks to represent visual features. This visual representation can provide refined 3D visual information while protecting privacy. We design a multi-scale enhancement module (MEM) with a multi-branch structure that extracts critical multi-scale features by using dilated convolutions and attention mechanisms. Besides, we propose an audio-visual fusion module (AFM) that uses a mutually reinforcing strategy to effectively fuse visual and audio features. To verify the effectiveness of our method, we construct two speech enhancement datasets based on lip landmarks and conduct related experiments. Extensive experimental results show that our proposed MFF-Net has competitive performance compared to other methods.

Pembangunan Peta Virtual 3D untuk Gedung K, Gedung H, Gedung I, dan Gedung R di Fakultas Pertanian Universitas Lampung

The University of Lampung conducts routine activities of Introduction to New Student Campus Life (PKKMB) to introduce every building and public facility within the Faculty of Agriculture as well as every building and room inside. New Students and the general public every year. The current state of the COVID-19 pandemic has forced the University of Lampung to carry out learning activities online. Thus making every student activity impossible to do directly. So we need a method of presenting 3D visualization information. By using Multimedia Development Life Cycle (MDLC) system development method, the developed 3D virtual map application can display building layouts and facilities in 3D so that PKKMB activities related to the introduction of existing buildings and public facilities can still be delivered effectively and efficiently. This research was developed through 2 types of testing, namely testing by developers (Alpha) and customer satisfaction (Betha). Alpha testing provides test results. These Applications can operate on various Android with 7.1 to 12 of operating system and also on smartphone with 5.0 to 6.5 inches of screen. These applications can also be operated on smartphones with RAM range in 3GB to 8GB. Respondents stated that the 3D Virtual Map Application for Building K, Building H, Building I , and Building R, Faculty of Agriculture, University of Lampung, had good results with index percentages ranging from 87.5% to 95%.e basics.

3D Shape‐Morphing Display Enabled by Electrothermally Responsive, Stiffness‐Tunable Liquid Metal Platform with Stretchable Electroluminescent Device

Abstract3D displays are of great interest as next‐generation displays by providing intensified realism of 3D visual information and haptic perception. However, challenges lie in implementing 3D displays due to the limitation of conventional display manufacturing technologies that restrict the dimensional scaling of their forms beyond the 2D layout. Furthermore, on account of the inherent static mechanical properties of constituent materials, the current display form factors can hardly achieve robust and complex 3D structures, thereby hindering their diversity in morphologies and applications. Herein, a versatile shape‐morphing display is presented that can reconfigure its shape into various complex 3D structures through electrothermal operation and firmly maintain its morphed states without power consumption. To achieve this, a shape‐morphing platform, which is composed of a low melting point alloy (LMPA)‐graphene nanoplatelets (GNPs)‐elastomer composite, is integrated with a stretchable electroluminescent (EL) device. The LMPA in the composite, the key material for variable stiffness, imparts shape memory property and forms conductive pathways with GNPs enabling rapid electrothermal actuation. The stretchable EL device provides reliable illumination in 3D shape implementations. Experimental studies and proof‐of‐concept demonstrations show the potential of the shape‐morphing display, opening new opportunities for 3D art displays, transformative wearable electronics, and visio‐tactile automotive interfaces.

Study of Microbial Sulfur Metabolism in a Near Real-Time Pathway through Confocal Raman Quantitative 3D Imaging.

As microbial sulfur metabolism significantly contributes to the formation and cycling of deep-sea sulfur, studying their sulfur metabolism is important for understanding the deep-sea sulfur cycle. However, conventional methods are limited in near real-time studies of bacterial metabolism. Recently, Raman spectroscopy has been widely used in studies on biological metabolism due to its low-cost, rapid, label-free, and nondestructive features, providing us with new approaches to solve the above limitation. Here, we used the confocal Raman quantitative 3D imaging method to nondestructively detect the growth and metabolism of Erythrobacter flavus 21-3 in the long term and near real time, which possessed a pathway mediating the formation of elemental sulfur in the deep sea, but the dynamic process was unknown. In this study, its dynamic sulfur metabolism was visualized and quantitatively assessed in near real time using 3D imaging and related calculations. Based on 3D imaging, the growth and metabolism of microbial colonies growing under both hyperoxic and hypoxic conditions were quantified by volume calculation and ratio analysis. Additionally, unprecedented details of growth and metabolism were uncovered by this method. Due to this successful application, this method is potentially significant for analyzing the in situ biological processes of microorganisms in the future. IMPORTANCE Microorganisms contribute significantly to the formation of deep-sea elemental sulfur, so studies on their growth and dynamic sulfur metabolism are important to understand the deep-sea sulfur cycle. However, near real-time in situ nondestructive metabolic studies of microorganisms remain a great challenge due to the limitations of existing methods. We thus used an imaging-related workflow by confocal Raman microscopy. More detailed descriptions of the sulfur metabolism of E. flavus 21-3 were disclosed, which perfectly complemented previous research results. Therefore, this method is potentially significant for analyzing the in-situ biological processes of microorganisms in the future. To our knowledge, this is the first label-free and nondestructive in situ technique that can provide temporally persistent 3D visualization and quantitative information about bacteria.

Water Surface Targets Detection Based on the Fusion of Vision and LiDAR.

The use of vision for the recognition of water targets is easily influenced by reflections and ripples, resulting in misidentification. This paper proposed a detection method based on the fusion of 3D point clouds and visual information to detect and locate water surface targets. The point clouds help to reduce the impact of ripples and reflections, and the recognition accuracy is enhanced by visual information. This method consists of three steps: Firstly, the water surface target is detected using the CornerNet-Lite network, and then the candidate target box and camera detection confidence are determined. Secondly, the 3D point cloud is projected onto the two-dimensional pixel plane, and the confidence of LiDAR detection is calculated based on the ratio between the projected area of the point clouds and the pixel area of the bounding box. The target confidence is calculated with the camera detection and LiDAR detection confidence, and the water surface target is determined by combining the detection thresholds. Finally, the bounding box is used to determine the 3D point clouds of the target and estimate its 3D coordinates. The experiment results showed this method reduced the misidentification rate and had 15.5% higher accuracy compared with traditional CornerNet-Lite network. By combining the depth information from LiDAR, the position of the target relative to the detection coordinate system origin could be accurately estimated.

3D visualization of the state of the new secure confinement of CHNPP using microservice architecture

The purpose of this paper is to develop a software architecture for 3D visualization of information about the state of the New Safe Confinement (NSC) of the Chernobyl Nuclear Power Plant using microservices technology. The NSC was built to protect the population and the environment from the consequences of the accident at the ChNPP and to ensure the implementation of works aimed at transforming the "Shelter" facility into an environmentally safe system. The NSC is a large and complex system, the activity of which is ensured by a large number of different subsystems under the control of the Integrated Management System (IMS). However, existing information systems do not provide a sufficient level of capabilities for information visualization, forecasting and decision-making support, which determines the urgency of developing new information technologies. Given the long period of operation of the NSC, for a comprehensive solution to this problem, it is advisable to develop its digital twin, one of the functions of which is 3D visualization of information about the state of the NSC. The non-functional and functional requirements for the 3D visualization software were formed, on the basis of which its microservice architecture was developed as part of the architecture of the digital twin of the NSC. A database model of the visualization service was developed and a management system was chosen for it. Considering the large volumes of data on NSC, it is recommended to use Column-based systems to manage such databases, however, to create a prototype of the software product, you can use a relational model under PostgreSQL management. For modeling building structures, the 3D model was developed in the ArchiCAD system with the ability to display data at certain points in the three-dimensional coordinate system. The application of microservices technology ensures the flexibility of the developed architecture, the possibility of its scaling and further development throughout the long period of operation of the NSC as a component of its digital counterpart.

Points2Sound: from mono to binaural audio using 3D point cloud scenes

For immersive applications, the generation of binaural sound that matches its visual counterpart is crucial to bring meaningful experiences to people in a virtual environment. Recent studies have shown the possibility of using neural networks for synthesizing binaural audio from mono audio by using 2D visual information as guidance. Extending this approach by guiding the audio with 3D visual information and operating in the waveform domain may allow for a more accurate auralization of a virtual audio scene. We propose Points2Sound, a multi-modal deep learning model which generates a binaural version from mono audio using 3D point cloud scenes. Specifically, Points2Sound consists of a vision network and an audio network. The vision network uses 3D sparse convolutions to extract a visual feature from the point cloud scene. Then, the visual feature conditions the audio network, which operates in the waveform domain, to synthesize the binaural version. Results show that 3D visual information can successfully guide multi-modal deep learning models for the task of binaural synthesis. We also investigate how 3D point cloud attributes, learning objectives, different reverberant conditions, and several types of mono mixture signals affect the binaural audio synthesis performance of Points2Sound for the different numbers of sound sources present in the scene.

Using Mixed Reality for the Visualization and Dissemination of Complex 3D Models in Geosciences—Application to the Montserrat Massif (Spain)

In the last two decades, both the amount and quality of geoinformation in the geosciences field have improved substantially due to the increasingly more widespread use of techniques such as Laser Scanning (LiDAR), digital photogrammetry, unmanned aerial vehicles, geophysical reconnaissance (seismic, electrical, geomagnetic), and ground-penetrating radar (GPR), among others. Furthermore, the advances in computing, storage and visualization resources allow the acquisition of 3D terrain models (surface and underground) with unprecedented ease and versatility. However, despite these scientific and technical developments, it is still a common practice to simplify the 3D data in 2D static images, losing part of its communicative potential. The objective of this paper is to demonstrate the possibilities of extended reality (XR) for communication and sharing of 3D geoinformation in the field of geosciences. A brief review of the different variants within XR is followed by the presentation of the design and functionalities of headset-type mixed reality (MR) devices, which allow the 3D models to be investigated collaboratively by several users in the office environment. The specific focus is on the functionalities of Microsoft’s HoloLens 2 untethered holographic head mounted display (HMD), and the ADA Platform App by Clirio, which is used to manage model viewing with the HMD. We demonstrate the capabilities of MR for the visualization and dissemination of complex 3D information in geosciences in data rich and self-directed immersive environment, through selected 3D models (most of them of the Montserrat massif). Finally, we highlight the educational possibilities of MR technology. Today MR has an incipient and reduced use; we hope that it will gain popularity as the barriers of entry become lower.

Socially CompliAnt Navigation Dataset (SCAND): A Large-Scale Dataset of Demonstrations for Social Navigation

Social navigation is the capability of an autonomous agent, such as a robot, to navigate in a “socially compliant” manner in the presence of other intelligent agents such as humans. With the emergence of autonomously navigating mobile robots in human-populated environments (e.g., domestic service robots in homes and restaurants and food delivery robots on public sidewalks), incorporating socially compliant navigation behaviors on these robots becomes critical to ensuring safe and comfortable human-robot coexistence. To address this challenge, imitation learning is a promising framework, since it is easier for humans to demonstrate the task of social navigation rather than to formulate reward functions that accurately capture the complex multi-objective setting of social navigation. The use of imitation learning and inverse reinforcement learning to social navigation for mobile robots, however, is currently hindered by a lack of large-scale datasets that capture socially compliant robot navigation demonstrations in the wild. To fill this gap, we introduce Socially CompliAnt Navigation Dataset ( SCAND )–a large-scale, first-person-view dataset of socially compliant navigation demonstrations. Our dataset contains 8.7 hours, 138 trajectories, 25 miles of socially compliant, human tele-operated driving demonstrations that comprises multi-modal data streams including 3D lidar, joystick commands, odometry, visual and inertial information, collected on two morphologically different mobile robots–a Boston Dynamics Spot and a Clearpath Jackal–by four different human demonstrators in both indoor and outdoor environments. We additionally perform preliminary analysis and validation through real-world robot experiments and show that navigation policies learned by imitation learning on SCAND generate socially compliant behaviors.

Artificial intelligence and big data driven digital media design

Traditional digital media system can not complete the format conversion and video transcoding of massive image and video information at the same time, which leads to long time of information processing and loss of data storage. Therefore, a digital media system driven by artificial intelligence and big data is designed. Using FastDFS design of digital media data management module. Design digital media image, video conversion module and digital media resource data dictionary library. Develop image plug-ins based on the MapInfo platform. Design video plug-in, introduce virtual reality technology to retrieve image information, call video source, create CvCapture object. Design system software functions and digital media information acquisition algorithm. Intelligent artificial pixel feature acquisition technology is used to collect 3D visual information of digital media and design its pseudo-code. Compared with the traditional system, the information processing time of the designed system is shorter, and it takes 11.555 ms when there are more information objects. The experimental results show that the designed system can complete more complete storage of data.

A BIM/IFC – LADM SOLUTION ALIGNED TO THE GREEK LEGISLATION

Abstract. The ongoing rapid urbanization has led to the emerge of several complex infrastructures distinguished by multi-dimensional and overlapping property rights. The option of integrating BIM data into LA, aligned with international standards and able to handle the three-dimensional (3D) aspect of cadastral objects utilizing contemporary technologies, may be of a great importance. The main objective of this paper is to investigate the potential integration of the Building Information Model (BIM) and the Industry Foundation Classes (IFC) standard, with the international standard of Land Administration Domain Model (LADM ISO 19152 2012), in order to provide a unified solution for the registration and visualization of 3D spatial and sematic information regarding the complex Rights, Restrictions and Responsibilities (RRR) in the new complex constructions. The proposed solution aims to exploit the available 2D plans for the generation of BIMs; and rely on international LADM’s constructional guidelines to provide a standardized cadastral database harmonized with the national Greek cadastral and the legal framework and aligned with the current Greek building code. The proposed framework is tested for two multi-storey buildings. The first results are promising constituting an important tool, for the implementation of 3D cadastres.

Research of text visualization in 3D environment based on parameter study

The text visualization in 3D environment is quite complex due to the characteristics of 3D scene. In this paper we propose to study a group of 3D text information visualization based on parameter study. The primary text information visualization position is set as an initial point. Then, several different level-of-details for the texts are designed as the Event LoDs. Firstly, the parameter study for different EventLoDs is conducted, followed by the parameter study for the different size value. Based on the two groups of parameter study result, a use case is applied to test the effect. Finally, the summary and future work is given.

Application of 3D vision intelligent calibration and imaging technology for industrial robots

During modern flexible lean manufacturing, flexible operation of irregular and complex workpieces with different specifications and arbitrary placement is an essential ability of industrial robots, while it cannot be met by traditional clamping methods. Vision technology brings flexibility and convenience to industrial robots, but the common two-dimensional technology only involves three degrees of freedom (plane displacement and rotation), which hinders the positioning of arbitrarily placing workpieces (often six degrees of freedom) and disorderly sorting. In addition, for typical visual tasks in industrial environments like defect detection, accurate distinguishing of such defects as pits and scratches is challenging under two-dimensional plane imaging. The introduction of three-dimensional information provides an effective solution to this problem. Thus, in the face of increasingly complex, flexible, intelligent and personalized manufacturing needs, the acquisition and processing of 3D visual information are of much importance.

The application of 3D design technology in the design of 220kV substation in Miluo Western

Based on engineering information and geographic information data, through the integrated application of 3D modeling technology and digital collaborative design technology, the 3D visual design and information integration of power transmission and transformation projects are completed, 3D design can realize complex underground pipeline layout, cable optimization layout, collision inspection, visual display, etc. It is helpful to reduce the soft and hard collision of the project, ensure the design safety and quality, optimize the work quantity and guide the construction. In this paper, through the application of 3D design technology in the design of 220kV substation in Miluo Western, the main aspects and benefits of application are summarized, and the existing problems are analyzed for reference in other engineering applications.