3D World Research Articles

Light Field Compression With Graph Learning and Dictionary-Guided Sparse Coding

Light field (LF) data are widely used in the immersive representations of the 3D world. To record the light rays along with different directions, an LF requires much larger storage space and transmission bandwidth than a conventional 2D image with similar spatial dimension. In this paper, we propose a novel framework for light field image compression that leverages graph learning and dictionary learning to remove structural redundancies between different views. Specifically, to significantly reduce the bit-rates, only a few key views sampled and encoded, whereas the remaining non-key views are reconstructed via the graph adjacency matrix learned from the angular patch. Furthermore, dictionary-guided sparse coding is developed to compress the graph adjacency matrices and reduce the coding overheads. To the best of our knowledge, this paper is the first to achieve compact representation of cross-view structural information via adaptive learning on graphs. Experimental results demonstrate that the proposed framework achieves better performance than the standardized HEVC-based codec.

Improving Grade 8 Students’ Academic Performance and Attitude in Teaching Science through Augmented Reality

Strategic Intervention Material for Teaching Augmented Reality (SIMaTAR) is a collection of teaching and learning materials in Science which uses augmented reality technology to greatly improve learning experience through digital immersion in different 3D and 4D worlds and is under the innovative approaches in science education project of DOST-SEI. This study aimed to assess the effectiveness of using SIMaTAR mobile application as a tool on improving the Grade 8 students’ academic performance, motivation, and attitude towards learning science as well as the teachers’ assessment upon using the application. Results showed that all indicators on the teachers and students’ assessment on SIMaTAR mobile augmented reality application were rated as strongly agree. All indicators of students’ attitude towards science showed improvement after implementing the AR application in science teaching. Further, the scores of the student participants in pre-test and post-test were remarkably increased after using the application. As for the challenges, results revealed that students do not have the gadgets to use and they lack the knowledge on how to manipulate the application. Through data analysis using paired samples t-test, it is concluded that the integration of SIMaTAR in teaching science increases the engagement and interest of the students in learning the subject and the application of mobile augmented reality could substantially improve the teaching and learning process.

New Approaches to 3D Vision.

New approaches to 3D vision are enabling new advances in artificial intelligence and autonomous vehicles, a better understanding of how animals navigate the 3D world, and new insights into human perception in virtual and augmented reality. Whilst traditional approaches to 3D vision in computer vision (SLAM: simultaneous localization and mapping), animal navigation (cognitive maps), and human vision (optimal cue integration) start from the assumption that the aim of 3D vision is to provide an accurate 3D model of the world, the new approaches to 3D vision explored in this issue challenge this assumption. Instead, they investigate the possibility that computer vision, animal navigation, and human vision can rely on partial or distorted models or no model at all. This issue also highlights the implications for artificial intelligence, autonomous vehicles, human perception in virtual and augmented reality, and the treatment of visual disorders, all of which are explored by individual articles. This article is part of a discussion meeting issue 'New approaches to 3D vision'.

Circular fringe projection profilometry and 3D sensitivity analysis based on extended epipolar geometry

For fringe projection profilometry, the phase-shifting direction needs to fit the position layout of a camera and a projector, or the quality of 3D reconstruction will be poor. Two-direction fringe projection profilometry, i.e. scanning along both horizontal and vertical directions, achieves robust performance, while the number of patterns is doubled comparing with one-direction scanning. In this paper, based on the extended epipolar geometry, we propose circular fringe projection profilometry to overcome the direction issue without doubling the number of patterns. First, by introducing the farthest and nearest points to the epipole in projector space, we can robustly control the spatial frequency of circular fringes in patterns. Second, by analytically relating the circularly-shaped phase to 3D world coordinate, we naturally reduce the number of the equations from 6 down to 3 for the proposed algorithm, and compute 3D point clouds in real time. Finally, we propose 3D sensitivity to conveniently analyze the error of reconstruction point clouds. The experimental results show that, compared with two-direction fringe projection profilometry which is robust in accuracy, the proposed is of a little better quality while the number of the patterns is half.

AI기반 스마트 수질환경관리 서비스 플랫폼 개발

Recently, the frequency and range of algae occurrence in major rivers and lakes are increasing due to the increase in water temperature due to climate change, the inflow of excessive nutrients, and changes in the river environment. Abnormal algae include green algae and red algae. Green algae is a phenomenon in which blue-green algae such as chlorophyll (Chl-a) in the water grow excessively and the color of the water changes to dark green. In this study, a 3D virtual world of digital twin was built to monitor and control water quality information measured in ecological rivers and lakes in the living environment in real time from a remote location, and a sensor measuring device for water quality information based on the Internet of Things (IOT) sensor. We propose to build a smart water environment service platform that can provide algae warning and water quality forecasting by predicting the causes and spread patterns of water pollution such as algae based on AI machine learning-based collected data analysis.

Automatic concrete defect detection and reconstruction by aligning aerial images onto semantic‐rich building information model

AbstractConcrete defect information is of vital importance to building maintenance. Increasingly, computer vision has been explored for automated concrete defect detection. However, existing studies suffer from the challenging issue of false positives. In addition, 3D reconstruction of the defects to pinpoint their positions and geometries has not been sufficiently explored. To address these limitations, this study proposes a novel computational approach for detecting and reconstructing concrete defects from geotagged aerial images. A bundle registration algorithm is devised to align a batch of aerial photographs with a building information model (BIM). The registration enables the retrieval of material semantics in BIM to determine the regions of interest for defect detection. It helps rectify the camera poses of the aerial images, enabling precise defect reconstruction. Experiments demonstrate the effectiveness of the approach, which significantly reduced the false discovery rate from 70.8% to 56.8%, resulting in an intersection over union 6.4% higher than that of the traditional method. The geometry of the defects was successfully reconstructed in 3D world space. This study opens a new avenue to advance the field of defect detection by exploiting the rich information from BIM. The approach can be deployed at scale, supporting urban renovation, numerical simulation, and other smart applications.

Sensor Acquisition and Allocation for Real-Time Monitoring of Articulated Construction Equipment in Digital Twins.

The visibility available to an equipment operator on a dynamic construction site can often be blocked by various obstacles such as materials, temporary or permanent facilities, other equipment, and workers. Equipment monitoring in real-time digital twins can thus play a crucial role in accident prevention. This paper develops a scalable technical approach and presents a prototype application framework for transmitting real world sensor data to update 3D equipment models inside a graphical digital twin for concurrent visualization of a monitored construction operation. The developed framework and workflow can be extended to visualize any construction operation, as it occurs, inside a dynamic 3D world simply by outfitting the real equipment with appropriate sensors and connecting them to their virtual counterparts. The implemented proof-of-concept interface is described in the context of a real-time 3D digital twin for assisting excavator operators prevent unintended strikes with underground utilities. Experiments to validate the proposed technical approach by simulating the real-time motion of a backhoe loader’s articulated arm using orientation sensors installed on its boom, stick, and bucket are described. The experimental results characterize the scope and potential reasons for spatio-temporal discrepancies that can occur between a monitored real operation and its replicated digital twin. The effect of an operator warning mechanism based on preset safety thresholds is also investigated and described.

A learning-based crack defect detection and 3D localization framework for automated fluorescent magnetic particle inspection

With the rapid development of deep learning, target detection and segmentation in dirty backgrounds have been readily available. Fluorescent magnetic particle inspection (MPI) based on such technology will be a promising alternative for automated crack defect inspection. Most previous studies in MPI have focused only on crack detection. Instead, we frame it as a crack 3D localization problem, since cracks on non-machined surfaces of metal parts need to be polished and re-inspected, which relies on their 3D positions. Although good results were obtained in defect detection, it is still challenging to perform pixel-level segmentation of micro-cracks from the large background to obtain crack 2D pixels for 3D reconstruction. This paper proposes a two-stage convolutional neural network (CNN) method for metal parts crack defect detection and segmentation at the image-pixel level. The first stage detects and crops the potential cracks to a small area, and the second stage can learn the context of cracks in the detected patches. A window-based stereo matching method is then used to find matching pixels of cracks and to map crack image plane points to the 3D world points. We also illustrate the entire system’s model deployment and signaling work to apply these methods. Both computational and experimental results based on the system are presented for validation. The training precision of target detection reaches 96.3%, its average precision reaches 85.4%, and the average precision reaches 98.3% when the Intersection-over-Union (IoU) threshold is 0.5. The Dice score reaches 94% in pixel-level segmentation, and the average precision is 99.3% when the probability threshold is set to 0.5. The corresponding efficiency reaches 19 FPS and 18 FPS, and the mean absolute errors of 3D coordinates of reconstructed crack defects are all within 1 mm in X-, Y- and Z- directions.

RedNavi: Building a 3D Scene of the Current Sea from AIS Data

The Automatic Identification System (AIS) is a kind of navigation equipment that exchanges a wealth of essential information among vessels and between ships to shore through Very High Frequency. Currently, identification and other navigational information can be obtained in real time with AIS data integrated into other shipborne systems, such as the Electronic Chart Display and Information System and radar. However, at present, AIS information is represented in a two-dimensional (2D) way, which is not the same as the three-dimensional (3D) world people perceive visually. In this paper, we introduce RedNavi, a sustainable computer 3D scene building system that visualizes the current sea, specifically the environment and traffic conditions around the ownship, using received AIS data. RedNavi has a wide range of application scenarios. Applying to the maritime education and training field, it can serve as a bridge between the 2D and 3D worlds, helping less experienced trainees build up their capabilities. Applying to actual navigation, it can provide the deck officer with another visual aid to their lookout in addition to existing 2D information systems. In addition, given the microservices architecture RedNavi adopts, the development, deployment, and maintenance processes become relatively lighter, faster, and easier, and therefore more sustainable than traditional monolithic systems.

Metaverse ve Pazarlamaya Etkileri

Dijital teknolojiler; iş modellerini, iş ortamlarını, pazarları ve pazarlama iletişimi paradigmasını değiştirmektedir. Özellikle koronavirüs pandemisi ile birlikte dünya kültürü yüz yüze olmaktan çıkmış, dijital teknolojileri kullanan çevrimiçi kültürlerin çoğalmasına yol açmıştır. Bu eğilim ile birlikte sanal gerçeklik ve artırılmış gerçeklik gibi teknolojik gelişmelerin yanı sıra 3D sanal dünya olarak ifade edilen Metaverse kavramı 2021 yılı itibariyle kültür, sanat, eğitim, turizm gibi pek çok alanda dikkat çekmektedir. Metaverse, 3D interaktif dijital alanda her türlü markayı sunma konusundaki potansiyeli ile geleceğin yeni pazarlama platformu olması beklenmektedir. Metaverse, ürün ve hizmetlerin işlevselliğini artırarak müşteri deneyimi yaratmak amacı güden markalar için akıllıca bir yol olabilir. Bu çalışmanın amacı; hemen hemen her sektörde devrim yaratacak olan ve internetin 3 boyutlu versiyonu olarak kabul edilebilecek Metaverse kavramını ortaya koymayı ve pazarlamaya etkisinin kavramsal düzeyde açıklamayı amaçlamaktadır. Literatürde henüz sınırlı sayıda çalışma mevcut olduğu için bu çalışma da literatüre dayalı keşifsel araştırma yapılmıştır. Literatürde metaverse pazarlaması ile ilgili çalışmalar incelenmiştir. Sonuç olarak; işletmelerin metaverse ile ilgili henüz emekleme evresinde olduğu, işletmelerin metaverse uygulamalarında başarıya ulaşmaları için en önemli etkenin ultra hızlı internetin olması gerektiği, işletmelerin hizmet tedarik ekosistemlerini destekleyebileceği ve markaların henüz başlangıç düzeyindeki metaverse uygulamaları ile hayal ile sınırlı hedonik ve faydacı müşteri deneyimi yaşatabilecekleri söylenebilir. Ayrıca metaverse ortamında işletmelerin fiziksel ve dijital ürün ve hizmetleri pazarlayabilecekleri, işletmelerin yaratıcı faaliyetlerde üretim maliyetlerini düşürebildiği, NFT’ler aracılığıyla gelir elde edebileceği, markaların yaratıcılıklarını en üst düzeye çıkarabileceği, marka bilinirliklerini artırabileceği, yeni ürün tasarım fikirlerini test edebilecekleri, müşterilerin sanal mağazadaki tüm alışveriş deneyimlerini görebileceği, müşteri hakkında veri toplayabilecekleri ve bu verilerle fiziksel mağazalarını geliştirebilecekleri sonucuna ulaşılmıştır.

Metaverse - The Future of Virtual World

Since the popularization of the Internet in the 1990s, cyberspace has kept evolving. In addition to social networks, video conferencing, virtual 3D worlds (like VR Chat), augmented reality software (like Pokemon Go), and non-fungible token games, we have also developed other computer-mediated virtual environments (e.g., Upland). Such virtual environments, albeit non-perpetual and unconnected, have brought us various degrees of digital transformation. The term `Metaverse' has been coined to further facilitate digital transformation in every aspect of our physical lives. At the core of the metaverse stands the vision of an immersive Internet as a gigantic, unified, persistent and shared realm. While the metaverse may seem futuristic, catalyzed by emerging technologies such as Extended Reality, 5G and Artificial Intelligence, the digital `big bang' of our cyberspace is not far away. Technologies are the enablers that drive the transition from the current Internet to the metaverse. Enabling technologies that drive this transition rigorously are Extended Reality, User Interactivity (Human-Computer Interaction), Artificial Intelligence, Blockchain, Computer Vision, IoT and Robotics, Edge and Cloud computing and Future Mobile Networks. In terms of applications, the metaverse ecosystem allows human users to live and play within a self-sustaining, persistent, and shared realm. User-centric factors are Avatar, Content Creation, Virtual Economy, Social Acceptability, Security and Privacy and Trust and Accountability.

GoThrough: A tool to render and interact with arbitrary planar transformative portals

Portals are commonly used in video games (e.g., Portal and the classic Asteroids). In this work, we introduce GoThrough, a tool that enables users with little to no previous knowledge to add transformative portals to 3D scenes in the Unity game engine. GoThrough implements both the visual and physical aspects of portals, leaving room for other extensions by the user. We introduce an approach for portals to work robustly in terms of geometry and rendering and explore common pitfalls (and how to handle them). Lastly, we put in evidence several portal-based mechanics and implement use cases to demonstrate them. The tool is shown to work in various example scenarios and has been evaluated quantitatively for performance, performing in real-time in various scenarios. We conducted user tests to analyse GoThrough qualitatively and an experiment of experts to understand how the tool hinders or empowers their expressiveness. With a SUS score of 87.5, we concluded that GoThrough is intuitive enough to be used by non-experts, making the process of creating impossible 3D worlds much less cumbersome. At last, experts were able to conceive unanticipated mechanics and revealed that the tool allowed prototypes that would not be developed in a feasible time otherwise.

S-08-01 Two-dimensional (2D) materials in a three-dimensional (3D) world: an introduction

S-08-01 Two-dimensional (2D) materials in a three-dimensional (3D) world: an introduction

Real-Time Monocular Joint Perception Network for Autonomous Driving

Comprehensive and accurate perception of the real 3D world is the basis of autonomous driving. However, many perceptual methods focus on a single task or object type, and the accuracy of existing multi-task or multi-object methods is difficult to balance against their real-time performance. This paper presents a unified framework for concurrent dynamic multi-object joint perception, which introduces a real-time monocular joint perception network termed MJPNet. In MJPNet relative weightings are automatically learned by a series of developed network branches. By training an end-to-end deep convolutional neural network on a shared feature encoder and many proposed decoding sub-branches, the information of the 2D category and 3D position/pose/size of an object are reconstructed both simultaneously and accurately. Moreover, the effective information among subtasks is transferred by multi-stream learning, guaranteeing the accuracy of each task. Compared to various state-of-the-arts, comprehensive evaluations on the benchmark of challenging image sequences demonstrate the superior performance of our 2D detection and 3D reconstruction of depth, lateral distance, orientation, and heading angle. Moreover, on the KITTI test set, the real-time runtime (up to 15 fps) of MJPNet significantly outran the public state-of-the-art visual detection methods. Accompanying video: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://youtu.be/Z-goToOlI94</uri> .

Anti-aliasing convolution neural network of finger vein recognition for virtual reality (VR) human-robot equipment of metaverse.

Metaverse, which is anticipated to be the future of the internet, is a 3D virtual world in which users interact via highly customizable computer avatars. It is considerably promising for several industries, including gaming, education, and business. However, it still has drawbacks, particularly in the privacy and identity threads. When a person joins the metaverse via a virtual reality (VR) human-robot equipment, their avatar, digital assets, and private information may be compromised by cybercriminals. This paper introduces a specific Finger Vein Recognition approach for the virtual reality (VR) human-robot equipment of the metaverse of the Metaverse to prevent others from misappropriating it. Finger vein is a is a biometric feature hidden beneath our skin. It is considerably more secure in person verification than other hand-based biometric characteristics such as finger print and palm print since it is difficult to imitate. Most conventional finger vein recognition systems that use hand-crafted features are ineffective, especially for images with low quality, low contrast, scale variation, translation, and rotation. Deep learning methods have been demonstrated to be more successful than traditional methods in computer vision. This paper develops a finger vein recognition system based on a convolution neural network and anti-aliasing technique. We employ/ utilize a contrast image enhancement algorithm in the preprocessing step to improve performance of the system. The proposed approach is evaluated on three publicly available finger vein datasets. Experimental results show that our proposed method outperforms the current state-of-the-art methods, improvement of 97.66% accuracy on FVUSM dataset, 99.94% accuracy on SDUMLA dataset, and 88.19% accuracy on THUFV2 dataset.

Three-dimensional feature extraction using local reference frame for detecting human–object interaction

To understand the visual world, a device knows not only the instances but also how they interact. Humans are at the center of such interactions. Detection of human–object interaction (HOI) is one of the growing research fields in computer vision. However, identifying HOIs due to the large label space of verbs and their interaction with various object types still needs much research. We focus on HOIs in images, which is necessary for a deeper understanding of the scene. In addition to two-dimensional (2D) information, such as the appearance of humans and objects and their spatial location, three-dimensional (3D) status, especially in the configuration of the human body and object as well as their location and spatial, can play an important role in learning HOI. The mapping of 2D to 3D world adds depth information to the problem. These issues led us to collect 3D information along with the 2D features of the images to provide more accurate results. We show 3D attributes, such as face transformation, the viewing angle, the position of an object, and its related location to the human face, can improve HOI learning. The results of experiments on large-scale data show that our method has been able to improve the outcome of interactions.

Model for wireless image correlation assisted by sensors based on 3D display technology

The utilization of 3D display systems offers significant advantages to end-users. Sequencing technologies are essential for accessing an autonomous, auto-balanced, non-glass 3D world with greatly improved visual acuity. To have both social and emotional depth cues on a 3D display is difficult. Because of the theoretically constant feature vectors provided, certain 3D overall volume display approaches cannot produce shadows or structures. Hence, in this paper, a Wireless Sensor Assisted Image Correlation framework (WSAICF) has been proposed to enhance the current competitiveness, performance experience, and better visual understanding of the 3d display systems. In 3D presentations, the Steep Learning algorithm is used to enhance social depth estimations rapidly. The WSAICF method is integrated with the Deep Learning algorithm to enhance the emotional depth indicators of 3D display systems. The simulation result shows that an image correlation system achieves 97 % reliability, 99 % performance, 96 % viability, 11 % resilience, and 95 % efficiency.

Conceptual juggling

Physics, with its emphasis on what we observe physically happening in the 3D world, probably has more scope for kinaesthetic learning than most subjects, finds Laura Hiscott.

Flight AI using Reinforcement Learning

This paper will be featuring an aircraft with fully featured physics simulated in unity engine. The agent or the AI will be created for possessing and controlling the aircraft so as to navigate it in the 3D world space environment provided. The agent will have to consider the different physical dynamics applied on a real-world aircraft and based on these parameters it will have to create an effective piloting for the aircraft. Various other aspects such as engine dynamics and the fuel parameters will also be considered for an effective training environment.

Vision - A space for digital learning and exploring pedagogies: Virtual world education

&#x0D; &#x0D; &#x0D; The Australian and New Zealand Virtual Worlds Working Group (VWWG) was established in 2009. Members of the group have written papers for ASCILITE conferences since 2010. Each paper’s intention is to provide an overview on using virtual worlds in higher education, especially the practical aspects of incorporating 3D worlds to enhance student learning. This VWWG’s paper looks at a virtual world as a space for digital learning and exploratory pedagogy – meaning, that the authors perhaps do not perceive virtual worlds as ordinary. However, they do acknowledge that those who do not regularly engage in virtual worlds may see it as new and innovative, perhaps even untested ground, but still with a degree of unfamiliarity. A survey was sent to the members of the VWWG in which the narratives have provided rich data for in depth understanding. The themes focused in the survey were ‘disruptive thinking’, ‘emerging ideas’ and ‘lateral connections’, from the perspective of the respondents’ experiences in the implementation of virtual worlds in education. They discuss the idea of a virtual world used in education as a new treasure, or perhaps it has been in use for some time and now classified as traditional.&#x0D; &#x0D; &#x0D;