3D Model Of Object Research Articles

Neural radiance fields as a complementary method to photogrammetry for forensic 3D documentation: Initial comparative insights

ObjectivesPhotogrammetry is widely used in forensic practice to create 3D models of crime scenes, bodies, living individuals, and objects. However, it has limitations in accurately capturing transparent, reflective, and low-texture surfaces, which can hinder forensic investigations. Neural Radiance Fields (NeRFs), a recently developed method, offer a potential solution by creating more accurate and detailed 3D models in these challenging contexts. This study aims to evaluate whether NeRFs can serve as effective alternatives to structure-from-motion (SfM) photogrammetry for recording forensic autopsies. Materials and MethodsPhotogrammetric scans were performed on a variety of forensic subjects, including a cadaver with skin discoloration and epidermal exfoliation, a metal trashcan, a vehicle, and a mock crime scene. The scans were processed using traditional photogrammetry software (Meshroom) and compared with NeRF-based visualizations generate using instant neural graphics primitives. ResultsNeRF-based models provided more lifelike and detailed visualizations than photogrammetry, particularly when documenting transparent, reflective, or featureless surfaces. NeRF demonstrated superior capability in capturing complex details that photogrammetry struggled with. ConclusionNeRF technology shows considerable promise for improving the documentation of forensic autopsies, offering enhanced visual fidelity for challenging surfaces such as transparent or reflective materials. While the method presents challenges related to editing, software compatibility, and high computational demands, its potential benefits in forensic investigations are evident and merit further exploration.

Ultrafast diffraction algorithms for light-in-flight holography

Digital light-in-flight (LIF) holography is an ultrafast imaging technique capable of single-shot simultaneous 3D and femtosecond time resolution acquisitions of light pulse propagation. However, the numerical diffraction algorithms used to model light on femtosecond timescales are currently limited in scope, accuracy, and efficiency. We derive an analytical model capable of modeling LIF hologram formation for various optical setup configurations, able to model 3D objects and precisely account for the limited temporal coherence of the signal. We design an efficient algorithmic implementation and validate the system in numerical simulations and with an experimental LIF holographic recording setup. We report ultrafast numerical diffraction over 10,000 times faster than the reference technique, with higher accuracy and capable of modeling 3D samples, thereby broadening its application domain.

Monocular Pose and Shape Reconstruction of Vehicles in UAV imagery using a Multi-task CNN

Estimating the pose and shape of vehicles from aerial images is an important, yet challenging task. While there are many existing approaches that use stereo images from street-level perspectives to reconstruct objects in 3D, the majority of aerial configurations used for purposes like traffic surveillance are limited to monocular images. Addressing this challenge, a Convolutional Neural Network-based method is presented in this paper, which jointly performs detection, pose, type and 3D shape estimation for vehicles observed in monocular UAV imagery. For this purpose, a robust 3D object model is used following the concept of an Active Shape Model. In addition, different variants of loss functions for learning 3D shape estimation are presented, focusing on the height component, which is particularly challenging to estimate from monocular near-nadir images. We also introduce a UAV-based dataset to evaluate our model in addition to an augmented version of the publicly available Hessigheim benchmark dataset. Our method yields promising results in pose and shape estimation: utilising images with a ground sampling distance (GSD) of 3 cm, it achieves median errors of up to 4 cm in position and 3° in orientation. Additionally, it achieves root mean square (RMS) errors of \\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\pm 6$$\\end{document} cm in planimetry and \\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\pm 18$$\\end{document} cm in height for keypoints defining the car shape.

Refined Prior Guided Category-Level 6D Pose Estimation and Its Application on Robotic Grasping

Estimating the 6D pose and size of objects is crucial in the task of visual grasping for robotic arms. Most current algorithms still require the 3D CAD model of the target object to match with the detected points, and they are unable to predict the object’s size, which significantly limits the generalizability of these methods. In this paper, we introduce category priors and extract high-dimensional abstract features from both the observed point cloud and the prior to predict the deformation matrix of the reconstructed point cloud and the dense correspondence between the reconstructed and observed point clouds. Furthermore, we propose a staged geometric correction and dense correspondence refinement mechanism to enhance the accuracy of regression. In addition, a novel lightweight attention module is introduced to further integrate the extracted features and identify potential correlations between the observed point cloud and the category prior. Ultimately, the object’s translation, rotation, and size are obtained by mapping the reconstructed point cloud to a normalized canonical coordinate system. Through extensive experiments, we demonstrate that our algorithm outperforms existing methods in terms of performance and accuracy on commonly used benchmarks for this type of problem. Additionally, we implement the algorithm in robotic arm-grasping simulations, further validating its effectiveness.

ARCHAEOLOGICAL RESEARCH IN 2023 ON THE TERRITORY OF THE DOMINICAN MONASTERY IN VINNYTSIA

Archaeological researches of 2023 on the territory of the site of architecture and urban planning «Dominican Monastery» in Vinnytsia was carried out by the Archaeological Expedition of the Vinnytsia Regional Museum of Local Lore as part of the «Program of Archaeological Research on the Territory of Sites which are part of the Vinnytsia City Territorial Community for 2022—2024» approved by the Vinnytsia City Council. The main task for 2023 was to open a preserved excavation area (excavation area 1) and to complete it. This excavation area was established and explored in 2013—2018 by the Archaeological Expedition of the Institute of Archaeology of the National Academy of Sciences of Ukraine, headed by Dr. Larysa Vynogrodska. In the Excavation area 1 several objects in 2023 were discovered: building 1 — the remains of the foundations of a cult (?) building from the late 17th — the first half of the 18th century and constructive ditches of the building (the name and number according to the numbering of Dr. L. Vynogrodskaya); building 2 — the remains of the foundations of the building, constructed in the late 19th — the first half of the 20th century; building 3 is a deepened structure, explored in 2013—2014 (the name and number according to the numbering made by Dr. L. Vynogrodska); building 4 — a brick crypt (?) of the late 17th — early 18th centuries; object 1 — deepened with a step (partially explored) — late 16th — 17th centuries; object 2 is a burial contained the remains of five persons (burials 3—7), burials 5 and 6 are dated ca. late 17th — early 18th century; pits 1, 3 — objects of the 20th century (pit 2 is the object explored in 2013—2014); burial 1 — an animal skeleton in anatomical order which was discovered in the 17th—19th century layer (sector B6—B7 in the excavation area); burial 2 — a human burial approximately dated to the 18th century. Photofixation, processing and drawing of the material have been done, orthophoto plans and 3D modeling of objects in excavation area 1 were made based on the results of the archaeological expedition of the Vinnytsia Regional Museum of Local Lore in 2023. In addition to archaeological research, all known written sources of the 17th century about the territory of the Dominican monastery, its founders and clergy were widely involved. These sources were processed by researcher of Vinnytsia Museum Victoria Kolesnyk. Due to her sophisticated research in 2023 it was possible to attribute the architectural remains in excavation area-1 as the stone foundations of the Dominican monastery during its first phase of its existence. Probably, the discovered remains are traces of the renovation of the monastery building from the late 17th — the first half of the 18th century and existed until the time of the second phase of construction, which began in the second half of the 18th century. The result of the second phase of construction is the raising of a complex of buildings which are in use for now and as a complex with a status of National Importance, named the «Dominican Monastery».

Applications of 3D printing and digital transformation in aerospace engineering and telecommunication technologies

The aim of the development is to present the overall process of building functional prototypes for the purposes of aerospace engineering and telecommunication technologies, based on specific examples from the author's scientific and practical work. The scientific-applied contribution of the research is expressed in the correct selection and comparative analysis of compatible software products and devices for 3D modeling, digital transformation and 3D printing of objects from the space infrastructure and. in particular, of radio technical devices as key components of telecommunication satellites. The presented practical results are in the form of a final product that, if the criteria for precise modeling and the selection of a suitable material for 3D printing are met, can satisfy the quality criteria of the relevant industry for which it is intended.

Conversion of an Entire Image into a 3D Scene with Object Detection and Single 3D Models

Abstract: The paper presents a framework for the extraction of objects from 2D images using object detection techniques in open CV and then generating 3D models of the segmented objects. To achieve precise object detection, the suggested methodology blends deep learning and computer vision methods. The 3D models that are produced have fine-grained geometric and textural features, which improves the object reconstruction's accuracy. The usefulness and robustness of our technique in various settings are demonstrated by experimental findings on a variety of datasets. For use in robotics, virtual reality, and the preservation of cultural heritage, where precise 3D models of actual objects are essential, this research shows great promise.

Prompt determination of predictive parameters for mining-technogenic landscape objects

The presented research is aimed at developing tools for quickly determining the geometric parameters of mining-technogenic landscape objects to solve various geotechnical and engineering problems, such as reclamation of lands disturbed by mining operations, as well as accumulated industrial waste utilization. The task of quickly determining the geometric parameters of the mining-technogenic landscape forms is proposed to be performed using the Blender software product for 3D-modeling and the Blender GIS Addon. The 3D models for mining-technogenic landscape objects have been constructed. In addition, the volumes of mined-out space at the granite quarry, the earth’s surface failure zone at the iron-ore mine and the mine waste rock dump have been determined. As a result, the values obtained were compared with actual data. It has been found that with the correct display of mining-technogenic landscape objects in the Google geospatial system, the proposed tools based on the Blender GIS Addon achieve high accuracy in determining parameters. The proposed method is predictive and cannot be independent, and it must be used in synthesis with surveying measurements of the parameters for mining-technogenic landscape objects and is a kind of a certain addition. However, it allows solving large-scale strategic problems in geoengineering.

Fast 3D Reconstruction for Unknown Space Targets via Neural Radiance Field

In this study, a novel residual space object 3D model reconstruction approach is proposed. First, several low-cost CubeSats are assigned to carry out multi-view image capture in the vicinity of the space target, yielding a series of images with relative pose information. Then, a fast 3D reconstruction approach based on NeRF is introduced, in which real-time performance and reconstruction quality are improved for fast on-board 3D reconstruction. Finally, a simulation test is designed to verify the performance of the proposed method, results show that the proposed NeRF-based method is superior to traditional method in terms of reconstruction speed and texture richness.

Challenges in geometric modelling–A comparison of students’ mathematization with real objects, photos, and 3D models

Mathematical modelling aims at contributing to the involvement of reality in mathematics education. As an example, geometric modelling can be implemented by the use of real objects in modelling tasks. Still, (geometric) modelling tasks can be a challenge for students, especially in the transfer from reality to mathematics, which is referred to as mathematization. Since the representation of a real object in tasks might differ, the question arises, which challenges can be observed when working in different task settings. In a study with 19 secondary school students, the task settings (1) outdoors at the real object, (2) indoors with photos of the real object, and (3) indoors with a 3D model of the real object are compared. Based on video recordings, differences concerning the students’ challenges are examined. The results highlight challenges in estimating and measuring when working at the real object, scale and perspective when working with photos and the transfer between representation and object when working with 3D models.

Streamlining LEGO® model design: an automated optimisation approach

LEGO® models have become increasingly popular beyond their traditional toy and entertainment applications, and are now used in fields such as education, cognitive science, engineering and robotics. However, designing LEGO® models for such diverse applications poses new challenges in terms of automation and optimisation. To address such issues, this paper proposes a design support system that automatically generates LEGO® models from any 3D model of a target object using a specific set of standard LEGO® bricks in a layer-by-layer fashion. The system converts the 3D model into a LEGO® model using the minimum number of LEGO® bricks required. The core of the proposed system lies in the use of a greedy algorithm for merging optimisation of each layer of bricks and a depth-first search algorithm for optimising the connection between multiple layers of bricks. Experimental results on several 3D graphics validate the effectiveness of the proposed methodology and its ability to output the corresponding LEGO® model and detailed layout illustrations for each layer of bricks. The proposed system can provide LEGO® builders with an automated optimisation approach for designing LEGO® models, minimising errors compared to manual design.

Експертна система для підтримки побудови тривимірних моделей об’єктів методом фотограмметрії

The task of building high-quality three-dimensional (3D) models of objects is relevant, since such 3D models are widely used in various fields of science, technology and medicine. In this work, the construction of 3D models is performed by the photogrammetry method, which consists in the construction of a 3D model of an object based on a series of its photographs. The advantages of the photogrammetry method are low hardware requirements and relatively high accuracy. To build 3D models of objects by photogrammetry, the 3DF Zephyr program was used, which contains a set of tools for pre-processing images, reconstructing 3D models, editing and measuring the dimensions of 3D models, and exporting the obtained models. The principles of building three-dimensional models of objects by the method of photogrammetry based on initial images are considered. The main stages of building 3D models are described: calculation of sparse point cloud, key points, dense point cloud, polygon grid, texture grid. Model parameters are also edited and analyzed. An expert system was developed in the CLIPS environment to select the correct modes for building a 3D model. The knowledge base of the expert system contains production rules that allow you to establish the correct modes of building a 3D model based on the initial facts. 30 facts-conditions have been developed that describe the conditions for building a three-dimensional model. 20 facts-consequences and 15 facts-recommendations for building a 3D model have been developed. Using the developed rules, 36 production rules were built. Experimental verification of the developed system was carried out. Three-dimensional models of objects were built using the 3DF Zephyr program. After entering the facts that describe the process of obtaining the model into the expert system, a number of recommendations were obtained, in particular, to increase the area of textured surfaces and use uniform lighting of objects. After following these recommendations, the model was built with satisfactory accuracy.

Part2Point: A Part-Oriented Point Cloud Reconstruction Framework.

Three-dimensional object modeling is necessary for developing virtual and augmented reality applications. Traditionally, application engineers must manually use art software to edit object shapes or exploit LIDAR to scan physical objects for constructing 3D models. This is very time-consuming and costly work. Fortunately, GPU recently provided a cost-effective solution for massive data computation. With GPU support, many studies have proposed 3D model generators based on different learning architectures, which can automatically convert 2D object pictures into 3D object models with good performance. However, as the demand for model resolution increases, the required computing time and memory space increase as significantly as the parameters of the learning architecture, which seriously degrades the efficiency of 3D model construction and the feasibility of resolution improvement. To resolve this problem, this paper proposes a part-oriented point cloud reconstruction framework called Part2Point. This framework segments the object's parts, reconstructs the point cloud for individual object parts, and combines the part point clouds into the complete object point cloud. Therefore, it can reduce the number of learning network parameters at the exact resolution, effectively minimizing the calculation time cost and the required memory space. Moreover, it can improve the resolution of the reconstructed point cloud so that the reconstructed model can present more details of object parts.

VEHICLE POSE AND SHAPE ESTIMATION IN UAV IMAGERY USING A CNN

Abstract. Vehicle reconstruction from single aerial images is an important but challenging task. In this work, we introduce a new framework based on convolutional neural networks (CNN) that performs monocular detection, pose, shape and type estimation for vehicles in UAV imagery, taking advantage of a strong 3D object model. In the final training phase, all components of the model are trained end-to-end. We present a UAV-based dataset for the evaluation of our model and additionally evaluate it on an augmented version of the Hessingheim benchmark dataset. Our method presents encouraging pose and shape estimation results: Based on images of 3 cm GSD, it achieves median errors of up to 5 cm in position and 3° in orientation, and RMS errors of ±7 cm and ±24 cm in planimetry and height, respectively, for keypoints describing the car shape.

Chaotic color multi-image compression-encryption/ LSB data type steganography scheme for NFT transaction security

With the rapid development of blockchain technology, the security of non-fungible tokens (NFT) in the transaction process has attracted much attention. In the transaction process, the commoditized NFT images will inevitably involve display and leakage problems, which is likely to lead to economic losses drink copyright disputes between the trading parties. To protect transaction security, a chaotic color multi-image compression encryption/LSB data-type steganography scheme is proposed in the paper. A series of chaotic sequences are obtained by iterating the chaotic map, while compression sensing (CS) is introduced to compress multiple secret images and fuse the compressed secret images into one large secret image. Then this image is encrypted, and the encrypted large secret image is hidden on multiple cover images by steganography. This encryption can hide the change in the statistical properties caused by steganography. Finally, the cover image is combined with the 3D object model by using it as a texture for the 3D object model to realize the type steganography of the image data. The simulation and performance test results of the scheme illustrate that the scheme has a large enough key space and steganography capacity, as well as good resistance to differential attacks, statistical attacks, compression reconstruction quality, and robustness. This scheme can well protect the transaction security of NFT images, and at the same time open a channel connecting 2D images and 3D obj textured models, which provides a new idea for steganography.

Robotic Continuous Grasping System by Shape Transformer-Guided Multiobject Category-Level 6-D Pose Estimation

Robotic grasping is one of the key functions for realizing industrial automation and human-machine interaction. However, current robotic grasping methods for unknown objects mainly focus on generating the 6D grasp poses, which cannot obtain rich object pose information and are not robust in challenging scenes. Based on this, we propose a robotic continuous grasping system that achieves end-to-end robotic grasping of intra-class unknown objects in 3D space by accurate category-level 6D object pose estimation. Specifically, to achieve object pose estimation, first, we propose a global shape extraction network (GSENet) based on ResNet1D to extract the global shape of an object category from the 3D models of intra-class known objects. Then, with the global shape as the prior feature, we propose a transformer-guided network to reconstruct the shape of intra-class unknown object. The proposed network can effectively introduce internal and mutual communication between the prior feature, current feature, and their difference feature. The internal communication is performed by self-attention. The mutual communication is performed by cross-attention to strengthen their correlation. To achieve robotic grasping for multiple objects, we propose a low-computation and effective grasping strategy based on the pre-defined vector orientation, and develop a GUI for monitoring and control. Experiments on two benchmark datasets demonstrate that our system achieves state-of-the-art (SOTA) 6D pose estimation accuracy. Moreover, the real-world experiments show that our system also achieves superior robotic grasping performance, with a grasping success rate of 81.6 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\%$</tex-math></inline-formula> for multiple objects. Code and trained models are released at https://github.com/CNJianLiu/6D-CLGrasp

Augmented Reality-Based BIM Data Compatibility Verification Method for FAB Digital Twin implementation

With the advancement of state-of-the-art technologies, the semiconductor industry plays a key role as an essential component in the manufacture of various electronic products. Since the manufacturing of a semiconductor goes through very sophisticated and complex processes, efficient and accurate work and management are essential in the design, construction, and operation stages of the semiconductor fabrication (FAB) plant. Recently, the combined application of building information modeling (BIM) and augmented reality (AR) technology has gained increased attention in this semiconductor FAB industry as an advanced way to improve work efficiency and accuracy while eliminating other related problems, such as human errors. Despite the perceived benefits of combined use of BIM and AR, many technical problems still exist when integrating the target test model and the 3D virtual object model using BIM data and existing AR visualization technology, due to the unique characteristics of the FAB sites. To solve these problems, this study proposed an AR-based real-time BIM data compatibility verification method for future FAB digital twin implementation and demonstrated that it could be converted into a system and applied to actual FAB sites. As a result of the development and verification of this system, the proposed AR-based real-time BIM data compatibility verification system enables the accurate fitting of the AR model and actual object through AR tracking and anchoring technology considering the characteristics of FAB sites. After the fitting, the system was able to maintain compatibility, even when the camera moved and the marker moved away from the screen. By expanding the effective distance of compatibility between the AR model and the actual object, it was possible to increase the AR application range between the 3D virtual object model and the test target model and to improve the compatibility.

Utilising tourist pictures to generate 3D models and automatic multi-view stereo network design

The design of image networks plays a pivotal role in image-based 3D shape reconstruction and data processing, particularly when applying combined SfM/MVS methods. This paper presents an approach to designing and planning a multi-view images network for cultural heritage objects and sites based on a rough 3D model generated from public datasets of tourist images. Images for three famous cultural heritage buildings in India, namely Taj Mahal, Qutub Minar and Hawa Mahal, were used in this study to achieve the targeted goal. The results obtained confirmed that the Agisoft Metashape software made it possible to acceptably reconstruct the entire body of the studied structures. However, it the results showed when generating point clouds and 3D models of slender objects, i.e., the Qutub Minar, it is not possible to use only images acquired from the ground since most tourists wishing to correctly represent the shape take tilted images. For this reason, it is necessary to use videos or photos from UAVs. Generally, it can be concluded that the adopted approach in this study is a promising alternative for the conventional methods that are currently used for the shape reconstruction of cultural heritage objects and sites.

Viewpoint dependence and scene context effects generalize to depth rotated three-dimensional objects.

Viewpoint effects on object recognition interact with object-scene consistency effects. While recognition of objects seen from "noncanonical" viewpoints (e.g., a cup from below) is typically impeded compared to processing of objects seen from canonical viewpoints (e.g., the string-side of a guitar), this effect is reduced by meaningful scene context information. In the present study we investigated if these findings established by using photographic images, generalize to strongly noncanonical orientations of three-dimensional (3D) models of objects. Using 3D models allowed us to probe a broad range of viewpoints and empirically establish viewpoints with very strong noncanonical and canonical orientations. In Experiment 1, we presented 3D models of objects from six different viewpoints (0°, 60°, 120°, 180° 240°, 300°) in color (1a) and grayscaled (1b) in a sequential matching task. Viewpoint had a significant effect on accuracy and response times. Based on the viewpoint effect in Experiments 1a and 1b, we could empirically determine the most canonical and noncanonical viewpoints from our set of viewpoints to use in Experiment 2. In Experiment 2, participants again performed a sequential matching task, however now the objects were paired with scene backgrounds which could be either consistent (e.g., a cup in the kitchen) or inconsistent (e.g., a guitar in the bathroom) to the object. Viewpoint interacted significantly with scene consistency in that object recognition was less affected by viewpoint when consistent scene information was provided, compared to inconsistent information. Our results show that scene context supports object recognition even when using extremely noncanonical orientations of depth rotated 3D objects. This supports the important role object-scene processing plays for object constancy especially under conditions of high uncertainty.

Building 3D Generative Models from Minimal Data.

We propose a method for constructing generative models of 3D objects from a single 3D mesh and improving them through unsupervised low-shot learning from 2D images. Our method produces a 3D morphable model that represents shape and albedo in terms of Gaussian processes. Whereas previous approaches have typically built 3D morphable models from multiple high-quality 3D scans through principal component analysis, we build 3D morphable models from a single scan or template. As we demonstrate in the face domain, these models can be used to infer 3D reconstructions from 2D data (inverse graphics) or 3D data (registration). Specifically, we show that our approach can be used to perform face recognition using only a single 3D template (one scan total, not one per person). We extend our model to a preliminary unsupervised learning framework that enables the learning of the distribution of 3D faces using one 3D template and a small number of 2D images. Our approach is motivated as a potential model for the origins of face perception in human infants, who appear to start with an innate face template and subsequently develop a flexible system for perceiving the 3D structure of any novel face from experience with only 2D images of a relatively small number of familiar faces.