3D Depth Sensor Research Articles

Infrared-based temperature measurement in three dimensions

A highly accurate and low-cost mobile platform for multiple object temperature measurement in 3D coordinate positions is introduced. The key idea relies on not only a combination of a 3D optical depth sensor and the 2D infrared thermal imager but also a determined distant-dependent compensation model. As a result, a cost-effective yet wider (>100.0∘C) temperature measurement range under the fluctuation of working distant shift is realized with a low standard deviation of 0.16°C at the working distance of 1.0 m. Accurate temperature measurement at different positions of objects along the suitable working distance is also demonstrated.

High-statistics pedestrian dynamics on stairways and their probabilistic fundamental diagrams

Staircases play an essential role in crowd dynamics, allowing pedestrians to flow across large multi-level public facilities such as transportation hubs, shopping malls, and office buildings. Achieving a robust quantitative understanding of pedestrian behavior in these facilities is a key societal necessity. What makes this an outstanding scientific challenge is the extreme randomness intrinsic to pedestrian behavior. Any quantitative understanding necessarily needs to be probabilistic, including average dynamics and fluctuations. To this purpose, large-scale, real-life trajectory datasets are paramount.In this work, we analyze the data from an unprecedentedly high statistics year-long pedestrian tracking campaign, in which we anonymously collected millions of trajectories of pedestrians ascending and descending stairs within Eindhoven Central train station (The Netherlands). This has been possible thanks to a state-of-the-art, faster than real-time, computer vision approach hinged on 3D depth imaging, sensor fusion, and YOLOv7-based depth localization. We consider both free-stream conditions, i.e. pedestrians walking in undisturbed, and trafficked conditions, unidirectional/bidirectional flows. We report on Eulerian fields (density, velocity and acceleration), showing how the walking dynamics changes when transitioning from stairs to landing. We then investigate the (mutual) positions of pedestrian as density changes, considering the crowd as a “compressible” physical medium. In particular, we show how pedestrians willingly opt to occupy fewer space than available, accepting a certain degree of compressibility. This is a non-trivial physical feature of pedestrian dynamics and we introduce a novel way to quantify this effect. As density increases, pedestrians strive to keep a minimum distance d≈0.6m (two treads of the staircase) from the person in front of them. Finally, we establish first-of-kind fully resolved probabilistic fundamental diagrams, where we model the pedestrian walking velocity as a mixture of a slow and fast-paced component (both in non-negligible percentages and with density-dependent characteristic fluctuations). Notably, averages and modes of velocity distribution turn out to be substantially different. Our results, of which we include probabilistic parametrizations based on few variables, are key towards improved facility design and realistic simulation of pedestrians on staircases.

151 Depth Video Data-Enabled Predictions of Longitudinal Dairy Cow Body Weight Using Thresholding and Mask R-Cnn Algorithms

Abstract Cow body weight monitoring is critical in farm management because it is associated with the growth, nutritional status, and health of dairy cows. Regularly collecting body weight records often requires human labor or expensive equipment. Recently, a low-cost 3D depth sensor camera-enabled computer vision analysis has been proposed to address the body weight prediction challenges. Cow body weight is a repeated trait; however, past body weight prediction research only used data collected at a single time point. Furthermore, the utility of deep learning on body weight predictions remains unanswered. Therefore, the objectives of this study were to predict cow body weight from repeatedly measured video data and compare the performance of thresholding and Mask R-CNN deep learning approaches. We set up an Intel Realsense D435 camera at the Virginia Tech Kentland farm to collect top-view videos for 10 Holstein and 2 Jersey cows for 28 days, twice per day. Real body weight records were also collected by a walk-over-weighing system simultaneously. A total of 40,405 depth images and depth map files were obtained. We explored three approaches to segment the cow body from the background, including single thresholding, adaptive thresholding, and Mask R-CNN. Four image-derived biometric measurements, length, width, height, and volume, were estimated from segmented images and fitted in ordinary least squares. Two cross-validation designs, forecasting and leave-several-cow-out, were used to evaluate prediction performance. The correlation between image-derived biometric measurements and body weight ranged from 0.84 to 0.95. On average, the Mask-RCNN approach resulted in the best prediction coefficient of determination and mean absolute percentage error of 0.96 and 3.54% in the forecasting cross-validation, respectively. The predictive performance of the single and adaptive thresholding approaches was similar. The Mask-RCNN approach was also the best in the leave-several-cow-out cross-validation, followed by adaptive and single thresholding. The prediction coefficients of determination of Mask-RCNN for leave-one-cow-out, leave-two-cow-out, leave-three-cow-out, leave-four-cow-out were 0.05, 0.69, 0.85, and 0.90, respectively, while mean absolute percentage errors were 4.51%, 4.51%, and 4.59%, 4.70% respectively. Our results suggest predicting cow body weight from depth video data using deep learning is feasible.

Application of mobile systems with various software products to measuring animal exterior parameters

Relevance. Currently, contactless remote measurements of the parameters of the exterior of animals are of considerable interest in the field of breeding and genetics of cattle. They can reduce the risk of dangerous reactions of animals to stress and significantly reduce the time needed to get measurements. We offer a way to obtain information about the size of an animal in real time by obtaining an image using the Structure Sensor 3D depth sensor.Methods. Scientific experimental studies were conducted in 2018–2022 on Black-and-White cows in breeding enterprises of the Udmurt Republic (Russian Federation). The volume of the sample population of animals was about 2000 cows. The animals were evaluated from the 90th to the 150th day of lactation. We have selected the following measurements: height at the withers, chest depth, chest width, hips width, straight length of the hip area, straight length of the trunk, circumference of the cannon bone, length of the rump bone, depth in the lower back, width. These parameters most accurately characterize the dimensions (frame) and hip area of the animal.Results. The analysis of the obtained results indicates the validity of using the Structure Sensor 3D depth sensor as a priority method for obtaining animal measurements. A comparative analysis of contactless methods for obtaining measurements of cows shows that in approximately the same time, it is possible to obtain numerical values of up to 10 exterior parameters. Calibration is performed easily and quickly within 10–15 minutes. The use of the Structure Sensor 3D depth sensor eliminates inaccuracies of obtaining a silhouette, that arise due to the continuous movement of animals, and does not require the creation of a new technique for obtaining measurements of the body of animals that differ in age, size and live weight. Using Structure Sensor 3D, examples were obtained with an accuracy of 1 mm without causing stress in animals.

Prediction of Cobb Angle Using Deep Learning Algorithm with Three-Dimensional Depth Sensor Considering the Influence of Garment in Idiopathic Scoliosis.

Adolescent idiopathic scoliosis (AIS) is the most common pediatric spinal deformity. Early detection of deformity and timely intervention, such as brace treatment, can help inhibit progressive changes. A three-dimensional (3D) depth-sensor imaging system with a convolutional neural network was previously developed to predict the Cobb angle. The purpose of the present study was to (1) evaluate the performance of the deep learning algorithm (DLA) in predicting the Cobb angle and (2) assess the predictive ability depending on the presence or absence of clothing in a prospective analysis. We included 100 subjects with suspected AIS. The correlation coefficient between the actual and predicted Cobb angles was 0.87, and the mean absolute error and root mean square error were 4.7° and 6.0°, respectively, for Adam's forward bending without underwear. There were no significant differences in the correlation coefficients between the groups with and without underwear in the forward-bending posture. The performance of the DLA with a 3D depth sensor was validated using an independent external validation dataset. Because the psychological burden of children and adolescents on naked body imaging is an unignorable problem, scoliosis examination with underwear is a valuable alternative in clinics or schools.

An Embedded Framework for Fully Autonomous Object Manipulation in Robotic-Empowered Assisted Living

Most of the humanoid social robots currently diffused are designed only for verbal and animated interactions with users, and despite being equipped with two upper arms for interactive animation, they lack object manipulation capabilities. In this paper, we propose the MONOCULAR (eMbeddable autONomous ObjeCt manipULAtion Routines) framework, which implements a set of routines to add manipulation functionalities to social robots by exploiting the functional data fusion of two RGB cameras and a 3D depth sensor placed in the head frame. The framework is designed to: (i) localize specific objects to be manipulated via RGB cameras; (ii) define the characteristics of the shelf on which they are placed; and (iii) autonomously adapt approach and manipulation routines to avoid collisions and maximize grabbing accuracy. To localize the item on the shelf, MONOCULAR exploits an embeddable version of the You Only Look Once (YOLO) object detector. The RGB camera outcomes are also used to estimate the height of the shelf using an edge-detecting algorithm. Based on the item's position and the estimated shelf height, MONOCULAR is designed to select between two possible routines that dynamically optimize the approach and object manipulation parameters according to the real-time analysis of RGB and 3D sensor frames. These two routines are optimized for a central or lateral approach to objects on a shelf. The MONOCULAR procedures are designed to be fully automatic, intrinsically protecting sensitive users' data and stored home or hospital maps. MONOCULAR was optimized for Pepper by SoftBank Robotics. To characterize the proposed system, a case study in which Pepper is used as a drug delivery operator is proposed. The case study is divided into: (i) pharmaceutical package search; (ii) object approach and manipulation; and (iii) delivery operations. Experimental data showed that object manipulation routines for laterally placed objects achieves a best grabbing success rate of 96%, while the routine for centrally placed objects can reach 97% for a wide range of different shelf heights. Finally, a proof of concept is proposed here to demonstrate the applicability of the MONOCULAR framework in a real-life scenario.

Enabling human–machine collaboration in infrastructure inspections through mixed reality

In this study, we propose a novel end-to-end system called Human–Machine Collaborative Inspection (HMCI) to enable collaboration between inspectors with Mixed Reality (MR) headsets and a robotic data collection platform (robot) for structural inspections. We utilize the MR headset’s holographic display and precise head tracking to allow inspectors to visualize and localize information (e.g., structural defect) on the real scenes, which are gathered by the robot and processed by an offsite computational server. The primary use case of HMCI is to enable the inspector to visualize, supervise, and improve results produced by automated defect detection algorithms in near real-time. The workflow in HMCI starts with collecting images and depth data to generate 3D maps of the site from the robot. A technique called single-shot localization is developed to create visual anchors for real-time spatial alignment between the robot and the MR headset. The 3D map and images are then sent to the computational server for analysis to detect defects and their locations. Then, the information is received by the MR headset and overlaid on the actual scenes to visualize it with spatial context. An experimental study is conducted in a lab environment to demonstrate HMCI using Microsoft HoloLens 2 (HL2) as the MR headset and Turtlebot2 as the robot. We start with the reconstruction of a 3D environment using a 3D depth sensor (Azure Kinect) on Turtlebot2 and visually detect fiducial markers as regions-of-interest (replicating structural damage) along a predefined inspection path. Then, regions-of-interest are successfully anchored to the real scene and visualized through HL2. To our knowledge, HMCI is one of the first human–machine collaborative systems that can integrate robots and inspectors with the MR headset, which has been developed, tested, and presented for structural inspection.

7. An algorithm for using deep learning convolutional neural networks with 3-dimensional depth sensor imaging in scoliosis detection

<h3>BACKGROUND CONTEXT</h3> Timely intervention in growing individuals, such as brace treatment, relies on early detection of adolescent idiopathic scoliosis (AIS). To this end, several screening methods have been implemented. However, these methods have limitations in predicting the Cobb angle. <h3>PURPOSE</h3> This study aimed to evaluate the performance of a 3-dimensional depth sensor imaging system with a deep learning algorithm (DLA), in predicting the Cobb angle in AIS. <h3>STUDY DESIGN/SETTING</h3> Retrospective analysis of prospectively collected, consecutive, nonrandomized series of patients at 5 scoliosis centers in Japan. <h3>PATIENT SAMPLE</h3> One hundred sixty human subjects suspected to have AIS were included. <h3>OUTCOME MEASURES</h3> Patient demographics, radiographic measurements, and predicted Cobb angle derived from the DLA were the outcome measures for this study. <h3>METHODS</h3> One hundred sixty data files were shuffled into 5 datasets with 32 data files at random (dataset 1, 2, 3, 4 and 5) and 5-fold cross validation was performed. The relationships between the actual and predicted Cobb angles were calculated using Pearson's correlation coefficient analyses. The prediction performances of the network models were evaluated using mean absolute error (MAE) and root mean square error (RMSE) between the actual and predicted Cobb angles. The shuffling into five datasets and five-fold cross validation was conducted ten times. There were no study-specific biases related to conflicts of interest. <h3>RESULTS</h3> The mean age was 14.7 ± 2.4 years, and the average Cobb angle was 30° (range: 0˚ to 64˚). The range of correlation coefficients was 0.87 to 0.89 in the five-fold cross validation with 10 repeats. The correlation between the actual Cobb angle and the mean predicted Cobb angle with 10 repeats was 0.91. The range of MAEs was 4.4˚ to 4.7˚in the 5-fold cross validation with 10 repeats. The range of RMSEs was 5.8˚ to 6.3˚. The MAE and RMSE between the actual Cobb angle and the mean predicted Cobb angle with 10 repeats was 4.0˚ and 5.4˚ respectively. At a Cobb angle of 10˚, which confirms a diagnosis of scoliosis, the mean predicted Cobb angle with 10 repeats showed a sensitivity of 0.99, a specificity of 0.42, PPV of 0.95, NPV of 0.71, accuracy of 0.94, PLR of 1.69, and an NLR of 0.03. <h3>CONCLUSIONS</h3> The 3D depth sensor imaging system with its newly innovated CNN for regression is objective and has significant ability to predict the Cobb angle in children and adolescents. This system is expected to be used for screening scoliosis in clinics or physical examination at schools. In addition, this is also a possible surrogate for radiographs to monitor curve progression, preventing unnecessary X-rays for mild case of scoliosis. <h3>FDA DEVICE/DRUG STATUS</h3> This abstract does not discuss or include any applicable devices or drugs.

Reconocimiento gestual con Kinect para detectar comportamientos inseguros en conductores

Los comportamientos inseguros por parte del conductor de un vehículo, como el uso de undispositivo celular durante la conducción, son unas de las principales causas de accidentesde tránsito alrededor del mundo. Esta práctica es difícil de regular y controlar, mas con el usode un sistema automatizado de detección de gestos que indique un comportamiento peligroso,es posible. El uso de cámaras con sensores de profundidad en 3D permite analizar objetos ypersonas en tiempo real con una alta precisión, identificando rasgos y figuras. Estos sensoresutilizan una tecnología denominada “point cloud”, la cual emite un arreglo de luz láser y mideel tiempo de regreso hacia el sensor, determinando la profundidad de cada uno de los puntos.Esta tecnología permitió realizar una aplicación de reconocimiento de gestos que es capazde detectar de manera automática la acción que indica que el conductor está utilizando undispositivo celular durante el tiempo de conducción. Utilizando algoritmos de substracciónde fondo, análisis de rasgos y análisis de histogramas se consiguió realizar inicialmente unasimulación funcional en el programa V-Rep que es capaz de detectar el gesto que indicaeste comportamiento. Posteriormente, se implementó un prototipo utilizando imágenes realestomadas con un Kinect que fueron procesadas en el programa Octave con los mismos algoritmosutilizados en la simulación, para así probar la efectividad de la aplicación desarrollada.

Collision Avoidance Interaction Between Human and a Hidden Robot Based on Kinect and Robot Data Fusion

Human-Robot Interaction (HRI) is a largely addressed subject today. In order to ensure co-existence and space sharing between human and robot, collision avoidance is one of the main strategies for interaction between them without contact. It is thus usual to use a 3D depth camera sensor (Microsoft Kinect V2) which may involve issues related to occluded robot in the camera view. While several works overcame this issue by applying infinite depth principle or increasing the number of cameras, in the current work we developed and applied an original new approach that combines data of one 3D depth sensor (Kinect) and proprioceptive robot sensors. This method uses the principle of limited safety contour around the obstacle to dynamically estimate the robot-obstacle distance, and then generate the repulsive force that controls the robot. For validation, our approach is applied in real time to avoid collisions between dynamical obstacles (humans or objects) and the end-effector of a real 7-dof Kuka LBR iiwa collaborative robot. Our method is experimentally compared with existing methods based on infinite depth principle when the robot is hidden by the obstacle with respect to the camera view. Results showed smoother behavior and more stability of the robot using our method. Extensive experiments of our method, using several strategies based on distancing and its combination with dodging were done. Results have shown a reactive and efficient collision avoidance, by ensuring a minimum obstacle-robot distance (of $\approx \text{240 mm}$ ), even when the robot is in an occluded zone in the Kinect camera view.

Predictive Generalized Graph Fourier Transform for Attribute Compression of Dynamic Point Clouds

As 3D scanning devices and depth sensors advance, dynamic point clouds have attracted increasing attention as a format for 3D objects in motion, with applications in various fields such as immersive telepresence, navigation for autonomous driving and gaming. Nevertheless, the tremendous amount of data in dynamic point clouds significantly burden transmission and storage. To this end, we propose a complete compression framework for attributes of 3D dynamic point clouds, focusing on optimal inter-coding. Firstly, we derive the optimal inter-prediction and predictive transform coding assuming the Gaussian Markov Random Field model with respect to a spatio-temporal graph underlying the attributes of dynamic point clouds. The optimal predictive transform proves to be the Generalized Graph Fourier Transform in terms of spatio-temporal decorrelation. Secondly, we propose refined motion estimation via efficient registration prior to inter-prediction, which searches the temporal correspondence between adjacent frames of irregular point clouds. Finally, we present a complete framework based on the optimal inter-coding and our previously proposed intra-coding, where we determine the optimal coding mode from rate-distortion optimization with the proposed offline-trained λ-Q model. Experimental results show that we achieve around 17% bit rate reduction on average over competitive dynamic point cloud compression methods.

Fuzzy-Based Obstacle Avoidance Control of Mobile Robots Using 3D Camera

This research develops a novel method of generating the velocities of the right and left motors of the two-wheeled mobile robotbase on the distances of the detected objects according to the image processing technique using the Fuzzy Logic Controller forobstacle avoiding system. The aim of this research is to build a controller for avoiding some obstacles with distance informationobtained from the 3D camera or depth sensor which is mounted on a mobile robot. This sensor is used in this study due to itsdouble integrated sensor, namely vision and distance. The fuzzy controller is designed for obstacles avoiding the system to control the mobile robot. The position of the obstacle is generated by using a contour operation on a captured image, together with a Canny edge detector and morphological operation. The distance between the robot and the obstacle is obtained from a depth frame. According to these data, the velocities of the left and right motors of the two-wheel-drive mobile robot are generated from the fuzzy controller. The effectiveness of the proposed approach was verified through several experiments

Tumble Avoidance System for Rescue Robot by Estimating the Contact Points using a 3D Depth Sensor

Rescue robots are expected to perform works in hazardous areas. However, when the robot runs on a rough terrain, fatal rollover falling sometimes occurs. Therefore, we propose a tumble-risk-assessment system, which calculates a normalized energy stability margin by using a 3D depth sensor. Moreover, a control system to avoid the tumble situation by moving the sub-crawlers automatically is proposed. We implemented the proposed systems to a real robot, carried out experiments, and confirmed the effectiveness of the proposed method.

A design on recommendations of sensor development platforms with different sensor modalities for making gesture biometrics-based service applications of the specific group

Gesture biometrics sensors will be extremely helpful for constructing a gesture-based application system. Compared with traditional acoustic sensor-based speech recognition applications and RGB image sensor-based surveillance applications, proper utilizations of the gesture biometrics sensor will absolutely be able to create an innovative and practical application and speed up the maturity of gesture recognition. Currently, three well-known gesture biometrics sensor platforms are Kinect, Leap Motion, and Myo. Kinect and Leap motion sensor devices belong to the categorization of 3D image sensors, containing both RGB and depth sensors. Myo armband is categorized into the type of wearable sensor devices. Nowadays, related studies on application system establishments using Kinect, Leap Motion, or Kinect have been frequently seen in the recent years. However, most of all these related gesture-based application systems are designed for end users. Extremely few investigations using these popular gesture biometrics sensors are done specifically for the system developer of a gesture-based application system. In fact, a correct and proper adoption of the gesture biometric sensor platform will speed up the development of the desired system and increase the practicality of the designed system. In this paper, a recommendation scheme of Kinect, Leap Motion, and Myo sensor platforms is presented for supporting developments of a mid-air gesture application. Compared with the conventional application system design procedure, the presented sensor recommendation scheme incorporated between two stages of functional operation and system design specifications will have a great help to the system developer to establish his desired system. In addition, a case study on system developments of efficient web-based streaming media control by disabled persons using the presented sensor platform recommendation scheme is also reported in this paper.

Local Frequency Interpretation and Non-Local Self-Similarity on Graph for Point Cloud Inpainting.

As 3D scanning devices and depth sensors mature, point clouds have attracted increasing attention as a format for 3D object representation, with applications in various fields such as tele-presence, navigation and heritage reconstruction. However, point clouds usually exhibit holes of missing data, mainly due to the limitation of acquisition techniques and complicated structure. Further, point clouds are defined on irregular non- Euclidean domains, which is challenging to address especially with conventional signal processing tools. Hence, leveraging on recent advances in graph signal processing, we propose an efficient point cloud inpainting method, exploiting both the local smoothness and the non-local self-similarity in point clouds. Specifically, we first propose a frequency interpretation in graph nodal domain, based on which we derive the smoothing and denoising properties of a graph-signal smoothness prior in order to describe the local smoothness of point clouds. Secondly, we explore the characteristics of non-local self-similarity, by globally searching for the most similar area to the missing region. The similarity metric between two areas is defined based on the direct component and the anisotropic graph total variation of normals in each area. Finally, we formulate the hole-filling step as an optimization problem based on the selected most similar area and regularized by the graph-signal smoothness prior. Besides, we propose voxelization and automatic hole detection methods for the point cloud prior to inpainting. Experimental results show that the proposed approach outperforms four competing methods significantly, both in objective and subjective quality.

Tumble avoidance system for rescue robot by estimating the contact points using a 3D depth sensor

Tumble avoidance system for rescue robot by estimating the contact points using a 3D depth sensor

Extracting Multiple Planar Surfaces Effectively and Efficiently Based on 3D Depth Sensors

Reliable and efficient planar surface extraction based on the 3D depth sensors is a crucial component in mobile robotics. However, extracting planes efficiently remains challenging, especially when small planar structures are urged to be perceived. This paper proposes a new method that can extract the multi-scale planar surfaces efficiently. The depth image is dynamically divided into rectangle regions, where points in each region lie on a common plane. Then, the planar primitives are generated by clustering these regions into some distinct groups according to their plane parameters. Finally, the pixel-wise segmentation results are achieved by growing each distinct group. The first novelty is the dynamic region size adjusting algorithm that can reduce the number of regions to be clustered and improve the plane fitting accuracy. The second one is the region clustering algorithm, whose worst-case time complexity is guaranteed to be log-linear. Comprehensive experiments show that our algorithm outperforms the state-of-the-art methods in quality and runs an order of magnitude faster.

2 3D VIEW: Designing of a Deception from Distorted View-dependent Images and Explaining interaction with virtual World

This paper presents an intuitive and interactive computer simulated augmented reality interface that gives the illusion of a 3D immersive environment. The projector displays a rendered virtual scene on a flat 2D surface (floor or table) based on the user’s viewpoint to create a head coupled perspective. The projected image is view-dependent which changes and deforms relative to user’s position in space. The nature of perspective projection is distorted and anamorphic such that the deformations in the image give an illusion of a virtual three dimensional holographic scene in which the objects are popping out or floating above the projection plane like real 3D objects. Also, the user can manipulate and interact with 3D objects in a virtual environment by controlling the position and orientation of 3D models, interacting with GUI incorporated in virtual scene and can view, move, manipulate and observe the details of objects from any angle naturally by using his hands. The head and hand tracking is achieved by a low cost 3D depth sensor ‘Kinect’. We describe the implementation of the system in OpenGL and Unity3D game engine. Stereoscopic 3D along with other enhancements are also introduced which further improves the 3D perception. The approach does not require head mounted displays or expensive 3D hologram projectors as it is based on perspective projection technique. Our experiments show the potential of the system providing users a powerful, realistic illusion of 3D.

3D VIEW: Designing of a Deception from Distorted View-dependent Images and Explaining interaction with virtual World

This paper presents an intuitive and interactive computer simulated augmented reality interface that gives the illusion of a 3D immersive environment. The projector displays a rendered virtual scene on a flat 2D surface (floor or table) based on the user’s viewpoint to create a head coupled perspective. The projected image is view-dependent which changes and deforms relative to user’s position in space. The nature of perspective projection is distorted and anamorphic such that the deformations in the image give an illusion of a virtual three dimensional holographic scene in which the objects are popping out or floating above the projection plane like real 3D objects. Also, the user can manipulate and interact with 3D objects in a virtual environment by controlling the position and orientation of 3D models, interacting with GUI incorporated in virtual scene and can view, move, manipulate and observe the details of objects from any angle naturally by using his hands. The head and hand tracking is achieved by a low cost 3D depth sensor ‘Kinect’. We describe the implementation of the system in OpenGL and Unity3D game engine. Stereoscopic 3D along with other enhancements are also introduced which further improves the 3D perception. The approach does not require head mounted displays or expensive 3D hologram projectors as it is based on perspective projection technique. Our experiments show the potential of the system providing users a powerful, realistic illusion of 3D.

3D VIEW: Designing of a Deception from Distorted View-dependent Images and Explaining interaction with virtual World

This paper presents an intuitive and interactive computer simulated augmented reality interface that gives the illusion of a 3D immersive environment. The projector displays a rendered virtual scene on a flat 2D surface (floor or table) based on the user’s viewpoint to create a head coupled perspective. The projected image is view-dependent which changes and deforms relative to user’s position in space. The nature of perspective projection is distorted and anamorphic such that the deformations in the image give an illusion of a virtual three dimensional holographic scene in which the objects are popping out or floating above the projection plane like real 3D objects. Also, the user can manipulate and interact with 3D objects in a virtual environment by controlling the position and orientation of 3D models, interacting with GUI incorporated in virtual scene and can view, move, manipulate and observe the details of objects from any angle naturally by using his hands. The head and hand tracking is achieved by a low cost 3D depth sensor ‘Kinect’. We describe the implementation of the system in OpenGL and Unity3D game engine. Stereoscopic 3D along with other enhancements are also introduced which further improves the 3D perception. The approach does not require head mounted displays or expensive 3D hologram projectors as it is based on perspective projection technique. Our experiments show the potential of the system providing users a powerful, realistic illusion of 3D.