Holoscopic 3D Imaging Research Articles

3D Holoscopic Image Compression Based on Gaussian Mixture Model

We introduce a Gaussian Mixture Model (GMM) framework for 3D holoscopic image compression in this paper. The elemental-images of the 3D holoscopic image are predicted using GMM and the parameters of GMM are estimated using the common Expectation-Maximization (EM) algorithm. GMM Model Optimization (GMO) is used in this framework to select the optimal number of distributions and avoid local optimum of EM at the same time. A three-dimensional distribution-rotation based decomposition is proposed to change covariance parameters to meaningful features and improve the coding efficiency. The features and the remaining parameters of the GMM are encoded using fixed-length bits. A feature-based dictionary is proposed in this framework to match the similar Gaussian distributions utilizing the similar GMM features. And the offsets of the matched distributions are recorded as motion vectors to replace the similar areas in the elemental-images of the 3D holoscopic image. The residual between the original image and the prediction is encoded using Screen Content Coding Extension of High Efficiency Video Coding (HEVC-SCC). Experimental results show that our method performs better than HEVC-SCC, two coding methods based on pseudo-sequences and a state-of-the-art content-based compression method with Gaussian process regression.

Holoscopic 3D Microgesture Recognition by Deep Neural Network Model Based on Viewpoint Images and Decision Fusion

Finger microgestures have been widely used in human computer interaction (HCI), particularly for interactive applications, such as virtual reality (VR) and augmented reality (AR) technologies, to provide immersive experience. However, traditional 2D image-based microgesture recognition suffers from low accuracy due to the limitations of 2D imaging sensors, which have no depth information. In this article, we proposed an innovative 3D microgesture recognition system based on a holoscopic 3D imaging sensor. Due to the lack of holoscopic 3D datasets, a comprehensive holoscopic 3D microgesture (HoMG) database is created and used to develop a robust 3D microgesture recognition method. Then, a fast algorithm is proposed to extract multiviewpoint images from one holoscopic image. Furthermore, we applied a CNN model with an attention-based residual block to each viewpoint image to improve the algorithm performance. Finally, bagging classification tree decision-level fusion is applied to combine the predictions. The experimental results demonstrate that the proposed method outperforms state-of-the-art methods and delivers a better accuracy than existing methods.

Low-delay single holoscopic 3D computer-generated image to multiview images

Due to the nature of holoscopic 3D (H3D) imaging technology, H3D cameras can capture more angular information than their conventional 2D counterparts. This is mainly attributed to the macrolens array which captures the 3D scene with slightly different viewing angles and generates holoscopic elemental images based on fly’s eyes imaging concept. However, this advantage comes at the cost of decreasing the spatial resolution in the reconstructed images. On the other hand, the consumer market is looking to find an efficient multiview capturing solution for the commercially available autostereoscopic displays. The autostereoscopic display provides multiple viewers with the ability to simultaneously enjoy a 3D viewing experience without the need for wearing 3D display glasses. This paper proposes a low-delay content adaptation framework for converting a single holoscopic 3D computer-generated image into multiple viewpoint images. Furthermore, it investigates the effects of varying interpolation step sizes on the converted multiview images using the nearest neighbour and bicubic sampling interpolation techniques. In addition, it evaluates the effects of changing the macrolens array size, using the proposed framework, on the perceived visual quality both objectively and subjectively. The experimental work is conducted on computer-generated H3D images with different macrolens sizes. The experimental results show that the proposed content adaptation framework can be used to capture multiple viewpoint images to be visualised on autostereoscopic displays.

Investigating 3D holoscopic visual content upsampling using super-resolution for cultural heritage digitization

Through this paper, we aim at investigating the impact of using deep learning-based technologies such as super-resolution on Holoscopic 3D (H3D) images. Holoscopic 3D imaging is a technology that aims at providing cost-effective alternatives for 3D content viewing and consumption without requiring a special headgear or posture. The technique is using a special lens array fitted to standard DSLR or mirrorless cameras to generate or capture 3D content. The output is a Holoscopic 3D image that can be displayed in lightfield displays or Multiview displays following a post-processing procedure. The main advantage of this technique is its cost-effectiveness in viewing and interacting with 3D content. However, one of its drawbacks is the low spatial density of the commercial cameras CMOS sensors and the lens induced imperfections. The latter can be fixed in software using some distortion correction techniques. However, the former is still challenging in terms of techniques that result in naturally looking output. Mitigating such issues with hardware will lead to higher costs and the technique loses its main advantage. Our approach consists of designing a framework that leverages software tools in order to upscale the output of H3D cameras whilst solving the low spatial density problem of H3D images. We also investigate the impact of deep learning-based video motion interpolation on the output quality of the cultural H3D imaging framework.

Scalable coding of 3D holoscopic image by using a sparse interlaced view image set and disparity map

3D holoscopic imaging, also known as integral imaging, light field imaging or plenoptic imaging, can provide natural and fatigue-free 3D visualization. Holoscopic contents captured by the plenoptic camera contain both spatial and angular information of a 3D scene. Therefore, view images with different perspectives can be rendered from the holoscopic contents. A coding scheme to compress the 3D holoscopic image by exploiting the high spatial correlation among the rendered view images will be advantageous. Therefore, in this paper, an efficient scalable coding scheme is proposed to compress the 3D holoscopic image by utilizing such high spatial correlation. We firstly re-arrange the holoscopic contents to form an interlaced view image. A sparse format is then proposed to express the interlaced view image. With the reconstructed image derived by disparity map based sifting and interpolation, the full interlaced view image is coded by using the reconstructed image as a reference frame. As an outcome of the representation, a scalable structure with three layers can be provided by the proposed scheme. Experimental results demonstrate that the 3D holoscopic image can be compressed efficiently with over 44 percent bit rate reduction compared with HEVC. Meanwhile, the proposed scheme can also surpass several other prediction schemes in this field.

3D holoscopic image coding scheme using HEVC with Gaussian process regression

3D holoscopic image, also known as integral imaging, light field imaging and plenoptic imaging, can provide a natural and fatigue-free 3D visualization. However, a large amount of data is required to represent the 3D holoscopic content. Therefore, efficient coding schemes for such particular type of image are needed. In this paper, we propose a Gaussian process regression based prediction scheme to compress the 3D holoscopic image. In the proposed scheme, the coding block and its prediction supports are modeled as a Gaussian process (GP) and Gaussian process regression (GPR) is used to obtain a better prediction of the coding block. Limited searching windows in horizontal and vertical directions are used to obtain the prediction supports, and a filtration method is designed to judge the reliability of the obtained prediction supports. Moreover, in order to alleviate the high complexity caused by GPR, a sparsification method is also put forward. Experimental results demonstrate the advantage of the proposed scheme for 3D holoscopic image coding in terms of different quality metrics as well as the visual quality of the views rendered from decompressed 3D holoscopic content, compared to the HEVC intra-prediction method and several other prediction methods in this field.

3D Depth Measurement for Holoscopic 3D Imaging System

Holoscopic 3D imaging is a true 3D imaging system mimics fly’s eye technique to acquire a true 3D optical model of a real scene. To reconstruct the 3D image computationally, an efficient implementation of an Auto-Feature-Edge (AFE) descriptor algorithm is required that provides an individual feature detector for integration of 3D information to locate objects in the scene. The AFE descriptor plays a key role in simplifying the detection of both edge-based and region-based objects. The detector is based on a Multi-Quantize Adaptive Local Histogram Analysis (MQALHA) algorithm. This is distinctive for each Feature-Edge (FE) block i.e. the large contrast changes (gradients) in FE are easier to localise. The novelty of this work lies in generating a free-noise 3D-Map (3DM) according to a correlation analysis of region contours. This automatically combines the exploitation of the available depth estimation technique with edge-based feature shape recognition technique. The application area consists of two varied domains, which prove the efficiency and robustness of the approach: a) extracting a set of setting feature-edges, for both tracking and mapping process for 3D depthmap estimation, and b) separation and recognition of focus objects in the scene. Experimental results show that the proposed 3DM technique is performed efficiently compared to the state-of-the-art algorithms.

3D-holoscopic imaging: a new dimension to enhance imaging in minimally invasive therapy in urologic oncology.

Existing imaging modalities of urologic pathology are limited by three-dimensional (3D) representation on a two-dimensional screen. We present 3D-holoscopic imaging as a novel method of representing Digital Imaging and Communications in Medicine data images taken from CT and MRI to produce 3D-holographic representations of anatomy without special eyewear in natural light. 3D-holoscopic technology produces images that are true optical models. This technology is based on physical principles with duplication of light fields. The 3D content is captured in real time with the content viewed by multiple viewers independently of their position, without 3D eyewear. We display 3D-holoscopic anatomy relevant to minimally invasive urologic surgery without the need for 3D eyewear. The results have demonstrated that medical 3D-holoscopic content can be displayed on commercially available multiview auto-stereoscopic display. The next step is validation studies comparing 3D-Holoscopic imaging with conventional imaging.