3D Mobile Devices Research Articles

Block DCT Based Optimization for Wireless SoftCast of Depth Map

With the recent emergence of naked-eye 3D mobile devices, the 3D video-based applications on the mobile terminal are showing great potential. However, the conventional transmission scheme based on separate source coding and channel coding may fail in the broadcast and mobile scenarios due to the drastic changes in wireless channel condition. Recently, an uncoded transmission scheme called SoftCast was introduced, which shows efficient performance for wireless 2D video transmission. However, it may not work well for 3D video wireless transmission, especially for the depth maps. Based on the piecewise smoothness of depth maps, we propose an uncoded wireless depth map transmission scheme, which incorporates three customized components of optimization into the SoftCast framework. First, compared with the frame-level discrete cosine transform (DCT) used in 2D video SoftCast, the mean removed block-based DCT is preferred as the decorrelation transform for depth maps to accommodate its piecewise smooth property, which also facilitates the following subsequent optimization. Second, view synthesis distortion-based chunk dropping is utilized for more efficient power allocation in the case of limited bandwidth, where the block DCT provides a better estimation for synthesis distortion. Third, inter-block DCT coefficients are rearranged to form chunks in terms of minimizing the intra-chunk variance, further enhancing the chunk-level power allocation efficiency. The experimental results demonstrate that the proposed scheme can provide superior performance than the state-of-the-art SoftCast methods while achieving graceful synthesis quality transition within a wide channel signal-to-noise ratio range.

Multi-scale CNNs for 3D model retrieval

Recent advancements in low-cost 3D sensors and mobile devices of virtual 3D models have additionally facilitated the accessibility of 3D data. 3D model retrieval is becoming an indispensable function for modern search engines. An effective retrieval model is at the core of computer vision. With the continuous improvement of 3D data, there are large number of methods to solve this problem. Existing works proposed numerous works to deal with feature extraction and object matching. Most of them are unable to fully exploit the information of 3D representations. To address this problem, we propose a novel multi-layer deep network in this paper. First, multiple rendered images are extracted from a 3D object, and combined into one representative view, which is the actual input of the network. Then, the novel multi-layer network structure is trained and tested on these representative views, generating the feature leaning model, which owns the local and global information of a 3D object. Finally, simple Euclidean metric is used to compute the similarity between two different 3D models to complete the retrieval problem. Extensive experiments and corresponding experimental results have demonstrated the superiority of our approach.

Efficient Multi-View 3D Video Multicast with Depth-Image-Based Rendering in LTE-Advanced Networks with Carrier Aggregation

With the recent emergence of naked-eye 3D mobile devices and various 3D-enabled laptops, service providers now afford the opportunity to provide mobile 3D video streaming in LTE-Advanced networks. Differing from traditional single-view 3D videos, multi-view 3D videos allow users to choose preferred view angles and thus are promising for new applications, such as free-viewpoint television (FTV). Nevertheless, enabling multi-view 3D video services may overwhelm the network resource when transmitting all views of every video. Fortunately, Depth-Image-Based Rendering (DIBR) allows each mobile client to synthesize the desired view from a nearby left view and right view, so that not all views of a video are necessarily transmitted. A new challenge with DIBR, however, is to carefully choose the transmitted views to limit the video distortion and minimize the bandwidth consumption. In this paper, therefore, we first formulate a new optimization problem, called View and MCS Selection (VMS) Problem, to minimize the bandwidth consumption for multi-view 3D video multicast in LTE networks. An algorithm, called View and MCS Aggregation (VMAG) is proposed to find the optimal solution to VMS. For Carrier Aggregation (CA) in LTE-Advanced networks, we formulate a new View, MCS and Carrier Selection (VMCS) Problem and prove that the problem is NP-Hard. We first design a dynamic programming algorithm, called the View Assignment with MCS and Carrier (VAMC) algorithm, to find the optimal solution for small instances. We then propose the View and MCS Aggregation with Carrier (VMAGC) algorithm based on VMAG to effectively find the near-optimal solution to VMCS. The simulation results show that bandwidth consumption can be effectively reduced by over 30 percent in VMS and VMCS.

Visual Comfort for Stereoscopic 3D by Using Motion Sensors on 3D Mobile Devices

Advanced 3D mobile devices attract a lot of attentions for 3D visualization nowadays. Stereoscopic images and video taken from the 3D mobile devices are uncomfortable for 3D viewing experiences due to the limited hardware for stereoscopic 3D stabilization. The existing stereoscopic 3D stabilization methods are computationally inefficient for the 3D mobile devices. In this article, we point out that this critical issue deteriorates the 3D viewing experiences on the 3D mobile devices. To improve visual comfort, we propose an efficient and effective algorithm to stabilize the stereoscopic images and video for the 3D mobile devices. To rectify the video jitter, we use the gyroscope and accelerometer embedded on the mobile devices to obtain the geometry information of the cameras. Using a different method than video-content-based motion estimation, our algorithm based on the gyroscope and acceleration data can achieve higher accuracy to effectively stabilize the video. Therefore, our approach is robust in video stabilization even under poor lighting and substantial foreground motion. Our algorithm outperforms previous approaches in not only smaller running time but also the better comfort of the stereoscopic 3D visualization for the 3D mobile devices.

An Adaptive Error Concealment Method Based on Fuzzy Reasoning for Multi-View Video Coding

This study proposes an adaptive error concealment selection process in order to improve the quality of reconstructed images which have low computation complexity for multi-view video coding. In this study, the proposed algorithm uses the motion vector variances and texture histogram bins in the blocks neighboring the damaged block in order to define two correlations; the degree of motion and the degree of texture similarity. To increase the accuracy of the suitable EC for the damaged macro-block, these two correlations are used to select a suitable error concealment method for reconstructing the damaged macro-block through fuzzy reasoning. The motion degree indicates the motion complexity for the damaged macro-block, and the texture similarity degree indicates the spatial continuity between the damaged macro-block and its neighboring block. The proposed adaptive error concealment is selected to reduce concealing time on homogenous damaged blocks and to improve the reconstructed image quality on non-homogenous damaged blocks. The experiment results show that the proposed algorithm can reduce the concealing time by at least 1.16 s and can improve the peak signal-to-noise (PSNR) by 0.13 dB–2.44 dB and 0.11 dB–3.52 dB in the main views and auxiliary views, respectively. This proposed algorithm is suitable for the multi-view video transmission on the consumer electronic fields such as 3D mobile devices, 3D television.

Three-dimensional display technologies.

The physical world around us is three-dimensional (3D), yet traditional display devices can show only two-dimensional (2D) flat images that lack depth (i.e., the third dimension) information. This fundamental restriction greatly limits our ability to perceive and to understand the complexity of real-world objects. Nearly 50% of the capability of the human brain is devoted to processing visual information [Human Anatomy & Physiology (Pearson, 2012)]. Flat images and 2D displays do not harness the brain's power effectively. With rapid advances in the electronics, optics, laser, and photonics fields, true 3D display technologies are making their way into the marketplace. 3D movies, 3D TV, 3D mobile devices, and 3D games have increasingly demanded true 3D display with no eyeglasses (autostereoscopic). Therefore, it would be very beneficial to readers of this journal to have a systematic review of state-of-the-art 3D display technologies.