Three-dimensional Coding Research Articles

Compressive spectral testbed imaging system based on thin-film color-patterned filter arrays.

Compressive spectral imaging systems can reliably capture multispectral data using far fewer measurements than traditional scanning techniques. In this paper, a thin-film patterned filter array-based compressive spectral imager is demonstrated, including its optical design and implementation. The use of a patterned filter array entails a single-step three-dimensional spatial-spectral coding on the input data cube, which provides higher flexibility on the selection of voxels being multiplexed on the sensor. The patterned filter array is designed and fabricated with micrometer pitch size thin films, referred to as pixelated filters, with three different wavelengths. The performance of the system is evaluated in terms of references measured by a commercially available spectrometer and the visual quality of the reconstructed images. Different distributions of the pixelated filters, including random and optimized structures, are explored.

Reusable HEVC Design in 3D-HEVC

This paper proposes a reusable design for the merging process used in three-dimensional High Efficiency Video Coding (3D-HEVC), which can significantly reduce the implementation complexity by eliminating duplicated module redundancies. The majority of inter-prediction coding tools used in 3D-HEVC are utilized through a merge mode, whose extended merging process is based on built-in integration to completely wrap around the HEVC merging process. Consequently, the implementation complexity is unavoidably very high. To facilitate easy market implementation, the design of a legacy codec should be reused in an extended codec if possible. The proposed 3D-HEVC merging process is divided into the base merging process of reusing HEVC modules and reprocessing process of refining the existing processes that have been newly introduced or modified for 3D-HEVC. To create a reusable design, the causal and mutual dependencies between the newly added modules for 3D-HEVC and the reused HEVC modules are eliminated, and the ineffective methods are simplified. In an application of the proposed reusable design, the duplicated reimplementation of HEVC modules, which account for of the 3D-HEVC merging process, can be eliminated while maintaining the same coding efficiency. The proposed method has been adopted as a normative coding tool in the 3D-HEVC international standard.

Research of XML Structural Join Based on the Cone Three-dimensional Coding

Research of XML Structural Join Based on the Cone Three-dimensional Coding

High performance encryption tools of visual information based on cellular automata

This article describes a method of encryption of visual information, based on the use of cellular automata. This method allows you to solve problems with noise errors in deciphering information, low-speed, low resistance to cracking, as well as move away from the use of generators of noise signals in the known methods of encryption of visual information. Analyzed modern encryption methods of visual information, as well as problems encountered in their implementation and use in the communication channels. The article also presents the structure of the device that implements - encryption is the visual information based on cellular automata, programs, texts, diagrams and results of the encryption method, the basic characteristics of the used field-programmable gate array. The essence of the method is to encrypt the visual information by overlaying additional transformations besides the usual encryption. The first additional conversion is carried out by the chosen method of encoding and digitizing images. The second transformation is to select the sequence of bit layers and their principles scan each block. Data conversions in the form of numerical values are used as additional fields to the key. In addition, the key is not given as a ready-bit sequence, but as the operation code, the units form a key sequence. Options propagation path of the excitation signal and a three-dimensional map of a cellular automaton states belong to the key sequence. The formation of pseudo-random key range increases reliability and protection to burglary resistance. Due to the three-dimensional image coding, the use of technologies of programmable integrated circuits and cellular automata increases speed encryption. The method allows you to create a key range implicitly, that reduces the probability of selecting an opponent. Encryption tools are implemented on cheap field-programmable gate array with high performance in speed, allowing you to encrypt visual information in real time during its transfer via communication channels.

Fast 3D-HEVC encoder algorithm for multiview video plus depth coding

The three-dimensional high efficiency video coding (3D-HEVC) is an extension of the HEVC standard for the compression of the multiview video plus depth (MVD) format. In 3D-HEVC, additional coding tools have been added to HEVC for improving the coding efficiency of the dependent video views and depth maps. Those tools achieve the highest possible coding efficiency, but also bring extremely high computational complexity which limits 3D-HEVC from real-time applications. In this paper, we propose a fast 3D-HEVC encoder algorithm based on the texture video and depth map correlation to reduce MVD coding computational complexity. Since the video texture and depth map represent the same scene at the same time instant (they have the same motion characteristics), it is not efficient to use all the prediction modes in 3D-HEVC mode decision procedure. The basic idea of the algorithm is to utilize the texture video and depth map property of coding information to predict the current CU prediction mode and early skip unnecessary variable-size mode decision. Experimental results demonstrate that the proposed fast 3D-HEVC encoder algorithm can save 75% computational complexity on average, while maintaining almost the same rate-distortion (RD) performance as the original 3D-HEVC encoder.

Efficient wedgelet pattern decision for depth modeling modes in three-dimensional high-efficiency video coding

The three-dimensional (3-D) video extension of high-efficiency video coding is an emerging coding standard for multiple-view-plus-depth that allows view synthesis for multiple displays with depth information. In order to avoid mixing between the foreground and background, the depth discontinuities defined at the object boundary should be retained. To solve this issue, a depth intramode, i.e., the depth-modeling mode (DMM), is introduced in 3-D-high-efficiency video coding as an edge predictor. The test model HTM 8.1 includes DMM1 and DMM3. However, the mode-decision strategy of DMM increases the complexity drastically. Therefore, we propose a fast DMM1 decision algorithm that estimates sharp edges by a subregional search method. The optimal wedgelet pattern of DMM1 is then searched only in the most probable region. Additionally, another fast method is raised to skip DMM3 when mismatch occurs between the depth prediction unit (PU) and its colocated texture PU. Simulation results show that the proposed algorithm has slightly better performance in terms of complexity reduction compared with the wedgelet-pattern-reducing algorithm from the literature while better maintaining the coding performance. In addition, the proposed algorithm has a performance similar to that of an existing DMM-skipping algorithm. Moreover, it could be integrated with that category of algorithms for additional time savings.

Hole-filling map-based coding unit size decision for dependent views in three-dimensional high-efficiency video coding

The three-dimensional high-efficiency video coding (3-D-HEVC) is an emerging compression standard for multiview video plus depth data. In addition to the quad-tree coding structure inherited from HEVC, some tools are integrated, which significantly improve the coding efficiency but also result in remarkably high computational complexity. We propose a fast coding unit (CU) size decision algorithm for both depth and texture components in dependent views, where hole-filling maps created through view synthesis are utilized. First, after coding the base view, warp it onto each dependent view via depth image based rendering, during which hole-filling maps are generated. Then for depth in dependent views, CU splitting can be early terminated considering the disoccluded information from hole-filling maps; for texture in dependent views, combining the disoccluded information and the interview correlations, the CU partitioning process can also be accelerated. Experimental results show that the proposed algorithm can achieve on average 54.3% time reduction, with a negligible Bjontegaard delta bitrate increase of 0.15% on synthesized views, and a 0.05% increase on all the coded plus synthesized views compared with the original encoding scheme in a 3-D-HEVC test model.

Adaptive merge list construction for 3D‐HEVC fast encoder

Adaptive merge list construction for the fast encoder of three-dimensional high efficiency video coding (3D-HEVC) scheme is proposed in this Letter. The encoder complexity of 3D-HEVC has been unavoidably very high in comparison with HEVC, because it should handle the multiple view images and their corresponding depth images. Proposed method is designed to reduce the encoder-side computational complexity of the 3D-HEVC merging process through a fast decision with early termination arranged by two conditions. Simulation results show that the proposed method can reduce overall encoding time by 9.7% without coding losses in comparison with the recent 3D-HEVC HTM-14.0 fast encoder that consisted of every encoder optimisation algorithm informatively adopted in both HEVC and 3D-HEVC standardisation activities.

Multiview and 3D Video Compression Using Neighboring Block Based Disparity Vectors

Compression of the statistical redundancy among different viewpoints, i.e., inter-view redundancy, is a fundamental and critical problem in multiview and three-dimensional (3D) video coding. To exploit the inter-view redundancy, disparity vectors are required to identify pixels of the same objects within two different views; in this way, the enhancement coding tools can be efficiently employed as new modes in block-based video codecs to achieve higher compression efficiency. Although disparity can be converted from depth, it is not possible in multiview video coding since depth information is not considered. Even when depth information is coded, it breaks the so-called multiview compatibility wherein texture views can be decoded without depth information. To resolve this problem, in this paper, a neighboring block-based disparity vector derivation (NBDV) method is proposed. The basic concept of NBDV is to derive a disparity vector (DV) of a current block by utilizing the motion information of spatially and temporally neighboring blocks predicted from another view. Through extensive experiments and analysis, it is shown that the proposed NBDV method achieves efficient DV derivation in the state-of-art video codecs, and it keeps the multiview compatibility with a relatively lower complexity. The proposed method has become an essential part of the 3D video standard extensions of H.264/AVC and HEVC.

Efficient depth intraprediction method for H.264/AVC-based three-dimensional video coding

We present an intracoding method that is applicable to depth map coding in multiview plus depth systems. Our approach combines skip prediction and plane segmentation-based prediction. The proposed depth intraskip prediction uses the estimated direction at both the encoder and decoder, and does not need to encode residual data. Our plane segmentation-based intraprediction divides the current block into biregions, and applies a different prediction scheme for each segmented region. This method avoids incorrect estimations across different regions, resulting in higher prediction accuracy. Simulation results demonstrate that the proposed scheme is superior to H.264/advanced video coding intraprediction and has the ability to improve the subjective rendering quality.

DMMFast: a complexity reduction scheme for three-dimensional high-efficiency video coding intraframe depth map coding

We present a complexity reduction scheme for the depth map intraprediction of three-dimensional high-efficiency video coding (3-D-HEVC). The 3-D-HEVC introduces a new set of specific tools for depth map coding, inserting additional complexity to intraprediction, which results in new challenges in terms of complexity reduction. Therefore, we present the DMMFast (depth modeling modes fast prediction), a scheme composed of two new algorithms: the simplified edge detector (SED) and the gradient-based mode one filter (GMOF). The SED anticipates the blocks that are likely to be better predicted by the traditional intramodes, avoiding the evaluation of DMMs. The GMOF applies a gradient-based filter in the borders of the block and predicts the best positions to evaluate the DMM 1. Software evaluations showed that DMMFast is capable of achieving a time saving of 11.9% on depth map intraprediction, considering the random access mode, without affecting the quality of the synthesized views. Considering the all intraconfigurations, the proposed scheme is capable of achieving, on average, a time saving of 35% considering the whole encoder. Subjective quality assessment was also performed, showing that the proposed technique inserts minimal quality losses in the final encoded video.

Fractal Depth Map Sequence Coding Algorithm with Motion-vector-field-based Motion Estimation

Three-dimensional video coding is one of the main challenges restricting the widespread applications of 3D video and free viewpoint video. In this paper, a novel fractal coding algorithm with motion-vector-field-based motion estimation for depth map sequence is proposed. We firstly add pre-search restriction to rule the improper domain blocks out of the matching search process so that the number of blocks involved in the search process can be restricted to a smaller size. Some improvements for motion estimation including initial search point prediction, threshold transition condition and early termination condition are made based on the feature of fractal coding. The motion-vector-field-based adaptive hexagon search algorithm on the basis of center-biased distribution characteristics of depth motion vector is proposed to accelerate the search. Experimental results show that the proposed algorithm can reach optimum levels of quality and save the coding time. The PSNR of synthesized view is increased by 0.56 dB with 36.97% bit rate decrease on average compared with H.264 Full Search. And the depth encoding time is saved by up to 66.47%. Moreover, the proposed fractal depth map sequence codec outperforms the recent alternative codecs by improving the H.264/AVC, especially in much bitrate saving and encoding time reduction.

The Three-dimensional Coding Based on the Cone for XML Under Weaving Multi-documents

In view of the deficiency of the XML document definitions extend the XML elements have to modify the origi- nal elements, under the multi-document analyzes the problem of low efficiency of the coding and the query when expand the node, this paper proposes the three-dimensional coding scheme applied to multi-document environment. The XML tree will be regarded as a cone, combining height, radian and marks to encode. Using the corresponding symbol to repre- sent and using mathematical model to describe several new added marker elements, the coding method avoids the recod- ing of the node due to expansion of the XML document, improving the reusability of the document. The results show that, the method of definition improves the modularity and portability of the document definitions, compared with the existing coding methods, the three-dimensional coding has better performance and feasibility.

Optimizing rate allocation between multiview videos and associated depth maps with quantization-based virtual view distortion model and genetic algorithm

In three-dimensional video coding (3-DVC), it is reasonable to allocate different coding bits for multiview videos and associated depth maps because of their different characteristics to meet the bits’ restraints of bandwidth/storage. We first propose a virtual view average distortion model. Then, based on the quantization parameter (QP), the average distortion-QP models and sum bitrate-QP models are proposed to depict the average distortions and sum bitrates of the referenced views for multiview videos and depth maps. Finally, the 3-DVC bit allocation problem is converted as a constrained optimization problem, which is solved by a genetic algorithm to search for the optimal QP pair. Experimental results demonstrate the effectiveness of our proposed models. Since the bit allocation scheme takes the performance of synthesized view and bitrate utilization into consideration, the absolute difference between the constraint and the actual coding bitrates (referred to as “rate inaccuracy”) of the proposed method is only 7.405% on average, which greatly outperforms the fixed 5 ∶ 1 ratio-based method with a rate inaccuracy of 19.103% and the planar model-based method with a rate inaccuracy of 20.556%. Compared with these two methods, our proposed method can achieve a maximum 1.951-dB gain under the same bitrates constraint.

Early SKIP mode decision for three-dimensional high efficiency video coding using spatial and interview correlations

In the test model of high efficiency video coding (HEVC) standard–based three-dimensional (3-D) video coding (3-D-HEVC), the variable size motion estimation (ME) and disparity estimation (DE) have been employed to select the best coding mode for each treeblock in the encoding process. This technique achieves the highest possible coding efficiency, but it brings extremely high computational complexity that limits 3-D-HEVC from practical applications. An early SKIP mode decision algorithm based on spatial and interview correlations is proposed to reduce the computational complexity of the ME/DE procedures. The basic idea of the method is to utilize the spatial and interview properties of coding information in previous coded frames to predict the current treeblock prediction mode and early skip unnecessary variable-size ME and DE. Experimental results show that the proposed algorithm can significantly reduce computational complexity of 3-D-HEVC while maintaining nearly the same rate distortion performance as the original encoder.

Visible region of interest extraction method for three-dimensional video coding

A visible region of interest (ROI) extraction method based on region expansion is proposed. It is a method that is combined with a new reverse realization of calibration based on homography. The proposed method extracts the ROI automatically and successfully and simultaneously calibrates the distortion caused by the stereoscopic camera system. It is a practical and easy method because it does not require any extra processing before display. It can be used to discard the confusing margin of the synthesized image of several parallax images and reduce the data amount needed to be processed and encoded. Here, real-time capturing, transmitting, and three-dimensional displaying systems are demonstrated to verify its effectiveness and feasibil- ity. © 2014 SPIE and IS&T (DOI: 10.1117/1.JEI.23.4.043008)

Effective early termination algorithm for depth map intra coding in 3D‐HEVC

In the current high-efficiency video coding-based three-dimensional video coding (3D-HEVC) design, new depth intra modes including depth modelling modes and region boundary chain coding are applied for depth map coding. These partition-based intra modes achieve the highest possible coding efficiency, but result in extremely large encoding time which obstructs the 3D-HEVC from practical applications. An efficient early termination algorithm for depth map coding in 3D-HEVC is proposed. It makes use of the coding information from the spatial neighbouring depth map treeblock and the co-located texture video treeblock to predict the depth map intra mode treeblock and terminate its mode decision process early. Experimental results show that the proposed algorithm can achieve an average computational saving of about 40% with negligible loss of rate distortion performance in the 3D-HEVC encoder.

THE DISTRIBUTED CODING OF THREE-DIMENSIONAL STRUCTURES OF VIDEO DATA

In article tendency existence concerning appearance of a video data services providing services of a quality three-dimensional digital video is explained. It creates some kind of hindrance concerning extension of new services, including applications which use power effective communication technologies. Therefore an actual perspective of researches is enhancement of theoretical and technological basis for creation of new methods of compression representation of video streams. One of approaches is additional increase of compression ratio is provided at the expense of the accounting of structural regularities at the same time on three coordinates. Relevance of creation of technologies of processing as separate frame and their sequences is justified. Shortcomings of step-by-step three-dimensional polyadic coding of data in the direction from low elements for the universal computing architecture are justified. The bit-by-bit recurrent diagram of computation in the direction fom low elements is developed. In this case formation of a code will be organized with taking into account adding of the element located in random place of three-dimensional structure of video data. Distinctive possibility of the developed coding is that value of weight coefficient depends only on the bases of the TSV previous elements. This feature provides possibility of the organization of restoration process for one pass.

METHODOLOGY OF THE ASSESSMENT FOR ALGORITHMIC COMPLEXITY OF PARALLEL IMPLEMENTATION OF THREE-DIMENSIONAL POLYADIC CODING

It is shown that further enhancement of video information technologies in the direction of resolution enhancement that inevitably lead to the sharp growth of video data volumes. Origin of a contradiction between requirements on the one hand concerning safety of the led-up information, and on the other side of relatively timeliness of its receiving is in turn justified. One of the effective directions of creation of three-dimensional coding technologies are the code structures created for three-dimensional poliadic numbers. It is developed methodologies of an assessment of complexity of three-dimensional three-level poliadic coding algorithmic implementation with use of the parallel diagram of processing. The methodology is based on property of the three-dimensional poliadic coding consisting in a possibility to parallelize computation of a code for all three-dimensional poliadic number (TDPN). It is justified that the number of operations is reduced as result of the following multisequencing: parallel computation of a code at the expense of sequential on conjugate enlargement of discharges on columns; parallel computation of line code at the expense of sequential on conjugate enlargement of discharges in the lines.

Joint Bit Allocation and Rate Control for Coding Multi-View Video Plus Depth Based 3D Video

In three-dimensional (3D) video coding, distortion in texture video and depth maps can all affect the quality of the synthesized virtual views. Therefore, under the total bitrate constraint, effective bit allocation between texture and depth information is very important for 3D video coding. In this paper, the major technical contribution is to formulate view synthesis quality for optimal resource allocation in 3D video coding, since such quality is what that matters most to the ultimate user (i.e., the viewer) of the system; to be more specific, a new joint bit allocation and rate control method for multi-view video plus depth (MVD) based 3D video coding is proposed accordingly. We firstly derive a view synthesis distortion model to characterize the effect of coding distortion of texture video and depth maps on the synthesized virtual views. Based on this model, we derive a rate-distortion model to characterize the relationship between the bitrate and the view synthesis distortion, and the optimal bitrate ratio between texture and depth is established adaptively by solving the associated optimization problem. Finally, the rate control algorithm is performed on view level, texture/depth level and frame level. Experimental results show that compared with other methods, the proposed bit allocation method obtains higher performance of view synthesis. Moreover, the proposed rate control method can accurately control the bitrate to satisfy the total bitrate constraint.