Standard Coder Research Articles

Impact of LiDAR point cloud compression on 3D object detection evaluated on the KITTI dataset

The rapid growth on the amount of generated 3D data, particularly in the form of Light Detection And Ranging (LiDAR) point clouds (PCs), poses very significant challenges in terms of data storage, transmission, and processing. Point cloud (PC) representation of 3D visual information has shown to be a very flexible format with many applications ranging from multimedia immersive communication to machine vision tasks in the robotics and autonomous driving domains. In this paper, we investigate the performance of four reference 3D object detection techniques, when the input PCs are compressed with varying levels of degradation. Compression is performed using two MPEG standard coders based on 2D projections and octree decomposition, as well as two coding methods based on Deep Learning (DL). For the DL coding methods, we used a Joint Photographic Experts Group (JPEG) reference PC coder, that we adapted to accept LiDAR PCs in both Cartesian and cylindrical coordinate systems. The detection performance of the four reference 3D object detection methods was evaluated using both pre-trained models and models specifically trained using degraded PCs reconstructed from compressed representations. It is shown that LiDAR PCs can be compressed down to 6 bits per point with no significant degradation on the object detection precision. Furthermore, employing specifically trained detection models improves the detection capabilities even at compression rates as low as 2 bits per point. These results show that LiDAR PCs can be coded to enable efficient storage and transmission, without significant object detection performance loss.

A Geodesic Translation Model for Spherical Video Compression.

Spherical video coding is critical to the success of many virtual reality and related applications. This paper focuses on an important class of spherical videos whose dynamics involve camera motion. A common approach to spherical video coding is to project from the sphere onto a plane (or planes), where a standard video coder is applied. The projection induces warping resulting in complex non-linear motion in the projected domain that severely comprises the performance of motion models in standard coders. To overcome this shortcoming, we propose a new motion model that captures the motion field on the sphere, and capitalizes on insights into the perceived motion on the sphere due to camera translation. Specifically, surrounding static points are perceived as moving along their respective geodesics, which all intersect at the points where the camera velocity vector intersects the sphere. We analyze the rate of translation along geodesics and its dependence on the elevation of a pixel on the sphere with respect to the camera velocity vector. The analysis leads to a motion vector modulation scheme that perfectly captures the perceived motion of each pixel. Complementary to the new motion model, we propose a search grid tailored to capture expected geodesic motion on the sphere for effective motion estimation. The proposed method yields significant bit-rate savings over employing standard HEVC after projection, which validates its efficacy.

Assessing Medical Students' Tobacco Dependence Treatment Skills Using a Detailed Behavioral Checklist

Construct: This article describes the development and implementation of an assessment intended to provide objective scores that would be valid indications of medical students' abilities to counsel patients about tobacco dependence. Background: Assessing medical students' advanced communication skills, particularly in the context of providing tobacco-dependence treatment, consistently and accurately is challenging; doing so across multiple medical schools is even more difficult. Approach: Ten medical schools implemented a tobacco-dependence treatment case as part of an Objective Structured Clinical Examination for 3rd-year medical students. A 33-item checklist with detailed criteria and examples was developed for scoring students' performances. Trained coders viewed and coded 660 videotaped encounters; approximately 10% also were coded by the coding supervisor to check accuracy. Results: Average time required to code an encounter was approximately 31 minutes; accuracy (i.e., agreement with the gold standard coder) was excellent. Overall, students performed an average of 1 in 4 of the 33 behaviors included on the checklist, and only 1 in 10 discussed setting a quit date. Most students (almost 9 in 10) asked how much the patient smoked in a day, and just over 7 in 10 informed the patient that the cough was due to smoking. Conclusions: The authors developed and implemented a rigorous assessment that will be used to evaluate medical students' tobacco-dependence treatment skills. Operationalizing the specific counseling behaviors, training coders to accurately capture students' performances using a structured checklist, and conducting the coding all required substantial time commitments but will provide confidence in the objectivity of the assessment results. In addition, this assessment can be used to provide formative information on medical students' tobacco-dependence treatment skills and to tailor ongoing training for medical students in this area.

Non-linear speech representation based on local predictability exponents

Looking for new perspectives to analyze non-linear dynamics of speech, this paper presents a novel approach based on a microcanonical multiscale formulation which allows the geometric and statistical description of multiscale properties of the complex dynamics. Speech is a complex system whose dynamics can be, to some extent, geometrically and statistically accessed by the computation of Local Predictability Exponents (LPEs) unlocking the determination of the most informative subset (Most Singular Manifold or MSM), leading to associated compact representation and reconstruction. But the complex intertwining of different dynamics in speech (added to purely turbulent descriptions) suggests the definition of appropriate multiscale functionals that might influence the evaluation of LPEs, hence leading to more compact MSM. Consequently, by using the classical and generic Sauer/Allebach algorithm for signal reconstruction from irregularly spaced samples, we show that speech reconstruction of good quality can be achieved using MSM of low cardinality. Moreover, in order to further show the potential of the new methodology, we develop a simple and efficient waveform coder which achieves almost the same level of perceptual quality as a standard coder, while having a lower bit-rate.

A 2.4kb/s Speech Coder with Improved Analysis and Synthesis Based on MELP

This paper describes a speech coder which has an input signal frame interval of 20ms percent in duration, contains 160 voice samples (8,000 samples/s) and 48 bits. A key feature of the coder is a novel Line Spectral Frequencies (LSF) quantization scheme, requiring only 19 bits per frame. This new coder, through algorithmic improvements, enhanced quantization techniques and new harmonic synthesis method, produces better speech quality at 2.4 kb/s transnission rate than the new U.S. Federal Standard MELP coder.

A Scalable Multiple Description Scheme for 3D Video Coding Based on the Interlayer Prediction Structure

The most recent literature indicates multiple description coding (MDC) as a promising coding approach to handle the problem of video transmission over unreliable networks with different quality and bandwidth constraints. Furthermore, following recent commercial availability of autostereoscopic 3D displays that allow 3D visual data to be viewed without the use of special headgear or glasses, it is anticipated that the applications of 3D video will increase rapidly in the near future. Moving from the concept of spatial MDC, in this paper we introduce some efficient algorithms to obtain 3D substreams that also exploit some form of scalability. These algorithms are then applied to both coded stereo sequences and to depth image-based rendering (DIBR). In these algorithms, we first generate four 3D subsequences by subsampling, and then two of these subsequences are jointly used to form each of the two descriptions. For each description, one of the original subsequences is predicted from the other one via some scalable algorithms, focusing on the inter layer prediction scheme. The proposed algorithms can be implemented as pre- and postprocessing of the standard H.264/SVC coder that remains fully compatible with any standard coder. The experimental results presented show that these algorithms provide excellent results.

An HVS based adaptive quantization scheme for the compression of color images

In this paper a Human Visual System based adaptive quantization scheme is proposed. The proposed algorithm supports perceptually lossless as well as lossy compression. The algorithm uses a transform based compression approach using the wavelet transform, and has incorporated vision models for the compression of both luminance and chrominance components. The major strength of the coder is the incorporation of the vision model for the chrominance components and the optimum way in which the scales are distributed among the luminance and chrominance components to achieve higher compression ratios. The perceptual model developed for the color components gives flexibility for giving more compression for the color components without causing any color degradations. For each image the visual thresholds are evaluated and an optimum bit allocation is done in such a way that the quantization error is always less than the visual distortion for the given rate. To validate the strength of the proposed algorithm, the perceptual quality of the images reconstructed using the proposed coder is compared with the images reconstructed with JPEG2000 standard coder, for the same compression. To evaluate the perceptual quality of the compressed images latest perceptual quality matrices such as Structural Similarity Index, Visual Information Fidelity and Visual Signal-to-Noise Ratio are used. The results obtained reveal that the proposed structure gives excellent improvement in perceptual quality compared to the existing schemes, for both lossy as well as lossless compression. These advantages make the proposed algorithm a good candidate for replacing the quantizer stage of the current image compression standards.

Memory and architectural optimizations for soft video encoders

This paper presents a set of platform-independent architectural optimizations for improving the performance of software-based standard video coders. These denote changes that affect the underlying memory model and physical architecture of the encoder with an objective of achieving maximum encoder performance with respect to execution time and memory usage. The coding quality does not suffer due to these modifications because the algorithm itself is not changed. An interface driven methodology has been developed to identify and ameliorate performance bottlenecks for any encoder. Appropriate data flow between components has been proposed so that memory intensive operations including memory accesses and copying are minimized. The proposed methods have been applied to an MPEG4 reference implementation to demonstrate the computational improvements achieved while avoiding any algorithmic modifications. These techniques have been shown to result in improvements in the range of 15–50% in the overall encoding time of a video codec on a general-purpose computing platform. The resulting implementation is also shown to be faster than some well-known open source solutions.

Frame-Based Multiple-Description Video Coding with Extended Orthogonal Filter Banks

We propose a frame-based multiple-description video coder. The analysis filter bank is the extension of an orthogonal filter bank which computes the spatial polyphase components of the original video frames. The output of the filter bank is a set of video sequences which can be compressed with a standard coder. The filter bank design is carried out by taking into account two important requirements for video coding, namely, the fact that the dual synthesis filter bank is FIR, and that loss recovery does not enhance the quantization error. We give explicit results about the required properties of the redundant channel filter and the reconstruction error bounds in case of packet errors. We show that the proposed scheme has good error robustness to losses and good performance, both in terms of objective and visual quality, when compared to single description and other multiple description video coders based on spatial subsampling. PSNR gains of 5 dB or more are typical for packet loss probability as low as 5%.

Balanced multiple description video coding using optimal partitioning of the DCT coefficients

In this paper, we propose a balanced multiple description video coding scheme which is based on the partitioning of the discrete cosine transform (DCT) coefficients. Our scheme splits the single-layered stream generated by a standard coder into two correlated and balanced (virtually identical rates and distortions) substreams. The optimization is in the redundancy-rate-distortion sense using Lagrangian relaxation for optimum allocation of redundancy among the blocks in the frame. A greedy algorithm to meet the equal distortions criterion and an algorithm to optimally achieve equal bit rates for the two descriptions are proposed. Our simulation results substantiate our claim of achieving balanced descriptions at high peak signal-to-noise ratios values for any specified redundancy and any target bit rate. Our design complies with the existing motion-compensated DCT-based standards, i.e., each description can be independently decoded by any standard decoder.

Model-Based Speech Signal Coding Using Optimized Temporal Decomposition for Storage and Broadcasting Applications

A dynamic programming-based optimization strategy for a temporal decomposition (TD) model of speech and its application to low-rate speech coding in storage and broadcasting is presented. In previous work with the spectral stability-based event localizing (SBEL) TD algorithm, the event localization was performed based on a spectral stability criterion. Although this approach gave reasonably good results, there was no assurance on the optimality of the event locations. In the present work, we have optimized the event localizing task using a dynamic programming-based optimization strategy. Simulation results show that an improved TD model accuracy can be achieved. A methodology of incorporating the optimized TD algorithm within the standard MELP speech coder for the efficient compression of speech spectral information is also presented. The performance evaluation results revealed that the proposed speech coding scheme achieves 50%-60% compression of speech spectral information with negligible degradation in the decoded speech quality.

Improving the transcoding capability of speech coders

With the trend of merging various communication networks, a need arises to provide transcoding between different speech coding formats. Presently this means a cross tandem between the two coders in each case. This results in both quality loss and extra delay. A possible alternative is using a bitstream mapping approach that directly converts parameter values. For several standard coders having a similar coding structure, it should be possible to generate comparable or better quality without adding much delay or complexity. This paper proposes a bitstream mapping method between ITU-T Recommendation G.729 and TIA IS-641. Informal listening tests and the perceptual subjective quality measure (PSQM) scores show that the proposed method has better quality than the cross tandem method, while it has at least 5 ms less delay and six times less computation.

Hybrid global–local motion compensated frame interpolation for low bit rate video coding

A hybrid global–local motion compensated frame interpolation (HMCI) scheme for low bit rate video coding is proposed in this paper. Different from most other global–local motion compensation algorithms, which were performed at the encoder end, the proposed scheme works solely on the decoded video stream. Thus, it can be applied to any MCP (motion compensated prediction) based standard coders without bit stream syntax modification. Along with the proposed HMCI scheme, a motion-based foreground–background segmentation method is presented. Traditional intensity-based segmentation methods can only segment moving objects from still background. In contrast, the proposed approach can segment foreground objects from both moving and still background by exploiting information carried by block-based motion vectors. Both the six-parameter affine or the eight-parameter perspective models can be applied to the foreground–background segmentation and the moving background reconstruction; while the six-parameter affine model with triangular patch warping is adopted to reconstruct the foreground blocks due to its low complexity. Experiments show that the proposed HMCI scheme achieves higher subjective (visual) quality compared to the conventional block-based frame interpolation without global motion compensation, where artifacts in moving backgrounds are significantly reduced.

Low bit rate speech coding using spectral trajectory modeling

In the U.S. Federal Standard coder for 2400 bps, a data frame containing 54 bits of encoded signal is transmitted every 22.5 ms. In each frame, 25 bits encode the spectral features (10 Line Spectrum Frequencies—LSF). In this paper we describe a method for reducing the transmission rate while preserving most of the quality and intelligibility. This method is based on modeling the spectral trajectories with polynomial functions and on encoding these functions for segments of speech extending over multiple frames. Here 10 polynomials are computed by fitting them to the 10 LSF trajectories in the least-squares sense. Then the polynomial coefficients are encoded for the whole segment instead of directly encoding the LSF vectors. The spectral parameters are thus reduced (compressed) to [(P+1)/N]×100%, where P represents the order of the polynomials and N the number of frames for each segment. Different compression rates can be achieved. For example, for P=5 and N=10 the spectral features are encoded using 40% less bits than those required to encode N=10 LSF vectors. For this example, the overall transmission rate is reduced to 1956 bps, yielding an average Itakura–Saito spectral distance of 0.14 between the original and reconstructed LPC parameters computed for a typical utterance.

Interactions between speech coders and disordered speech

We examined the impact of standard speech coders currently used in modern communication systems, on the quality of speech from persons with common speech and voice disorders. Four standardized coders, viz. G. 728 LD-CELP, GSM 6.10 RPE-LTP, FS1016 CELP, FS1015 LPC and the recently proposed US Federal Standard 2400 bps MELP were evaluated with speech samples collected from 30 disordered talkers. Objective speech quality measures, including the auditory distance parameter based on the measuring normalizing blocks technique, and the perceptual speech quality measure, and subjective impressions of speech coder performance were used to assess the interaction between speech coder and speech disorder. Objective speech quality measures revealed that the performance of the LD-CELP and GSM RPE-LTP coders was not measurably influenced by the type of input speech, and that MELP, FS1015 LPC and to a certain extent FS1016 CELP exhibited degraded performance with speech samples from disordered talkers. Results from perceptual experiments were in contrast with the objective measures of speech quality; ratings of speech coder performance indicated that the listeners are less sensitive to coder-induced distortions with abnormal speech samples.

Interaction of speech coders and atypical speech II: effects on speech quality.

We investigated how standard speech coders, currently used in modern communication systems, affect the quality of the speech of persons who have common speech and voice disorders. Three standardized speech coders (GSM 6.10 RPE-LTP, FS1016 CELP, and FS1015 LPC) and two speech coders based on subband processing were evaluated for their performance. Coder effects were assessed by measuring the quality of speech samples both before and after processing by the speech coders. Speech quality was rated by 10 listeners with normal hearing on 28 different scales representing pitch and loudness changes, speech rate, laryngeal and resonatory dysfunction, and coder-induced distortions. Results showed that (a) nine scale items were consistently and reliably rated by the listeners; (b) all coders degraded speech quality on these nine scales, with the GSM and CELP coders providing the better quality speech; and (c) interactions between coders and individual voices did occur on several voice quality scales.

Waveform interpolation for 4 kbit∕s wideband speech coding

An implementation of waveform interpolation (WI) applied to wideband speech coding at 4 kbit/s is presented. Listening tests comparing narrowband speech coded using standard coders are presented, with wideband speech coded using WI. Results show a clear preference for wideband speech over narrowband speech coded at an equivalent bit rate.

Removal of motion uncertainty and quantization noise in motion compensation

In a standard hybrid video coder, there are two important factors affecting motion-compensated prediction: motion uncertainty and quantization noise, motion uncertainty results from the use of block-decimated motion compensation, which cannot specify the motion for each pixel. The propagation of quantization noise results from interframe prediction. We analyze both these effects, and their block-structured nonstationary properties. We propose a block-adaptive linear filtering framework to reduce motion uncertainty and quantization noise propagation simultaneously. This new motion-compensated predictor can be viewed as joint application of overlapped block motion compensation and loop filtering (LF). Several system configurations are evaluated. We show that this linear filtering framework achieves better rate-distortion performance than the single use of either overlapped block motion compensation or LF.

Orientation-sensitive interpolative pyramids for lossless and progressive image coding

This paper presents a modified JPEG coder that is applied to the compression of mixed documents (containing text, natural images, and graphics) for printing purposes. The modified JPEG coder proposed in this paper takes advantage of the distinct perceptually significant regions in these documents to achieve higher perceptual quality than the standard JPEG coder. The region-adaptivity is performed via classified thresholding being totally compliant with the baseline standard. A computationally efficient classification algorithm is presented, and the improved performance of the classified JPEG coder is verified.

Image compression via edge-based wavelet transform

In this paper, an edge-based wavelet transform is proposed for image compression. In the encoder, first the dominant edges of an image are detected and coded as side information. Then, the wavelet transform (WT) is carried out in such a way that no filtering over previously detected edges is performed. Compression examples show that the edge-based WT achieves good reproduction of sharp edges even at very low bit rates. Because of the additional side information, the PSNR is typically slightly lower than for standard wavelet coders. On the other hand, the picture quality scale, an objective quality measure that better reflects the subjective impression, shows superior results for the proposed coder.