Bit Rate Video Coding Research Articles

Moving objects extraction method in H.264/advanced video coding bit stream of a complex scene

For the purpose of extracting moving objects from H.264/advanced video coding (AVC) bit stream of a complex scene, an algorithm based on maximum a posteriori Markov random field (MRF) framework to extract moving objects directly from H.264 compressed video is proposed in this paper. It mainly involves encoding information of motion vectors (MVs) and block partition modes in H.264/AVC bit stream and utilizes temporal continuity and spatial consistency of moving object’s pieces. First, it retrieves MVs and block partition modes of identical 4×4 pixel blocks in P frames and establishes Gaussian mixture model (GMM) of the phase of MVs as a reference background, and then creates MRF model based on MVs, block partition modes, the GMM of the background, spatial, and temporal consistency. The moving objects are retrieved by solving the MRF model. The experimental results show that it can perform robustly in a complex environment and the precision and recall have been improved over the existing algorithm.

Scalable Video Coding Bit Stream Extraction Based on Equivalent MSE Method

The bit stream extraction plays an important role in Scalable Video Coding (SVC) [1]. However, one downside of current video coding methods is to ignore the video contents which is in fact an important factor for video coding efficiency. Therefore, an equivalent MSE method is proposed in this paper to extract substreams in the temporal and spatial enhancement layers. When the Motion Vectors (MVs) are large in one video, a larger frame rate is necessary to maintain the continuity of the object movement which makes no jump in the visual sense. In this sense, substreams extraction in temporal enhancement layer has to be satisfied. On the other hand, if there are some larger high-frequency components in a single frame of the video, that is to say, there are some higher spatial details in the video stream. As a result, it should try to meet the extraction requirement in spatial enhancement layer. This method has the advantage of considering the contents of the video, which can effectively improve the coding performance and quality. The experimental results have demonstrated the improved quality of reconstructed video for the equivalent MSE method when extracting bit stream arbitrarily at the same bandwidth.

Mobile face detection and tracking for media streaming applications

This paper presents a method for mobile face detection and tracking in media streaming applications. To account specifics of mobile image processing and pattern recognition, the experimental benchmarking of Intel x86 and Nokia S60 platforms is done. Various aspects on mobile phone use are considered in order to record a profile on a mobile media streaming user. The profile is generated by analysing pictures of mobile phone users and screenshots of their devices. The mobile face detection model is based on the optimised version of robust real-time object detection algorithm. Our optimisation enables to reduce the computational complexity of the original version in 4 times. We also describe the tracking mode in which the system achieves a near real-time frame rate (45 12 fps depending on the video complexity). The training of the face detector is done on the databases of more than 400 hr of video and the major parameters such as FRR and FAR are evaluated on a database of 30 min of video (including more than 5000 facial images) and 200 hr of video, respectively. Face tracking is benchmarked using 20 mobile video sequences, which cover a broad range of human activities and environmental changes. We describe the construction of the low bit-rate video coder that uses face detection and tracking to segment foreground/background object and encode them with a different bit-rate. We also describe a framework for smart videoconferences and experimental results on low bit-rate image coding for this application.

Opinion Model Using Psychological Factors for Interactive Multimodal Services

We propose the concept of an opinion model for interactive multimodal services and apply it to an audiovisual communication service. First, psychological factors of an audiovisual communication service were extracted by using the semantic differential (SD) technique and factor analysis. Forty subjects participated in subjective tests and performed point-to-point conversational tasks on a PC-based video phone that exhibited various network qualities. The subjects assessed those qualities on the basis of 25 pairs of adjectives. Two psychological factors, i.e., an aesthetic feeling and a feeling of activity, were extracted from the results. Then, quality impairment factors affecting these two psychological factors were analyzed. We found that the aesthetic feeling was affected by IP packet loss and video coding bit rate, and the feeling of activity depended on delay time, video packet loss, video coding bit rate, and video frame rate. Using this result, we formulated an opinion model derived from the relationships among quality impairment factors, psychological factors, and overall quality. The validation test results indicated that the estimation error of our model was almost equivalent to the statistical reliability of the subjective score.

Low complexity deblocking method for DCT coded video signals

A new method to remove blocking artefacts in low bit-rate block-based video coding, such as MPEG-4 and H.264, is presented. A low computational deblocking filter with five modes is proposed, including three frequency-related modes (smooth mode, intermediate mode, and complex mode for low-frequency, mid-frequency, and high-frequency regions, respectively), one special mode (steep mode for a large offset between two blocks) and a refined mode (corner mode for the corner of four blocks). A mode decision procedure is also needed to decide which mode is given by observing pixel behaviour around the block boundary. To take the masking effect of the human visual system (HVS) into consideration, the filter for smooth mode is designed to be much stronger than that for complex mode, because human eyes are more sensitive to smooth regions. Experimental results show that, in most cases, the proposed algorithm removes more blocking artefacts than MPEG-4 deblocking filters do, and improves both subjective and objective image quality. The proposed algorithm keeps the computation lower than MPEG-4 and is suitable for most block-based image and video coding systems.

Spiral intra macroblock refresh with motion vector restriction for low bit-rate video telephony over a 3G network

A method to limit the effect of error propagation in low bit-rate error resilient video coding has been proposed and implemented on a 3G prototype videophone. In conventional methods, the effect of the bit-stream error does not vanish rapidly since the refreshed macroblocks are affected by the errors in the unrefreshed macroblocks. To overcome this ineffectiveness, two techniques are used; one is motion vector restriction that limits the error propagation, and the other is the spiral intra macroblock refresh that minimizes coding efficiency degradation caused by the motion vector restriction. Further, the latter technique concentrates on the center area in the screen, where the subjective preference is closely related. Experiments show that the proposed method removes the effect of the error more rapidly than the conventional methods.

Video scalability rate control method by dynamic controlling of hierarchical bit allocation ratio

AbstractThis paper considers the hierarchical video coding system, and proposes a bit allocation rate control method by dynamic control of the hierarchical bit allocation ratio. As the first step, the MPEG‐2 SNR scalability method is considered as the hierarchical video coding, and the relation between the hierarchical bit allocation and the basic/enhancement layer quality is analyzed. It is shown that the intermediate bit allocation ratio may degrade greatly the quality in the enhancement layer. The dependency of the image quality on the hierarchical bit allocation is noted, and video coding bit allocation control is proposed, where the hierarchical loss can be suppressed by dynamically controlling the bit allocation ratio for each picture. A control method is proposed particularly, where the optimal bit allocation ratio is calculated so that the characteristic function is maximized under the constraint that the bit allocation ratio for each GOP is kept constant. Lastly, a simulation experiment is presented. It is shown that the hierarchical loss, which is generated in the conventional bit allocation control, can be suppressed by the proposed control method, and the effectiveness of the proposed method is demonstrated. © 2002 Wiley Periodicals, Inc. Electron Comm Jpn Pt 1, 86(1): 83–96, 2003; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ecja.1149

H.263-based video coder for home-produced video sequences

We present an automatic technique employing the H.263 standard to compress video sequences which have been generated by consumer equipment. The purpose is to optimise H.263 parameters on sequences having a wide range of characteristics in terms of detail content and motion, to obtain a low cost and low bit-rate video coder with acceptable performances in terms of visual quality. This technique is conceived to be implemented as a set-top box for the upload of home-produced video sequences.

Significance-linked connected component analysis for very low bit-rate wavelet video coding

A novel hybrid wavelet video coding algorithm termed video significance-linked connected component analysis (VSLCCA) is developed for very low bit-rate applications. In the proposed VSLCCA codec, first, fine-tuned motion estimation based on the H.263 Recommendation is developed to reduce temporal redundancy, and exhaustive overlapped block motion compensation is utilized to ensure coherency in motion compensated error frames. Second, the wavelet transform is applied to each coherent motion compensated error frame to attain global energy compaction. Third, significant fields of wavelet-transformed error frames are organized and represented as significance-linked connected components so that both the within-subband clustering and the cross-scale dependency are exploited. Last, the horizontal and vertical components of motion vectors are encoded separately using adaptive arithmetic coding while significant wavelet coefficients are encoded in bit-plane order by using high order Markov source modeling and adaptive arithmetic coding. Experimental results on eight standard MPEG-4 test sequences show that for intraframe coding, on average the proposed codec exceeds H.263 and ZTE (zero-tree entropy) in peak signal-to-noise ratio by as much as 2.07 and 1.38 dB at 28 kbit/s, respectively. For entire sequence coding, VSLCCA is superior to H.263 and ZTE by 0.35 and 0.71 dB on average, respectively.

Adaptive vector quantization for fixed bit-rate videocoding

Abstract In this paper a new algorithm is introduced and discussed for self-adaptive vector quantization of video sequences. The structure of the vector quantizer and the mechanisms for its construction and updating are based on a binary tree whose topology and content adaptively track the varying statistics of the incoming data. Some tree branches are extended more than others depending on the time varying distribution of the input vectors, to guarantee an acceptable reconstruction quality, not far from the target performances of a full search approach. The resulting bit-rate stems from several terms carrying information for both frame reconstruction and codebook updating. An effective policy for codebit allocation has been devised and tested providing very good results within fixed bit-rate applications. Exhaustive simulations have been carried out on standard sequences at a rate of 30 frames/sec with progressive transmission. An average reconstruction quality of 34.5 dB in the peak signal-to-noise ratio has been achieved for the luminance component at a constant bit-rate of 512 kbit/sec corresponding to a compression of 30:1 (0.26 bit/pixel).