There has been an increasing consensus that the machine vision is gradually replacing human vision in numerous tasks, with the demonstrated success of artificial intelligence. In this paper, we propose a deep image compression scheme towards machine vision, with the principle of “begin with the end in mind”. In particular, a unified optimization scheme for end-to-end image compression towards machine vision is proposed, accompanied with the dedicated variable bitrate coding and generalized rate-accuracy optimization. The presented framework, which jointly optimizes the compression and the machine vision networks, exploits the utmost potential of robust machine vision for compressed images. The variable bitrate modules towards machine vision, which effectively shrink the storage space for model parameters, are further developed to accommodate to the real-world applications. Moreover, an iterative algorithm is presented to achieve the optimality in terms of the generalized rate-accuracy towards machine vision. Experimental results show that the proposed framework achieves the state-of-the-art object detection performance among the end-to-end image compression methods: in the exploration of Video Coding for Machines (VCM) in Moving Picture Experts Group (MPEG), and the proposed framework achieves 31.69% and 23.96% BD-rate gains compared with the VCM official test datasets, the Open Images dataset and the TVD dataset respectively, which are generated using the state-of-the-art standard Versatile Video Coding (VVC) standard. The generalization capability of the proposed framework is also verified with instance segmentation under various scenarios.

Traditional video compression approaches build upon the hybrid coding framework with motion-compensated prediction and residual transform coding. In this paper, we propose the first end-to-end deep video compression framework to take advantage of both the classical compression architecture and the powerful non-linear representation ability of neural networks. Our framework employs pixel-wise motion information, which is learned from an optical flow network and further compressed by an auto-encoder network to save bits. The other compression components are also implemented by the well-designed networks for high efficiency. All the modules are jointly optimized by using the rate-distortion trade-off and can collaborate with each other. More importantly, the proposed deep video compression framework is very flexible and can be easily extended by using lightweight or advanced networks for higher speed or better efficiency. We also propose to introduce the adaptive quantization layer to reduce the number of parameters for variable bitrate coding. Comprehensive experimental results demonstrate the effectiveness of the proposed framework on the benchmark datasets.

Accurate classification of input signals is the key prerequisite for variable bit-rate coding, which has been introduced in order to effectively utilize limited communication bandwidth. Especially, recent surge of multimedia services elevate the importance of speech/music classification. Among many speech/music classifier, the ones based on support vector machine (SVM) have a strong selling point, high classification accuracy, but their computational complexity and memory requirement hinder their way into actual implementations. Therefore, techniques that reduce the computational complexity and the memory requirement is inevitable, particularly for embedded systems. We first analyze implementation of an SVM-based classifier on embedded systems in terms of execution time and energy consumption, and then propose two techniques that alleviate the implementation requirements: One is a technique that removes support vectors that have insignificant contribution to the final classification, and the other is to skip processing some of input signals by virtue of strong correlations in speech/music frames. These are post-processing techniques that can work with any other optimization techniques applied during the training phase of SVM. With experiments, we validate the proposed algorithms from the perspectives of classification accuracy, execution time, and energy consumption.

Layered Coding Schemes for Video Transmission on ATM Networks

Variable bit rate (VBR) coding techniques have received great research interest as very promising tools for transmitting bursty multimedia traffic with low bandwidth requirements over a communication link. Statistically multiplexing the multimedia bursty traffic is a very efficient method of maximizing the utilization of the link capacity. The application of computer simulation techniques in analyzing a rate-based access control scheme for multimedia traffic such as voice traffic is discussed. The control scheme regulates the packetized bursty traffic at the user network interface of the link. Using a suitable congestion measure, namely, the multiplexer buffer length, the scheme dynamically controls the arrival rate by switching the coder to a different compression ratio (i.e., changing the coding rate). VBR coding methods can be adaptively adjusted to transmit at a lower rate with very little degradation in the voice quality. Reported results prove that the scheme greatly improves the link performance, in terms of reducing the probability of call blocking and enhancing the statistical multiplexing gain.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Variable bit-rate coding based on human visual system

A two layers video coding scheme for ATM networks

The bandwidth flexibility offered by the asynchronous transfer mode (ATM) technique makes it possible to select picture quality and bandwidth over a wide range in a simple and straightforward manner. A prototype model of a video codec was developed that demonstrates the feasibility of both variable bit rate (VBR) coding and user-selectable picture quality. The VBR coding algorithm is discussed and it is shown how a stabilized quality is achieved and how this quality and associated bandwidth can be selected by the user. How error propagation is limited to reduce the visibility of cell losses is also discussed. Interfaces with the ATM network are analyzed, with emphasis on decoder synchronization and absorption of cell delay jitter. The VBR codec offers very good picture quality for videophony applications at an equivalent load of 5.9 Mb/s. Picture quality remains relatively constant, even for heavy motion. >

Statistical characteristics of video signals for video packet coding, are clarified and a variable-bit-rate coding method for asynchronous transfer mode (ATM) networks is described that is capable of compensating for packet loss. ATM capabilities are shown to be greatly affected by delay, delay jitter, and packet loss probability. Packet loss has the greatest influence on picture quality. Packets may be lost either due to random bit error in a cell header or to network control when traffic is congested. A layered coding technique using discrete-cosine transform (DCT) coding is presented which is suitable for packet loss compensation. The influence of packet loss on picture quality is discussed, and decoded pictures with packet loss are shown. The proposed algorithm was verified by computer simulations.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>