Joint Bit Research Articles

Adaptive Video Streaming in Multi-Tier Computing Networks: Joint Edge Transcoding and Client Enhancement

With the advancement of multimedia technology and wireless networks, there is a growing demand for high-quality video streaming. Delivering stable video streaming in extremely dynamic wireless networks, nevertheless, is still an open problem. Recent developments in client computing and mobile edge computing (MEC) technologies have both shown promise in enhancing the adaptive bitrate (ABR) streaming services. In this paper, we consider a video streaming system in multi-tier computing networks, enabled by joint edge-side video transcoding and client-side video enhancement. By “enhancement,” we mean that the client improves the video chunk quality via client-side image processing modules. In particular, we aim to design a joint bitrate adaptation, edge transcoding, and client image-processing algorithm, maximizing the quality of experience (QoE) of streaming services. The majority of the prior art has concentrated on super-resolution-enabled video streaming. Contrarily, we show that the video enhancement method outperforms the super-resolution approach in terms of signal-to-noise ratio and frames per second, implying a superior alternative for client-side processing in ABR streaming. We formulate the problem as an event-triggered Markov decision process (E-MDP), and propose a deep reinforcement learning (DRL)-based framework, named EDTEA. To deal with the delayed feedback induced by multi-tier computing, the entropy and the expected re-buffering terms are introduced to the objective and the reward, respectively. Extensive simulations based on real-world videos and bandwidth traces manifest that compared with state-of-the-art approaches, EDTEA provides <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$10.4\%\sim 78.4\%$</tex-math></inline-formula> extra QoE while reducing re-buffering time by <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$85.5\%\sim 91.7\%$</tex-math></inline-formula> .

Twofold Sparsity: Joint Bit- and Network-Level Sparsity for Energy-Efficient Deep Neural Network Using RRAM Based Compute-In-Memory

Twofold Sparsity: Joint Bit- and Network-Level Sparsity for Energy-Efficient Deep Neural Network Using RRAM Based Compute-In-Memory

Structural similarity-based rate control algorithm for 3D video

Human vision easily identifies the structural degradation in a video and thus perceptual quality improvement is necessary. To adjust the video quality to match with the human vision we proposed a structural similarity (SSIM) based rate control algorithm for the 3D video. We incorporated a structural similarity index (SSIM) as the quality metric in rate control algorithm. The rate-distortion model and Lagrange multiplier are derived considering the structural dissimilarity (dSSIM) as the distortion metric to achieve the rate control with perceptual quality improvement. Furthermore, the optimal joint bit allocation scheme at texture video/depth map level, frame level, and basic unit (BU) level is modified to incorporate dSSIM for measuring distortion. The proposed algorithm is implemented in HEVC compression standard HTM-16.2. The performance of the proposed algorithm is compared using RD curves, BD-Rate comparison, and subjective evaluation. Besides, rate accuracy is also computed to measure the bit rate mismatch. Compared to the original λ-domain rate control algorithm, the proposed algorithm achieves a better SSIM along with a reduction in bit rate.

Model-Based Joint Bit Allocation Between Geometry and Color for Video-Based 3D Point Cloud Compression

Rate distortion optimization plays a very important role in image/video coding. But for 3D point cloud, this problem has not been investigated. In this paper, the rate and distortion characteristics of 3D point cloud are investigated in detail, and a typical and challenging rate distortion optimization problem is solved for 3D point cloud. Specifically, since the quality of the reconstructed 3D point cloud depends on both the geometry and color distortions, we first propose analytical rate and distortion models for the geometry and color information in video-based 3D point cloud compression platform, and then solve the joint bit allocation problem for geometry and color based on the derived models. To maximize the reconstructed quality of 3D point cloud, the bit allocation problem is formulated as a constrained optimization problem and solved by an interior point method. Experimental results show that the rate-distortion performance of the proposed solution is close to that obtained with exhaustive search but at only 0.68% of its time complexity. Moreover, the proposed rate and distortion models can also be used for the other rate-distortion optimization problems (such as prediction mode decision) and rate control technologies for 3D point cloud coding in the future.

Joint Bit Allocation and Hybrid Beamforming Optimization for Energy Efficient Millimeter Wave MIMO Systems

In this article, we aim to design highly energy efficient end-to-end communication for millimeter wave multiple-input multiple-output systems. This is done by jointly optimizing the digital-to-analog converter (DAC)/analog-to-digital converter (ADC) bit resolutions and hybrid beamforming matrices. The novel decomposition of the hybrid precoder and the hybrid combiner to three parts is introduced at the transmitter (TX) and the receiver (RX), respectively, representing the analog precoder/combiner matrix, the DAC/ADC bit resolution matrix and the baseband precoder/combiner matrix. The unknown matrices are computed as a solution to the matrix factorization problem where the optimal fully digital precoder or combiner is approximated by the product of these matrices. A novel and efficient solution based on the alternating direction method of multipliers is proposed to solve these problems at both the TX and the RX. The simulation results show that the proposed solution, where the DAC/ADC bit allocation is dynamic during operation, achieves higher energy efficiency when compared with existing benchmark techniques that use fixed DAC/ADC bit resolutions.

A Deep Reinforcement Learning Approach to Multiple Streams’ Joint Bitrate Allocation

For widely used real-time applications, encoding and transmitting multiple videos jointly over a limited bandwidth has become a popular topic. Allocating different bitrates for different sources is a better way to meet different demands from applications. In this paper, we focus on providing equal quality to users by minimizing the variance of distortion among sequences, which is denoted as the minVAR problem. The state-of-the-art Look-ahead and Feed-back Allocation Model (LFAM) allocates bitrate by taking both look-ahead complexity measures and feed-back information into consideration. However, LFAM brings additional delay to real-time applications. By taking the bitrate allocation problem as a time-series decision making problem, we propose a Deep-Reinforcement-Learning-based approach to allocate bitrate with only feed-back information to solve the two-source minVAR problem. Afterward, we introduce a binary-tree-based hierarchical approach to apply our model to arbitrary number of sources. Tested with the widely used open-source x264 encoder, our approach decreases the variance compared with LFAM in all experiments under two-, three- and four-source scenarios. Furthermore, the proposed approach also outperforms LFAM in the mean quality. The proposed approach is insensitive to the order of sequences and encoders with different complexities, showing its robustness and generalization capability.

Artificial Intelligence Aided Joint Bit Rate Selection and Radio Resource Allocation for Adaptive Video Streaming over F-RANs

Recently, fog-computing-based radio access networks (F-RANs) have been conceptualized to provide high quality of experience (QoE) for adaptive bit rate (ABR) streaming, where additional computing capacity is supplemented on fog nodes to facilitate complicated cross-layer optimization (i.e., joint bit rate selection and radio resource allocation). However, finding an optimal global solution with acceptable complexity is still infeasible by the conventional optimization methods. In this work, we propose an artificial intelligence (AI) aided joint bit rate selection and radio resource allocation scheme referred to as iABR, which provides a new vision for handling the over-complicated optimization in F-RANs. Based on multi-agent hierarchy deep reinforcement learning, the proposed iABR can dynamically allocate radio resource and select bit rate in a multiuser scenario, by perceiving the network environment and clients' player information. Moreover, long short-term memory (LSTM) is employed by the iABR algorithm, which enables accurate prediction of the change of channel quality by learning the history of the wireless channel. Hence, iABR is able to adjust the action policy in advance to accommodate the future channel quality for avoiding bit rate fluctuation. According to the experimental results, the iABR exhibits higher QoE in terms of high average bit rate, low rebuffering ratio, and average bit rate variance. Last but not least, the QoE performance of all the clients are fairly guaranteed by the iABR algorithm, enhancing the practicality of AI-driven F-RANs for multimedia service delivery.

Joint bit and power loading for adaptive MIMO OFDM VLC systems

AbstractVisible light communication (VLC) is a short‐range wireless access technology based on the dual use of illumination structure. The frequency‐selective characteristics of VLC channels motivate the use of orthogonal frequency division multiplexing (OFDM). In addition, the presence of multiple light sources in most indoor spaces makes multiple‐input–multiple‐output (MIMO) communication techniques a natural solution for VLC systems. In this paper, we investigate the design of an adaptive MIMO OFDM system. Specifically, we propose adaptive bit and power loading for a direct current–biased OFDM VLC system with MIMO mode switching between repetition coding and spatial multiplexing. We formulate the adaptive algorithm design as an optimization problem where we aim to maximize spectral efficiency through the proper selection of modulation order, power level, and MIMO mode, while satisfying a targeted bit error rate. To solve the underlying NP‐hard problems, we use a simulated annealing heuristic. We present Monte Carlo simulation results for a typical indoor setting and illustrate the performance improvements through link adaptation.

A Multi User based Joint Bit Error Rate improvements in PDMA

The development of mobile Internet and Internet of things (IoT), the 5th generation (5G) wireless communications will foresee explosive increase in mobile traffic. To address challenges in 5Gisuchiasihigherispectraliefficiency,imassive connectivity,iandilowerilatency,isomeiNon-OrthogonaliMultipleiAccessi(NOMA)ischemesihaveibeenirecentlyiactivelyiinvestigatediincludingipower-domainiNOMA,imultipleiaccess with Low-Density Spreading (LDS), Sparse Code Multiple Access (SCMA), Multi-user Shared Access (MUSA), pattern division multiple access (PDMA). Pattern Division Multiple Access is the key technique for analyzing group of users at a time. In this paper we investigate BER performance for 5G systems with different multiple accesses schemes, i.e Orthogonal Frequency Division Multiplexing (OFDM), Filter Bank Multicarrier (FBMC), Generalized Frequency Division Multiplexing (GFDM),in comparison to these we design a closed form analysis method of Multi-user FDMC technique far better. Simulation results show that MU FDMC performs better than conventional methods.

Joint rate adaptation and resource allocation for real-time H.265/HEVC video transmission over uplink OFDMA systems

We consider multiuser video communication over uplink orthogonal frequency-division multiple access (OFDMA) systems. A cross-layer algorithm of joint bit allocation, packet scheduling and wireless resource assignment are proposed to minimize the end-to-end expected video distortion. Video rate adaptation is performed under the wireless resource constraints. The target number of encoding bits for each video packet is obtained to minimize the estimated distortion based on the online content-based rate-distortion function. Due to the inaccuracy of the rate control algorithm in H.265/HEVC encoding, the actual number of bits may differ from the target. Accordingly, the actual encoder distortion may deviate from the estimated distortion. Then, we propose an iterative algorithm to re-assign wireless resources based on the actual number of encoded bits to obtain the final resource allocation policy and packet scheduling decision. Numerical simulation results show that our proposed approach significantly outperforms the baseline algorithms in terms of received video quality.

Reed–Muller Network Coded-Cooperation With Joint Decoding

This letter proposes a Reed–Muller network coded-cooperative (RMNCC) scheme with the joint decoding. The Kronecker product (KP) construction is utilized to construct RM codes. The KP construction is effectively employed to accomplish network coded-cooperation between two users. An asymptotic error performance bound is presented to validate the average BER performance of the RMNCC scheme over fast Rayleigh fading channel. Monte Carlo simulated results for the RMNCC scheme agree with the analytical framework at higher SNR. Furthermore, a joint bit selection algorithm for the optimum selection of information bits at the relay node is also proposed, which results in an improved BER performance.

SSIM-based joint-bit allocation for 3D video coding

The quality of a 3D video display depends on virtual view synthesis process which is affected by the bit allocation criterion. The performance of a bit allocation algorithm is dependent on various encoding parameters like quantization parameter, motion vector, mode selection, and so on. Rate-distortion optimization (RDO) is used to efficiently allocate bits with minimum distortion. In 3D video, rate-distortion (RD) property of synthesized view is used to assign bits between texture video and depth map. Existing literature on bit allocation methods use mean square error (MSE) as distortion metric which is not suitable for measuring perceptual quality. In this paper, we propose structural similarity (SSIM)-based joint bit allocation scheme to enhance visual quality of 3D video. Perceptual quality of a synthesized view depends on texture and depth map quality. Thus, SSIM-based RDO is performed on both texture and depth map where SSIM is used as distortion metric in mode decision and motion estimation. SSIM-based distortion model for synthesized view is determined experimentally. As SSIM cannot be related to quantization step, SSIM-MSE relation is used to convert distortion model in terms of MSE. The Lagrange multiplier method is used to solve the bit allocation problem. The proposed algorithm is implemented using 3DV-ATM as well as HEVC. RD curves show reduction in bitrate with an improvement in SSIM of synthesized view.

Power minimization for OFDM modulated two-way amplify-and-forward relay wireless sensor networks

This paper studies a power minimization problem of an orthogonal frequency-division multiplexing modulated two-way amplify-and-forward relay wireless sensor network (WSN). The objective is to develop effective strategies for controlling the transmit power levels of WSN nodes and assigning the bit rates among different subcarriers to minimize the overall transmit power of the network, while ensuring the end-to-end target rates of the two WSN sources. By using channel knowledge at transmitters, a two-step method is proposed to solve the power minimization problem, resulting in a joint power control and bit rate assignment (JPCBRA) algorithm. At the first step, a per-subcarrier optimization problem concerning the power minimization issue is solved, giving closed-form solutions for individual transmit powers at the WSN sources and the relay. At the second step, the end-to-end target rates of the two WSN sources are allocated to different subcarriers so that the total transmit power can be reduced further while satisfying the initial data-rate requirement of the two WSN sources. Computer simulation results confirmed the effectiveness of the proposed JPCBRA algorithm. It is shown that with the JPCBRA algorithm, the total transmit power of the network can be significantly reduced in any traffic and/or channel cases in contrast to the compared schemes.

Method of joint bit rate/modulation format identification and optical performance monitoring using asynchronous delay-tap sampling for radio-over-fiber systems

In the context of carrying a wide variety of modulation formats and data rates for home networks, the study covers the radio-over-fiber (RoF) technology, where the need for an alternative way of management, automated fault diagnosis, and formats identification is expressed. Also, RoF signals in an optical link are impaired by various linear and nonlinear effects including chromatic dispersion, polarization mode dispersion, amplified spontaneous emission noise, and so on. Hence, for this purpose, we investigated the sampling method based on asynchronous delay-tap sampling in conjunction with a cross-correlation function for the joint bit rate/modulation format identification and optical performance monitoring. Three modulation formats with different data rates are used to demonstrate the validity of this technique, where the identification accuracy and the monitoring ranges reached high values.

Joint Bit Allocation and Rate Control for Texture and Depth Video Coding

Three-dimensional (3D) video systems based on Depth-image-based Rendering (DIBR) include both texture videos and corresponding depth maps. The joint coding of texture and depth video is important for the 3D video data. In this paper, a joint bit allocation and rate control method is proposed for 3D video coding. First, the characteristics of bit allocation and rate control are investigated. Second, a generic exponential distortion model is derived for joint video coding. Finally, based on the distortion model, the joint bit allocation problem is formulated as a constrained optimization problem and solved through the Lagrangian multiplier method. The system performance has been evaluated with several 3D video sequences. Experimental results demonstrate that the proposed method achieves excellent rate-distortion performance.

CRC-Free Hybrid ARQ System Using Turbo Product Codes

This paper presents a hybrid automatic repeat request (HARQ) system using turbo product codes (TPC). The inherent word-error detection capability of TPC is exploited to replace the conventional cyclic redundancy check (CRC) used for packet error detection in conventional HARQ systems. Therefore, TPC are used for joint bit error correction and packet error detection. Consequently, the HARQ system efficiency is improved by increasing the system throughput when short packets are transmitted or by reducing the computational complexity/delay when the packets transmitted are long. Monte Carlo simulation results reveal that the CRC-free TPC-HARQ system consistently provides equivalent or higher throughput than CRC-based HARQ systems. Moreover, numerical results show that TPC self-detection has lower computational complexity than CRC detection, particularly for TPCs with high code rates. In particular scenarios, the relative complexity of the self-detection approach with respect to popular CRC techniques is about 0.3%.

Linear Transceiver Design for Nonorthogonal Amplify-and-Forward Protocol Using a Bit Error Rate Criterion

The ever growing demand of higher data rates can now be addressed by exploiting cooperative diversity. This form of diversity has become a fundamental technique for achieving spatial diversity by exploiting the presence of idle users in the network. This has led to new challenges in terms of designing new protocols and detectors for cooperative communications. Among various amplify-and-forward (AF) protocols, the half duplex non-orthogonal amplify-and-forward (NAF) protocol is superior to other AF schemes in terms of error performance and capacity. However, this superiority is achieved at the cost of higher receiver complexity. Furthermore, in order to exploit the full diversity of the system an optimal precoder is required. In this paper, an optimal joint linear transceiver is proposed for the NAF protocol. This transceiver operates on the principles of minimum bit error rate (BER), and is referred as joint bit error rate (JBER) detector. The BER performance of JBER detector is superior to all the proposed linear detectors such as channel inversion, the maximal ratio combining, the biased maximum likelihood detectors, and the minimum mean square error. The proposed transceiver also outperforms previous precoders designed for the NAF protocol.

Joint Subcarrier and Bit Allocation for Secondary User with Primary Users’ Cooperation

Interference between primary user (PU) and secondary user (SU) transceivers should be mitigated in order to implement underlay spectrum sharing in cognitive radio networks (CRN). Considering this scenario, an improved joint subcarrier and bit allocation scheme for cognitive user with primary users’ cooperation (PU Coop) in CRN is proposed. In this scheme, the optimization problem is formulated to minimize the average interference power level at the PU receiver via PU Coop, which guarantees a higher primary signal to interference plus noise ratio (SINR) while maintaining the secondary user total rate constraint. The joint optimal scheme is separated into subcarrier allocation and bit assignment in each subcarrier via arith-metric geo-metric (AM-GM) inequality with asymptotical optimization solution. Moreover, the joint subcarrier and bit optimization scheme, which is evaluated by the available SU subcarriers and the allocated bits, is analyzed in the proposed PU Coop model. The performance of cognitive spectral efficiency and the average interference power level are investigated. Numerical analysis indicates that the SU’s spectral efficiency increases significantly compared with the PU non-cooperation scenario. Moreover, the interference power level decreases dramatically for the proposed scheme compared with the traditional Hughes-Hartogs bit allocation scheme.

Joint Bit Allocation and Rate Control for Coding Multi-View Video Plus Depth Based 3D Video

In three-dimensional (3D) video coding, distortion in texture video and depth maps can all affect the quality of the synthesized virtual views. Therefore, under the total bitrate constraint, effective bit allocation between texture and depth information is very important for 3D video coding. In this paper, the major technical contribution is to formulate view synthesis quality for optimal resource allocation in 3D video coding, since such quality is what that matters most to the ultimate user (i.e., the viewer) of the system; to be more specific, a new joint bit allocation and rate control method for multi-view video plus depth (MVD) based 3D video coding is proposed accordingly. We firstly derive a view synthesis distortion model to characterize the effect of coding distortion of texture video and depth maps on the synthesized virtual views. Based on this model, we derive a rate-distortion model to characterize the relationship between the bitrate and the view synthesis distortion, and the optimal bitrate ratio between texture and depth is established adaptively by solving the associated optimization problem. Finally, the rate control algorithm is performed on view level, texture/depth level and frame level. Experimental results show that compared with other methods, the proposed bit allocation method obtains higher performance of view synthesis. Moreover, the proposed rate control method can accurately control the bitrate to satisfy the total bitrate constraint.

Rate-Distortion Model Based Bit Allocation for 3-D Facial Compression Using Geometry Video

With the extensive applications of 3-D multimedia technology, 3-D content compression has been an important issue, which ensures its smooth transmission on the network with constrained bandwidth. In this letter, we propose a new compression framework for dynamic 3-D facial expressions. Taking advantage of the near-isometric property of human facial expressions, we parameterize the dynamic 3-D faces into an expression-invariant canonical domain, which naturally generates 2-D geometry videos and allows us to apply the well-studied video compression techniques. Due to the difference from natural videos, each dimension (i.e., X, Y and Z, respectively) of the geometry video is regarded as a video sequence and encoded separately. Meanwhile, a model-based joint bit allocation scheme is designed to allocate reasonable bitrate to each dimension by detailed analysis of rate-distortion model for geometry videos, to obtain optimal results under given target bitrate. Experimental results show that up to 25% improvement in terms of bitrate reduction can be achieved, compared to existing algorithms.