Distributed Compressive Video Sensing Research Articles

MAP-Inspired Deep Unfolding Network for Distributed Compressive Video Sensing

Due to the lack of probabilistic inference guidance, the network structure of current deep learning-based Distributed Compressive Video Sensing (DCVS) algorithms cannot efficiently exploit frame measurements, inter-frame correlations, and intra-frame correlations. In this paper, we infer the optimization equation for DCVS reconstruction according to the maximum a posteriori estimation and propose a Deep Unfolding Multi-Hypothesis Aggregation Network (DUMHAN) to obtain high-quality reconstruction frames. In each stage of the DUMHAN, three modules are designed to implement the three update steps in the inferred optimization equation, which correspond to the three information flows, namely measurement, intra-frame, and inter-frame information flows. In particular, a Multi-Hypothesis Aggregation (MHA) module is developed to enhance the utilization of inter-frame and intra-frame correlation by generating and fusing multiple hypothesis sets. Moreover, we propose the stage-by-stage optical flow update approach in DUMHAN to improve the accuracy of both intra-frame and inter-frame correlation modeling. The experiments demonstrate that the proposed DUMHAN achieves state-of-the-art performance.

Low-cost and high-efficiency privacy-protection scheme for distributed compressive video sensing in wireless multimedia sensor networks

As a new video coding technology, distributed compressive video sensing (DCVS) uses compressed sensing (CS) independent encoding and joint decoding. Since DCVS breaks through the constraint of traditional video coding, it is suitable for resource-constrained wireless multimedia sensor networks (WMSNs). However, two major issues related to DCVS in WMSNs need to be solved urgently: one is to balance the storage burden of encoder and recovery quality of decoder; the other is to provide privacy-protection for video coding and transmission. We intend to break out of the existing limitations and design a new scheme which can simultaneously ensure privacy protection and high-efficiency coding for DCVS in WMSNs. Firstly, the two-pattern adaptive group of pictures selection is adopted to distinguish key frames and non-key frames. Secondly, the deterministic binary block diagonal measurement matrix is optimized to reduce sampling complexity. Thirdly, the scrambling-substitution-diffusion encryption method is proposed to resist various typical or potential attacks. Numerous experiments demonstrate that our scheme can not only perform valid and high-efficiency video coding, but also meet the demands of real-time and secure data transmission in WMSNs.

Iterative Reweighted Tikhonov-Regularized Multihypothesis Prediction Scheme for Distributed Compressive Video Sensing

Distributed compressive video sensing (DCVS) has great potential for signal acquisition and processing in source-limited communication, e.g., wireless video sensors networks, because it shifts complicated motion estimation and motion compensation from the encoder to the decoder. Known as a state-of-the-art technique in DCVS, multihypothesis (MH) prediction is widely used because of its acceptable performance and low computational complexity. However, this technique is restricted by inaccurate regularizations, which can cause susceptibility to inaccurate hypotheses. In this paper, we present an iterative reweighted Tikhonov-regularized scheme for MH prediction reconstruction. Specifically, to enhance robustness, this scheme proposes a reweighted Tikhonov regularization that synthetically considers three factors that affect the MH prediction performance—accuracy of the hypothesis set, number of hypotheses, and accuracy of regularizations—by utilizing the influence of each hypothesis. Furthermore, to avoid over-iteration in iterative MH prediction reconstruction, we propose a Bhattacharyya coefficient-based stopping criterion for use in the recovery of non-key frames, in which we exploit the similarity to an adjacent key frame rather than a previous iteration result. The simulation results show that the proposed scheme outperforms the state-of-the-art MH methods in terms of robustness to inaccurate hypotheses when there are a limited number of hypotheses.

High-Performance Distributed Compressive Video Sensing: Jointly Exploiting the HEVC Motion Estimation and the ℓ1 – ℓ1 Reconstruction

The distributed compressive video sensing (DCVS) system combines the advantages of compressed sensing (CS) and distributed video coding (DVC), suitable for the limited-resource video sensing and transmission environment. In this paper, we propose a comprehensive high performance DCVS system. First, we introduce the BM3D-AMP algorithm reconstruct key (K) frames. Second, we propose a new high efficiency video coding (HEVC) motion estimation (ME) algorithm with motion vector (MV) prediction method. By integrating the segmentation idea and motion estimation, this algorithm can gets more accurate side information (SI). Finally, we propose the l 1 - l 1 minimization model to achieve non-key (NK) frames joint high-quality reconstruction. We utilize the alternating direction method of multipliers (ADMM) algorithm to solve it. With the idea of dividing and conquering, the general problem is decomposed into several smaller pieces. Experimental results demonstrate that the proposed system has significant improvement over its counterparts.

JsrNet: A Joint Sampling-Reconstruction Framework for Distributed Compressive Video Sensing.

Huge video data has posed great challenges on computing power and storage space, triggering the emergence of distributed compressive video sensing (DCVS). Hardware-friendly characteristics of this technique have consolidated its position as one of the most powerful architectures in source-limited scenarios, namely, wireless video sensor networks (WVSNs). Recently, deep convolutional neural networks (DCNNs) are successfully applied in DCVS because traditional optimization-based methods are computationally elaborate and hard to meet the requirements of real-time applications. In this paper, we propose a joint sampling–reconstruction framework for DCVS, named “JsrNet”. JsrNet utilizes the whole group of frames as the reference to reconstruct each frame, regardless of key frames and non-key frames, while the existing frameworks only utilize key frames as the reference to reconstruct non-key frames. Moreover, different from the existing frameworks which only focus on exploiting complementary information between frames in joint reconstruction, JsrNet also applies this conception in joint sampling by adopting learnable convolutions to sample multiple frames jointly and simultaneously in an encoder. JsrNet fully exploits spatial–temporal correlation in both sampling and reconstruction, and achieves a competitive performance in both the quality of reconstruction and computational complexity, making it a promising candidate in source-limited, real-time scenarios.

Distributed Compressive Video Sensing with Mixed Multihypothesis Prediction

Traditional video acquisition systems require complex data compression at the encoder, which makes them unacceptable for resource-limited applications such as wireless multimedia sensor networks (WMSNs). To address this problem, distributed compressive video sensing (DCVS) represents a novel sensing approach with a simple encoder. This method shifts the computational burden from the encoder to the decoder and needs a robust reconstruction algorithm. In this paper, a mixed measurement-based multihypothesis (MH) reconstruction algorithm (mixed-MH) is proposed for DCVS to improve the reconstruction quality at low sampling rates. Considering the inaccuracy of MH prediction when measurements are insufficient, the available side information (SI) is resampled to obtain the artificial measurements, which are then integrated into real measurements via regularization. Furthermore, to avoid the negative effect of SI at high sampling rates, an adaptive regularization parameter is designed to balance the contributions of real and artificial measurements at different sampling rates. The experimental results demonstrate that the proposed mixed-MH prediction scheme outperforms other state-of-the-art algorithms in the reconstruction quality at the same low sampling rate.

Residual Reconstruction Algorithm Based on Half-Pixel Multi-Hypothesis Prediction for Distributed Compressive Video Sensing

Compressed sensing (CS) provides a method to sample and reconstruct sparse signals far below the Nyquist sampling rate, which has great potential in image/video acquisition and processing. In order to fully exploit the spatial and temporal characteristics of video frame and the coherence between successive frames, we propose a half-pixel interpolation based residual reconstruction method for distributed compressive video sensing (DCVS). At the decoding end, half-pixel interpolation and bi-directional motion estimation helps refine the side information for joint decoding of the non-key-frames. We apply a multi-hypothesis based on residual reconstruction algorithms to reconstruct the non-key-frames. Performance analysis and simulation experiments show that the quality of side information generated by the proposed algorithm is increased by about 1.5dB, with video reconstruction quality increased 0.3~2dB in PSNR, when compared with prior works on DCVS.

Perceptual hash algorithm‐based adaptive GOP selection algorithm for distributed compressive video sensing

Distributed compressive video sensing (DCVS) is a novel video coding technique that shifts sophisticated motion estimation and compensation from the encoder to the decoder and is suitable for resource-limited communication, namely wireless video sensor networks (WVSNs). In DCVS, key frames serve as the reference for subsequent non-key frames in a given group of pictures (GOP). However, in fast-motion sequences, e.g. scene-changing sequences, a fixed GOP size can cause inaccuracy in the selection of reference key frames. The difference in the peak signal-to-noise ratio between key frames and non-key frames caused by this inaccuracy appears as flicker in the decoded video, negatively affecting the quality of experience. To address this problem, the authors present a perceptual hash algorithm-based adaptive GOP selection algorithm for DCVS and a novel allocation model for the frame sampling rate. In addition, the authors define several indexes to assess the degree of flicker in decoded video. The experimental results demonstrate that the proposed algorithm reduces the degree of flicker in fast-motion sequences by 40-60% relative to the state-of-the-art architecture, while also outperforming other adaptive GOP selection strategies.

Distributed compressed video sensing based on the optimization of hypothesis set update technique

Compressed Sensing (CS) breakthroughs the limitation of Nyquist sampling rate and realizes the sampling and compression of data simultaneous. Hence, it is widely used in image processing and video compression. However, it remains a challenge to obtain the high quality reconstructed image and video. To this end, we focus on the reconstruction algorithm of CS and the melioration of the existing distributed compressed video sensing (DCVS). In the perspectives of hypothesis set design and reference frames selection, we give detailed analyses for existing schemes and propose hypothesis set updating (HSU) and dynamic reference frames selection (DRS) algorithms to polish up the reconstruction performance. Then the superiorities in performance for these schemes are illustrated. Finally, the simulation results indicate that at a low sampling rate, the block based DCVS with the proposed HSU and DRS (HD-BDCVS) ameliorates the quality of non-key frames and key frames simultaneously without increasing the complexity of the coding.

Rate-Distortion Optimized Distributed Compressive Video Sensing

Rate-Distortion Optimized Distributed Compressive Video Sensing

Optimal-correlation-based reconstruction for distributed compressed video sensing

Distributed compressed video sensing (DCVS) is a framework that integrates both compressed sensing and distributed video coding characteristics to achieve a low-complexity video coding. However, how to design an efficient joint reconstruction by leveraging more realistic signal models is still an open challenge. In this paper, we present a novel optimal-correlation-based reconstruction method for compressively sampled videos from multiple measurement vectors. In our method, the sparsity is mainly exploited through inter-signal correlations rather than the traditional frequency transform, wherein the optimization is not only over the signal space to satisfy data consistency but also over all possible linear correlation models to achieve minimum-l1-norm correlation noise. Additionally, a two-phase Bregman iterative based algorithm is outlined for solving the optimization problem. Simulation results show that our proposal can achieve an improved reconstruction performance in comparison to the conventional approaches, and especially, offer a 0.7–9.9dB gain in the average PSNR for DCVS.

A review of the state-of-the-art distributed compressive video sensing architectures

Low complexity video coding that provides efficient compression with reasonable reconstruction quality has been a desired requirement for resource-constrained video sensors in distributed vision-based sensing applications. In this paper, we present a review of the state-of-the-art codec architectures based on distributed compressive video sensing (DCVS), which is a relatively new video coding paradigm that integrates the techniques of distributed video coding (DVC) and compressive sensing (CS). The review includes a comparative discussion of several well-known DCVS architectures in literature with a focus on their functional aspects, and suggests a number of possible enhancements to the design of these architectures.

Dictionary learning based reconstruction for distributed compressed video sensing

Distributed compressed video sensing (DCVS) is a framework that integrates both compressed sensing and distributed video coding characteristics to achieve a low-complexity video coding. However, how to design an efficient reconstruction by leveraging more realistic signal models that go beyond simple sparsity is still an open challenge. In this paper, we propose a novel “undersampled” correlation noise model to describe compressively sampled video signals, and present a maximum-likelihood dictionary learning based reconstruction algorithm for DCVS, in which both the correlation and sparsity constraints are included in a new probabilistic model. Moreover, the signal recovery in our algorithm is performed during the process of dictionary learning, instead of being employed as an independent task. Experimental results show that our proposal compares favorably with other existing methods, with 0.1–3.5dB improvements in the average PSNR, and a 2–9dB gain for non-key frames when key frames are subsampled at an increased rate.

Dynamic global-principal component analysis sparse representation for distributed compressive video sampling

Video reconstruction quality largely depends on the ability of employed sparse domain to adequately represent the underlying video in Distributed Compressed Video Sensing (DCVS). In this paper, we propose a novel dynamic global-Principal Component Analysis (PCA) sparse representation algorithm for video based on the sparse-land model and nonlocal similarity. First, grouping by matching is realized at the decoder from key frames that are previously recovered. Second, we apply PCA to each group (sub-dataset) to compute the principle components from which the sub-dictionary is constructed. Finally, the non-key frames are reconstructed from random measurement data using a Compressed Sensing (CS) reconstruction algorithm with sparse regularization. Experimental results show that our algorithm has a better performance compared with the DCT and K-SVD dictionaries.

Adaptive Multihypothesis Prediction Algorithm for Distributed Compressive Video Sensing

A novel adaptive multihypothesis (MH) prediction algorithm for distributed compressive video sensing (DCVS) is proposed in this paper. In the proposed framework, consistent block-based random measurement for each video frame is adopted at the encoder independently. Meanwhile, a mode decision algorithm is applied in CS-blocks via block-based correlation measurements at the decoder. The inter-frame MH mode is selected for the current block wherein the interframe correlation coefficient value exceeds a predetermined threshold. Otherwise, the intraframe MH mode is worthwhile to be selected. Moreover, the adaptive search window and cross-diamond search algorithms on measurement domain are also incorporated to form the dictionary for MH prediction. Both the temporal and spatial correlations in video signals are exploited to enhance CS recovery to satisfy the best linear combination of hypotheses. The simulation results show that the proposed framework can provide better reconstruction quality than the framework using original MH prediction algorithm, and for sequences with slow motion and relatively simple scene composition, the proposed method shows significant performance gains at low measurement subrate.

Distributed Compressed Video Sensing in Camera Sensor Networks

With the booming of video devices ranging from low-power visual sensors to mobile phones, the video sequences captured by these simple devices must be compressed easily and reconstructed by relatively more powerful servers. In such scenarios, distributed compressed video sensing (DCVS), combining distributed video coding (DVC) and compressed sensing (CS), is developed as a novel and powerful signal-sensing and compression algorithm for video signals. In DCVS, video frames can be compressed to a few measurements in a separate manner, while the interframe correlation is explored by the joint recovery algorithm. In this paper, a new DCVS joint recovery scheme using side-information-based belief propagation (SI-BP) is proposed to exploit both the intraframe and interframe correlations, which is particularly efficient over error-prone channels. The DCVS scheme using SI-BP is designed over two frame signal models, the mixture Gaussian (MG) model and the wavelet hidden Markov tree (WHMT) model. Simulation results evaluated on two video sequences illustrate that the SI-BP-based DCVS scheme is error resilient when the measurements are transmitted through the noisy wireless channels.