MH-Net: A Learnable Multi-Hypothesis Network for Compressed Video Sensing

Chao Zhou,Fei Ding,Dengyin Zhang,Yao Zhang,Can Chen

doi:10.1109/access.2019.2954140

Chao Zhou, Fei Ding + Show 3 more

Open Access

https://doi.org/10.1109/access.2019.2954140

Copy DOI

Abstract

This paper establishes a novel recovery network called MH-Net-a framework for compressed video sensing (CVS) based on recently emerging deep neural networks (DNNs) techniques. MH-Net exploits temporal correlation between frames in the form of multi-hypothesis (MH) prediction, and learns a high-dimensional domain which is more robust for prediction generation. After the MH prediction, a special residual network is used in MH-Net to reconstruct the residuals between the MH prediction and the desired frame from their measurements. The final reconstruction is derived by adding the reconstructed residuals to the MH prediction. Unlike the block-wise reconstruction in existing DNN-based CS architecture, MH-Net builds a mapping from block measurements to a complete frame reconstruction, leading to better reconstruction quality. Benefitting from the DNN's nature, the forward propagation of MH-Net is extremely fast, making it suitable for real-time applications. Experimental results show that MH-Net presents a better recovery performance compared with existing DNN-based recovery methods and traditional iterative recovery algorithms.

Highlights

Compressed sensing (CS) [1]–[4] establishes a novel sensing paradigm to recover the signal x ∈ RN from few measurements y ∈ RM (M N ) derived by non-adaptive linear projections: y = x, (1)where ∈ RM×N is the sensing matrix
Where ∈ RM×N is the sensing matrix. This paradigm is popular in some video sensing applications where high sampling rate is costly or even prohibitive, such as wireless multimedia sensor networks (WMSNs)
We provide MH-Net with a large amount of training data to learn how to reconstruct residuals using the inside structure of these data

Summary

INTRODUCTION

Compressed sensing (CS) [1]–[4] establishes a novel sensing paradigm to recover the signal x ∈ RN from few measurements y ∈ RM (M N ) derived by non-adaptive linear projections:. In [12], the impact of each hypothesis was considered by a reweighted Tikhonov regularization, enhancing the robustness of MH prediction Among these MH models, MH-BCS-SPL [8] presents a competitive reconstruction quality with a low time complexity, yielding the state-of-the-art recovery performance in traditional CVS methods. (1) Unlike traditional MH methods, MH-Net utilizes the similarity between hypothesis and target block in a highdimensional space to form the hypothesis weights This high-dimensional space is learnable so that it is capable of representing the features of hypotheses and target block better than the measurement domain derived by the sensing matrix. (2) Compared with existing DNN-based methods, the recovery of MH-Net exploits the temporal and spatial correlation without the limitation of block scope.

TRADITIONAL MH PREDICTION METHODS

RESIDUAL RECONSTRUCTION

IMPLEMENTATION

EXPERIMENTS

CONCLUSION