Bidirectional Feature Aggregation Network for Stereo Image Quality Assessment Considering Parallax Attention-Based Binocular Fusion

Abstract

Inspired by the two-path visual information processing mechanism (i.e., a bottom-up path and a top-down path), we propose a bidirectional binocular feature aggregation based stereo image quality assessment (SIQA) network, which considers a two-path visual mechanism and realizes the binocular fusion based on parallax information. To better aggregate binocular features from different levels, a two-path feature aggregation structure, which simulates the bottom-up and top-down mechanism in human visual system (HVS), is proposed. It not only realizes the supplement of low-level detail information to high-level semantic in the bottom-up path, but also realizes the supplement of high-level semantic information to low-level detail in the top-down path. Simultaneously, because feature misalignment exists in binocular features of adjacent levels, a feature alignment module (FAM) based on deformable convolution is designed to integrate the binocular fusion features of adjacent levels. In addition, considering the importance role of parallax in guiding binocular fusion, a binocular fusion module (BFM) based on parallax attention mechanism, which is different with existing binocular fusion methods, is explicitly proposed to achieve the binocular fusion between the left and right view features. Extensive experiments are conducted on LIVE I, LIVE II, WIVC I and WIVC II databases to demonstrate the effectiveness of the proposed method.