Hallucinated-PQA: No reference point cloud quality assessment via injecting pseudo-reference features

Abstract

Point clouds (PCs) have been increasingly applied in business and life, but a variety of PC transmission and generation systems inevitably produce various types of distortions. Therefore, it is of great significance to design an objective point cloud quality assessment (PCQA), particularly in a no-reference manner to evaluate the PC systems in the actual situation. However, due to the lack of original PCs, the existing no-reference (NR) PCQA metrics cannot perceive the feature changes caused by distortions, resulting in inaccurate quality prediction. In addition, dependent on the viewing habits of the subjects in the subjective evaluation experiment, a strong contextual correlation among intra- and inter-view local regions at different scales naturally exists, but the existing projection-based no-reference PCQA only using concatenation or average pooling operations cannot reflect this relationship of multi-view features fusion. Considering the above challenges, we propose a novel hallucination-guided NR PCQA framework, namely Hallucinated-PQA. Specifically, we introduce a distortion restoration network to correct multiple projected images in preprocessing to provide pseudo-reference information for NR PCQA. In the feature extraction of distorted PCs, we designed a hallucination injection block (HIB) by utilizing feature differences to assist the feature description of distorted PCs, and a multi-view and multi-scale context fusion (MMCF) module to construct the contextual correlation among intra- and inter-view local regions at different scales. Experimental results show that our Hallucinated-PQA can achieve comparable or better performance than state-of-the-art (SOTA) metrics on four open PCQA databases. The source code will be released at https://github.com/QSBAOYANGMU/Hallucinated-PQA.