Existing panoramic image quality assessment models are relatively independent when extracting local features of each viewport, resulting in high computational complexity and difficulty in characterizing the correlation between viewports using an end-to-end fusion model. To address the above problems, a quality assessment method based on feature sharing and multi-viewport adaptive fusion is proposed. Using a shared backbone network, the viewport segmentation and calculation tasks that are independent of each other in the existing method are converted to the feature domain, so that the local features of the entire image can be extracted with only one feedforward calculation. On this basis, a feature domain viewport segmentation method based on spherical uniform sampling is introduced to ensure that the pixel density of the observation space and the representation space is consistent, and semantic information is used to guide the adaptive fusion of local quality features of each viewport. The linear correlation coefficient and rank correlation coefficient on the CVIQ and OIQA datasets are both above 0.96, which is the best compared with the existing mainstream evaluation methods. Compared with the traditional evaluation method SSIM, its average linear correlation coefficient and average rank correlation coefficient on the two datasets are improved by 9.52%and respectively 8.69%; compared with the latest evaluation method MPFIQA, its average linear correlation coefficient and average rank correlation coefficient are improved by 1.71%and respectively 1.44%.