Prediction of mobile image saliency and quality under cloud computing environment

Abstract

Recent years have witnessed the explosive growth of multimedia applications over networks and increasingly high requirements of consumers for multimedia signals in terms of quality of experience (QoE). Effective and efficient yet energy-saving saliency detection model and quality prediction method are eagerly desired, since they play critical roles in raising users' QoE and promoting the progress of green multimedia communication. Current studies of saliency detection and quality evaluation are far from ideal yet. In this paper we investigate the influence of complexity on visual saliency and quality. Complexity is an essential concept in human perception to visual stimulus, but it is substantially abstract and hard to be endowed with a clear definition. We suppose that brain systematically combines global and local features during the whole process of human perception. Global features lead a dominant position in seeking salient areas under the condition that image complexity is high, namely without obviously isolated foreground objects, whereas local features play a key role in an opposite situation. With this consideration, this paper establishes a novel framework for detecting visual saliency based on image complexity estimation before complexity-adaptive merging of global and local features. Furthermore, the concept of complexity is deployed for blind photographic image quality assessment (IQA) by means of saliency-based weighting. Features which refer to contrast, artifacts, brightness and natural scene statistics (NSS) are modified and integrated to derive a blind IQA model and predict the quality of photos. Based on the above two technologies, this paper introduces smart phones as mobile terminals, cloud platforms for speed-up and energy-saving, and wireless networks for transmission, and provides a practical mobile multimedia application. Comparative experiments validate that, within this application system, our proposed saliency detection model and blind photographic IQA method implement better than existing relevant competitors in terms of effectiveness and efficiency comparison.