Itti's Saliency Research Articles

Robust image hashing with visual attention model and invariant moments

Image hashing is an efficient technique of multimedia processing for many applications, such as image copy detection, image authentication, and social event detection. In this study, the authors propose a novel image hashing with visual attention model and invariant moments. An important contribution is the weighted DWT (discrete wavelet transform) representation by incorporating a visual attention model called Itti saliency model into LL sub-band. Since the Itti saliency model can efficiently extract saliency map reflecting regions of attention focus, perceptual robustness of the proposed hashing is achieved. In addition, as invariant moments are robust and discriminative features, hash construction with invariant moments extracted from the weighted DWT representation ensures good classification performance between robustness and discrimination. Extensive experiments with open image datasets are done to validate the performances of the proposed hashing. The results demonstrate that the proposed hashing is robust and discriminative. Performance comparisons with some hashing algorithms are also conducted, and the receiver operating characteristic results illustrate that the proposed hashing outperforms the compared hashing algorithms in classification performance between robustness and discrimination.

Visualization framework of digital paintings based on visual saliency for cultural heritage

In order to preserve cultural heritage, this paper proposes to develop a digital paintings visualization system. Our proposed system mainly consists of extracting regions of interest (ROI) from a digital painting to characterize them. These close-ups are then animated on the basis of the painting characteristics and the artist's or designer's aim. In order to obtain interesting results from short video clips, we developed a visual saliency map-based method by using the well-known Itti's saliency analysis which already proved its efficiency on paintings. The experimental results show the efficiency of our approach and an evaluation based on a Mean Opinion Score validates the proposed method. Eighteen volunters watched for a total of 216 views at 72 video clips build up from 6 paintings, and we found a significant difference between our saliency-based generated videos and random-based generated videos.

Image Processing Strategies Based on a Visual Saliency Model for Object Recognition Under Simulated Prosthetic Vision.

Retinal prostheses have the potential to restore partial vision. Object recognition in scenes of daily life is one of the essential tasks for implant wearers. Still limited by the low-resolution visual percepts provided by retinal prostheses, it is important to investigate and apply image processing methods to convey more useful visual information to the wearers. We proposed two image processing strategies based on Itti's visual saliency map, region of interest (ROI) extraction, and image segmentation. Itti's saliency model generated a saliency map from the original image, in which salient regions were grouped into ROI by the fuzzy c-means clustering. Then Grabcut generated a proto-object from the ROI labeled image which was recombined with background and enhanced in two ways--8-4 separated pixelization (8-4 SP) and background edge extraction (BEE). Results showed that both 8-4 SP and BEE had significantly higher recognition accuracy in comparison with direct pixelization (DP). Each saliency-based image processing strategy was subject to the performance of image segmentation. Under good and perfect segmentation conditions, BEE and 8-4 SP obtained noticeably higher recognition accuracy than DP, and under bad segmentation condition, only BEE boosted the performance. The application of saliency-based image processing strategies was verified to be beneficial to object recognition in daily scenes under simulated prosthetic vision. They are hoped to help the development of the image processing module for future retinal prostheses, and thus provide more benefit for the patients.

Object recognition based on the Region of Interest and optimal Bag of Words model

Bag of Words (BoW) model has been widely used in conventional object recognition tasks. Different from the existing methods, this paper proposed a method for object recognition based on Region of Interest (ROI) and Optimal Bag of Words model. It includes the following steps: (1) ROI extraction in combination with the Shi–Tomasi corner and Itti saliency map; (2) The SIFT feature descriptors are detected and described on images of interest; (3) A visual codebook is generated through utilizing the Gaussian mixture models, which relies on the clustering results of k-means++; (4) The similarities between each visual word and corresponding local feature are computed by posterior pseudo probabilities discriminative to construct a visual word soft histogram for image representation; (5) The Support vector machine (SVM) is used to perform image classification and recognition. The experiments are performed on the MSRC 21-class database. The results show that the proposed method can be more accurately recognize images.

Saliency Inspired Full-Reference Quality Metrics for Packet-Loss-Impaired Video

This paper explores the application of saliency information for perceptual quality assessment of packet-loss-impaired videos. We propose and validate two categories of error measures for full-reference saliency based quality metrics. The first category uses a weighted average of pixel errors between original and distorted videos, where the weight at a pixel depends on its visual saliency determined by Itti's saliency detection method after motion information incorporated. Motivated by the observation that packet-loss induced errors often change the spatial-temporal visual attention and correspondingly the saliency map, the second category measures the spatial deviation in saliency values between the original and distorted videos, and the temporal variation of saliency map of the distorted video, and further uses the products of both measures. We combine multiple error measures from the previous two categories using stepwise linear regression analysis. The final combined model includes three factors and provides significant gain over using the best single factor and other non-saliency based measurements.