Attention-aware perceptual enhancement nets for low-resolution image classification

Abstract

Classifying low-resolution (LR) images is notoriously challenging because of their noisy representation and limited information. Existing approaches mainly solve this challenge by training carefully designed architectures on LR datasets or by employing an image-resizing algorithm in a straightforward manner. However, the performance improvements of these methods are usually limited or even trivial in the case of LR images. In this work, we address the LR image classification problem by developing an end-to-end architecture that internally elevates representations of an LR image to “super-resolved” ones. This approach imparts characteristics similar to those of high-resolution (HR) images and is thus more discriminative and representative for image classification. For this purpose, we propose an innovative unified framework, named Attention-aware Perceptual Enhancement Nets (APEN), which integrates perceptual enhancement and an attention mechanism in an end-to-end manner for LR image classification. Specifically, the framework includes a perceptual enhancement network to generate super-resolved images from LR images. In addition, a novel attention mechanism is presented to highlight informative regions, while restricting the semantic deviation of super-resolved images. Additionally, we designed a feature rectification strategy to promote the adaptability of category decision. Experiments conducted on publicly available datasets demonstrate the superiority of our method against state-of-the-art methods on both LR and HR datasets.