Scene classification is an important basis for many modern intelligent applications, however the performance of pattern recognition or deep learning-based methods are still not sufficient since complicated structure and context of scene images. In this paper, we propose a novel fusion framework of adaptive nonnegative feature fusion (AdaNFF) for scene classification. The AdaNFF integrates nonnegative matrix factorization, adaptive feature fusion and feature fusion boosting into an end-to-end process. Firstly, feature fusion is known as a general strategy to strengthen weak features, and we observe that pixel values and most hand-craft features of the scene image are naturally nonnegative. Therefore we are motivated to build a fusion method based on nonnegative matrix factorization, which can preserve features nonnegative properties and improve their representation performance. Secondly, with the results of fused single or multiple features fusion, we develop an adaptive feature fusion and boosting algorithm to improve the efficiency of image features. Finally, a normalized l2-norm classifier and a deep-learning like multilayer perceptron (MLP) classifier are trained to predict label of scene image. Under this framework, there are two versions of the proposed feature fusion method for nonnegative single-feature fusion and multi-feature fusion. All methods were validated on scene classification benchmarks. Experiment results suggest that the proposed methods can deal with multi-class scene problems and achieve remarkable classification performance.