In these years, the task of fast unsupervised feature selection attracts much attentions with the increasing number of data collected from the physical world. To speed up the running time of algorithms, the bipartite graph theory has been applied in many large-scale tasks, including fast clustering, fast feature extraction, etc. Inspired by this, we present a novel bipartite graph based fast feature selection approach named Efficient Unsupervised Feature Selection (EUFS). Compared to the existing methods focusing on the same topic, EUFS is advanced in two aspects: (1) we learn a high-quality discrete indicator matrix for these unlabelled data by virtue of bipartite graph based spectral clustering, instead of obtaining an implicit cluster structure matrix; (2) we learn a row-sparse matrix for evaluating features via a generalized uncorrelated regression model supervised by the achieved indicator matrix, which succeeds in exploring the discriminative and uncorrelated features. Correspondingly, the features selected by our model could achieve an excellent clustering or classification performance while maintaining a low computational complexity. Experimentally, the results of EUFS compared to five state-of-the-art algorithms and one baseline on ten benchmark datasets verifies its efficiency and superiority.