It is a big challenge to identify the most effective features for enhancement of fault classification accuracy in rotating machines due to nonstationary and nonlinear vibration characteristics of the machines under varying operating conditions. To find discriminative features, a novel dimension reduction algorithm, referred to as the nearest and farthest distance preserving projection (NFDPP), is proposed for machine fault feature extraction and classification. With the NFDPP, both the nearest and farthest samples of the data manifold can be analyzed simultaneously to identify features leading to fault classification. Additionally, we denoise the features directly in the feature space to save computation time and storage space, and prove its equivalence to denoising the signals in the time domain. Through analysis of measured vibration data for bearings with different defects, it is demonstrated that the proposed NFDPP approach can effectively classify different bearing faults and identify the severity of the bearing ball defect, and the direct denoising of features yield a significant improvement in fault classification. The effectiveness of the proposed method is further validated in identifying compound faults in locomotive bearings in an industrial setting.