In order to develop an efficient angiosperm-genus classification system, we first collected petal image of Hibiscus, Orchis, and Prunus, by digital camera, and remove the backgrounds by region-growing method. Next, we proposed a novel feature-extraction method, which combined most abundant color index (MACI) and introduced the fractional Fourier entropy (FRFE). Third, we submitted the 41 features to a single-hidden layer feedforward neural-network (SLFN), with weight decay (WD) to avoid overfitting. The 10 × 10-fold cross validation showed our method achieved an overall accuracy of 98.92%. Without weight decay, the overall accuracy decreased to 95.50%. Our experiments validated that optimal decay factor is 0.1, and optimal number of hidden neurons is 15. This proposed method is excellent. It performs better than six state-of-the-art approaches and AlexNet. The weight decay helps to enhance generalization of our classifier.