Following a major disaster, most of the biological characteristics of the victims are typically destroyed, which increases the difficulty of forensic identification. However, palatal rugae cannot be easily destroyed owing to its unique location characteristics, owing to which it has become a new type of biological feature used in forensic identification under special environments. This study proposes a new method of 3D palatal rugae recognition which uses fractional Fourier transform (FRFT). The isometric slicing operation is first used to simplify the complexity of the algorithm and improve the efficiency of the 3D data storage and expression. FRFT is the used to extract the fractional domain features of the profile images obtained by slicing the 3D palatal rugae, and the support vector machine recursive feature elimination (SVM-RFE) method is used for feature selection in the fractional domain. The use of FRFT increases the number of the palatal rugae features extracted from the time-frequency domain, and can improve the effectiveness of feature extraction. The recognition accuracy rate of the proposed 3D palatal rugae recognition system can reach up to 97.8%. The proposed method meets the requirements of 3D palatal rugae recognition, and can help in performing rapid forensic identification. The method proposed in this study is useful in reducing the dependence on human experts and promoting the digitization of a forensic identification system based on palatal rugae.