For large rotating machinery with low speed and heavy load, the incipient fault characteristics of rolling bearings are particularly weak, making it difficult to identify them effectively by direct signal processing methods. To resolve this issue, we propose a novel approach to detecting incipient fault features that combines signal energy enhancement and signal decomposition. First, the structure of a conventional Teager algorithm is modified to further increase the energy of the micro-impact component and hence the impact amplitude. Then, a kind of composite chaotic mapping is constructed to extend the original fruit fly optimization algorithm (FOA) framework, improving the FOA’s randomness and search power. The effective intrinsic mode functions (IMFs) are determined by searching for the optimal combination values of the key parameters of the variational mode decomposition (VMD) with the improved chaotic FOA (ICFOA). The kurtosis index is then used to select the IMFs that are most relevant to the fault characteristics information. Finally, the sensitive components are analyzed to identify multiple early fault characteristics and determine detailed information about the faults. Moreover, the approach is evaluated by a simulation signal and a measured signal. The comprehensive evaluation indicates that the approach has clear advantages over other excellent methods in extracting the incipient fault feature information of the equipment and has great potential for application in engineering.