The early fault impulses of rolling bearing are often submerged by harmonic interferences and background noise. In this paper, a fault diagnosis scheme called probabilistic principal component analysis assisted optimal scale average of erosion and dilation hat filter (OSAEDH-PPCA) is presented for the fault detection of rolling bearing. Based on morphological erosion operator and morphological dilation operator, a new morphological top-hat operator, namely average of erosion and dilation hat (AEDH) operator is firstly proposed to extract the fault impulses in the vibration signal. Simulation analysis shows the filter characteristics of proposed AEDH operator. Comparative analyses demonstrate that the feature extraction property of the AEDH operator is superior to existing top-hat operators. Then, the probabilistic principal component analysis is introduced to enhance the filter property of AEDH for highlighting the fault feature information of rolling bearing further. Experimental signals collected from the test rig and the engineering are employed to validate the availability of proposed method. Experimental results show that the OSAEDH-PPCA can effectively extract the early fault impulses from vibration signal of rolling bearing. Comparison results verify that the OSAEDH-PPCA has advantage in early fault detection of rolling bearing than other morphological filters in existence.