Modern batch processes develop toward higher levels, and devices usually work at normal conditions with generally very few faults. Therefore, fewer fault data are collected than normal data, which makes the normal and fault modes imbalanced, and the fault diagnosis model is incapable focusing on the minority of fault samples as much as on most normal samples, consequently leading to insufficient generalization ability for the model. For that, we propose an adaptive imbalance-robust graph embedding broad learning system (AI-RGEBLS) in this paper. Firstly, it achieves adaptive correction of unbalanced samples by niche technique and synthetic minority over-sampling technique (SMOTE) with improved Mahalanobis distance. Then, broad learning system (BLS) is regularized by graph embedding, and further introduced the L2,1 norm constraint, which makes it possible to enhance the robustness of the model while considering the local manifold information of the data in the feature extraction process. Finally, the incremental learning approach is applied to the model to avoid the disastrous forgetting problem caused by training the whole model from zero. The effectiveness of the proposed method is verified by penicillin fermentation process and semiconductor etching process. It can effectively improve the speed of model training and provide better fault diagnosis for imbalanced batch processes compared with the existing methods.