In real-world fault diagnosis tasks, it is never possible to enumerate all fault types beforehand since there are always unseen situations that may arise unexpectedly. Therefore, fault diagnosis is essentially an open-world recognition task. A desirable open-world fault diagnosis model must be able to perform open-set recognition (OSR) and incremental learning (IL). Classifier calibration and prototype replay are popular in two subtasks. However, existing calibration and replay strategies suffer from tampering with raw data and ignoring resource allocation, respectively. In this work, we formulate an open-world fault diagnosis scheme to address both issues. Firstly, we calibrate the uncertainty of the classifier’s prediction by assigning soft labels to the samples based on their distance from the class center. Then, Shannon entropy is employed to quantify the uncertainty as an estimate of the probability that the sample belongs to the unknown. Secondly, we adaptively manage the memory of old and new known classes according to the training accuracy. Then, based on this memory budget, a dissimilarity-based sparse subset selection algorithm is utilized to pick more diverse exemplars as prototypes for replay. Furthermore, we extend the open-set F-Measure (OSFM) to the weighted OSFM to make it applicable even when the classes are imbalanced. Experimental results on multiphase flow facility and wastewater treatment plant demonstrate the feasibility and superiority of the proposed method.