In this article, a new fault diagnosis problem is formulated, which involves a large number of normal samples and in which almost all the fault classes are few-shot classes. Although this problem is common in many industrial scenarios, it remains a challenge overlooked in previous studies. To develop a novel solution for addressing this challenge, we employ long-tailed distribution in this work and name this new task the long-tailed fault diagnosis accordingly. Specifically, we divide the long-tailed fault diagnosis procedure into representation learning and classification. On this basis, we propose a method using progressively balanced supervised contrastive learning (PBS-SCL) for representation learning and a learnable linear classifier (LLC) for classification. The designed scheme consists of two phases. In the first phase, PBS-SCL is utilized to learn a more discriminative deep representation. In the second phase, an LLC is combined with the learned representation for better classification. Experiments are conducted on both the Tennessee Eastman process (TEP) benchmark dataset and a practical plasma etching process dataset. The results obtained show that the proposed method achieves significantly improved long-tailed fault diagnosis performance compared with existing methods.