Industrial fault diagnosis often faces challenges from insufficient examples. Methods leveraging Generative Adversarial Network or transfer learning address this problem. However, the model trained by the labeled examples of one component is not applicable to classify the new fault categories of other components. This problem aggravates when there exist very few examples. In this paper, we propose a cross-category fault diagnosis method (CFDM) based on few-shot learning. Our method constructs a convolutional Siamese neural network to extract fault features from example pairs. A cross-entropy based loss function is used that includes parameters for feature discrepancy to maximize the inter-category distances and minimize the intra-category distances. This enables the proposed method to learn the accurate classification boundaries between fault features of the example pairs. We conduct experiments on two public benchmark datasets and one lab-built dataset. Our evaluation includes analysis of the proposed method to classify fault types with one or five examples in each category of the target component. Our results demonstrate that the proposed method improves the fault diagnosis accuracy and robustness in comparison to the state-of-the-art methods.