Knowledge graphs have demonstrated significant impact in the power grid domain, facilitating various applications such as defect diagnosis and grid management. However, their reasoning capabilities have not been fully exploited. In this paper, we explore the utilization of knowledge graphs for power grid defect diagnosis. We construct an electrical equipment defect knowledge graph and predict missing links, which is also known as Knowledge Graph Completion (KGC). However, we notice the long-tail problem in electrical equipment knowledge graph. To tackle this challenge, we propose a novel text-based model named SPALME (Structure Prompt Augmented Language Model Embedding) that incorporates structural information as prompts. Our model leverages the power of pre-trained language models, allowing it to comprehend the semantic information of entities and relationships in the knowledge graph. Additionally, by integrating structural information as prompts during the learning process, our model gains a deeper understanding of the graph’s topological structure efficiently, effectively capturing intricate dependencies between grid equipments. We evaluate our approach on various datasets. The results demonstrate that our model consistently outperforms baseline methods on the majority of the datasets.