The interfacial compatibility between inorganic powder and polymeric matrix is one of the most important key factors for the applications of polymer ceramic composite electrolytes (PCCEs). In the present study, a surface modification strategy has been proposed by using three different terminating groups of silane coupling agents in the modification of the interfacial compatibility between LLZO and PEO in PEO/LLZO composite electrolyte. Experiments and density functional theory (DFT) calculations have been performed in studying the effect of different silane coupling agents on interfacial properties. Especially, the interface compatibility is found to be improved when using amino-terminated silane coupling agent KH550, which is due to hydrogen bond interactions between amino groups and PEO. In addition, the mechanical strength and conductivity are significantly increased by increasing the interaction energy and by decreasing the crystallinity, and the values of 1.39 MPa and 0.22 mS cm−1 at 30 °C are then successfully achieved. As a result, the solid-state LiFePO4|PCCE|Li battery demonstrates a high discharge specific capacity of 157 mAh g−1 at 60 °C and 1C. And it has a cycle capacity decay rate of only 0.034 % for 380 cycles. These results pave the way for future developments of high-performance PCCEs.