Solid-state lithium batteries are promising next-generation energy storage systems for electric vehicles due to their high energy density and high safety and require achieving and maintaining intimate solid-solid interfaces for lithium-ion and electron transport. However, the solid-solid interfaces may evolve over cycling, disrupting the ion and electron diffusion pathways and leading to rapid performance degradation. The development of solid-state lithium batteries has been hindered by the lack of fundamental understanding of the interfacial microstructure change over cycling and its relation to electrochemical properties. Herein, we prepared a quasi-solid-state lithium battery, 30%LiFePO4-55%Li1.5Al0.5Ge1.5(PO4)3-15%C| Li1.5Al0.5Ge1.5(PO4)3|Li, by spark plasma sintering, and employed it as a model system to reveal the microstructure evolution at the solid-solid interfaces with electrochemical performance of the batteries. The electrochemical assessment showed that the quasi-solid-state lithium battery exhibited a discharge specific capacity of about 150 mAh g-1 in the first 80 cycles and then experienced severe capacity attenuation afterward, accompanied by a gradual internal resistance increase. Scanning electron microscopy observation showed that more cracks were formed inside the solid-state electrolyte and at the solid-solid interfaces as the battery cycled from 10 to 67 and 157 cycles. Detailed microstructure and phase analysis by high-resolution transmission electron microscopy and selected area electron diffraction discovered that the crack formation and performance decay were mainly caused by (1) the volume change of the LiFePO4 composite cathode during cycling, (2) the grain expansion of the Li1.5Al0.5Ge1.5(PO4)3 solid-state electrolyte at its interface with lithium anode, and (3) the formation of a solid electrolyte interphase layer, comprising Li2CO3, LiF, and LiTFSI, at the cathode-solid-state electrolyte interface. These microstructure changes built up over repeated battery cycling, ultimately causing the structure collapse and battery failure. The microstructure evolution information is expected to guide the design of better structures and interfaces for solid-state lithium batteries.