This study presents a novel approach for real-time vision-based structural health monitoring, focusing on evaluating the deformation state of lattice structures. The structures are renowned for their remarkable recovery capabilities and exhibit similar mechanical responses under compressive loads. Despite these characteristics, quickly assessing the health status of the applied structure using knowledge from another related lattice structure is usually time-consuming and impractical. To address this, we propose to combine Gaussian process classification with transfer learning, termed TL-GPC, to detect damage states under compressive loads while also achieving uncertainty quantification. By employing structural deformations captured via the optical flow algorithm as inputs, the internal transfer kernel factor in TL-GPC is tailored to model knowledge transfer between source and target domain inputs. Experimental results show that the proposed TL-GPC model can deliver higher damage detection accuracy while ensuring stable uncertainty quantification in scenarios with limited and unbalanced experimental data.