There is an urgent demand to develop a portable and user-friendly biosensing platform for accurately and rapidly detecting biomarkers in clinical diagnosis. In this work, a photothermal and smartphone-based colorimetric dual-mode biosensing platform was proposed for trypsin assay based on the localized surface plasmon resonance (LSPR) effect of gold nanoparticles (Au NPs). Two competing processes were presented between the protection and the destabilization of dispersed Au NPs, respectively. Bovine serum albumin (BSA) can restrain the Au NPs aggregation due to the surface protection of Au NPs in the salt solution. In the presence of trypsin, the Au NPs are aggregated due to the enzymatic digestion of surface BSA ligand and electrostatic interaction, resulting strong photothermal effect and obvious RGB (Red/Green/Blue) value change (color from red to blue). The whole process can be performed within 1 h and the point-of-care testing (POCT) was realized by using a thermometer and smartphone as signal recorders. The limits of detection (LODs) were calculated to be 6 µg mL−1 by photothermal sensing and 1.2 µg mL−1 by RGB recognition. This biosensing platform was successfully used for trypsin assay in clinical acute pancreatitis samples and showed great consistency with clinical diagnostic results, validating the potential POCT application.