Impact locations play a critical role in determining the residual performance of the composite stiffened panel, since the local damage behavior at different locations is not identical. The objective of this paper is to systematically investigate the influence of impact locations on damage behavior, stability and residual compression strength of T-shaped stiffened CFRP panels. Low-velocity impact tests are carried out by impacting at the skin center or the flange tip. The damage patterns and post-buckling behavior are monitored by ultrasonic C-scan and digital image correlation. An effective computational framework with explicit formulations considering complicated damage mechanisms is proposed to investigate the details of damage behavior. The skin center impact causes severe fiber breakage and matrix squeezing, while the flange tip impact induces complex damage patterns including ply splitting, fiber breakage, matrix cracks, as well as extensive skin–rib interfacial delamination. Compression after impact results indicate that impact damage at the skin center has little effect on the residual performance of the structure, and the compression failure load is only reduced by 2.02%, from 276.8 kN to 271.2 kN. However, impact damage at the flange tip weakens the supporting effect of the stiffener, leading to early buckling of the local skin. The propagation of the interfacial delamination at the damaged flange tip directly triggers the premature compression failure, resulting in an 18.79% decrease in the failure load, 224.8 kN. Numerical predictions correlate well with experimental data. The reported results may be useful in structural design and maintenance.