Monitoring and estimating drought impact on plant physiological processes over large regions remains a major challenge for remote sensing and land surface modeling, with important implications for understanding plant mortality mechanisms and predicting the climate change impact on terrestrial carbon and water cycles. The Orbiting Carbon Observatory 3 (OCO-3), with its unique diurnal observing capability, offers a new opportunity to track drought stress on plant physiology. Using radiative transfer and machine learning modeling, we derive a metric of afternoon photosynthetic depression from OCO-3 solar-induced chlorophyll fluorescence (SIF) as an indicator of plant physiological drought stress. This unique diurnal signal enables a spatially explicit mapping of plants' physiological response to drought. Using OCO-3 observations, we detect a widespread increasing drought stress during the 2020 southwest US drought. Although the physiological drought stress is largely related to the vapor pressure deficit (VPD), our results suggest that plants' sensitivity to VPD increases as the drought intensifies and VPD sensitivity develops differently for shrublands and grasslands. Our findings highlight the potential of using diurnal satellite SIF observations to advance the mechanistic understanding of drought impact on terrestrial ecosystems and to improve land surface modeling.