This study investigates how interest rate deregulation affects firms' financing choice between bank debt and public debt. Our analysis exploits China's 2013 bank interest rate floor deregulation as an exogenous shock to the supply of bank credit. Using a difference-in-difference design, we find that firms with higher default risk substitute away from bank loan and switch to public debt after the 2013 deregulation. However, this substitution to public debt is limited, leading to a dramatic decline in debt ratio. Our result also demonstrates that the effect on firms' public debt financing is more pronounced for firms with better information environments, suggesting that good information environment is an important prerequisite for making the switch. This switching, contradicting to traditional financing framework that high-risk firms prefer bank loans, inevitably is costly. Compared with low-risk firms, bonds issued by high-risk firms have significantly higher spreads, a higher likelihood of being secured, and a higher tendency of including an interest-adjusted clause. More importantly, we also document that high-risk firms subsequently improve their information transparency after the interest rate deregulation. Our findings highlight the role of interest rate deregulation in firms' financing choice and illustrate that firms incur high switching costs when their choice deviates from the optimal financing choice.