Financial penalties for delayed enrollment could be useful tools to encourage people to enroll earlier in health insurance markets, but little is known about how effective they are. We use a large administrative dataset for a 10% random sample of all Australian tax-filers to study how people respond to a step-wise age-based penalty, and whether the effect has changed over time. Individuals must pay a 2% premium surcharge for each year they delay enrollment beyond age 31. The penalty stops after 10years of continuous hospital cover. The age-based penalty creates discontinuities in the incentive to insure by age, which we exploit to estimate causal effects. We find that people respond as expected to the initial age-penalty, but not to subsequent penalties. The 2% premium loading results in a 0.78-3.69 percentage points (or 2.1%-9.0%) increase in the take-up rate at age 31. We simulate the penalty impact and implications of potential reforms, and conclude that modest changes around the policy make little difference in the age distribution of insured, premiums or take-up rates. Our study provides important evidence on an understudied area in the literature and offers insights for countries considering financial penalties.