Many migratory birds have changed their timing of arrival at breeding grounds in response to recent climate change. Understanding the adaptive value and the demographic consequences of these shifts are key challenges. To address these questions we extend previous models of phenological adaptation to climate change under territory competition to include feedback from population dynamics, winter survival and habitat productivity. We study effects of improved pre-breeding survival and of earlier food abundance peak. We show that phenological responses depend strongly on equilibrium population density via effects on territory competition. When density is high, improved pre-breeding survival affects selection pressures more than shifts of the resource peak. Under certain conditions, an advanced food peak can even select for later arrival due to competitive release. Improved pre-breeding survival has positive effects on population density that in many cases is stronger than negative effects of an advanced food peak. The fraction of young in the population decreases in all scenarios of change, but food peak shifts only affect population structure marginally unless population density is low. This work thus provides several missing links between phenological adaptation and demographic responses, and augments the toolbox for interpreting ongoing phenological shifts in migratory birds. We illustrate the utility of our model by explaining different patterns in demographic trends and phenological shifts in populations of Pied flycatchers (Ficedula hypoleuca) across Western Europe.