Defensive responses to visually threatening stimuli represent an essential fear-related survival instinct, widely detected across species. The neural circuitry mediating visually triggered defensive responses has been delineated in the midbrain. However, the molecular mechanisms regulating the development and function of these circuits remain unresolved. Here, we show that midbrain-specific deletion of the transcription factor Brn3b causes a loss of neurons projecting to the lateral posterior nucleus of the thalamus. Brn3b deletion also down-regulates the expression of the neuropeptide tachykinin 2 (Tac2). Furthermore, Brn3b mutant mice display impaired defensive freezing responses to visual threat precipitated by social isolation. This behavioral phenotype could be ameliorated by overexpressing Tac2, suggesting that Tac2 acts downstream of Brn3b in regulating defensive responses to threat. Together, our experiments identify specific genetic components critical for the functional organization of midbrain fear-related visual circuits. Similar mechanisms may contribute to the development and function of additional long-range brain circuits underlying fear-associated behavior.