Aircraft noise remains a major problem for many people living near airports, despite the efforts of the various air traffic operators and aircraft manufacturers to reduce the nuisance. Noise abatement usually relies on quantitative approaches based on noise measurements at source or in residential areas. However, field studies have shown that the level of annoyance experienced within a given noise exposure zone varies widely, demonstrating that noise levels alone do not explain the experience of local residents. The present work brought together a wide range of disciplines (aeronautics, acoustics, linguistics, psychoacoustics, social and environmental psychology) to define a common framework for reconciling field and laboratory studies on aircraft noise annoyance. A survey of 1250 people living near airports enabled us to precisely qualify the annoyance experienced, and to identify 6 highly differentiated annoyance profiles based on non-acoustic factors. Focus groups and laboratory experiments with people living near airports revealed an impact of annoyance profiles on both qualitative and quantitative evaluation of noise annoyance, which confirms the diversity of their experiences in the same listening situation.