α5 subunit-containing γ-aminobutyric acid type A (GABAA) receptors represent a promising drug target for neurological and neuropsychiatric disorders. Altered expression and function contributes to neurodevelopmental disorders such as Dup15q and Angelman syndromes, developmental epilepsy and autism. Effective drug action without side effects is dependent on both α5-subtype selectivity and the strength of the positive or negative allosteric modulation (PAM or NAM). Here we solve structures of drugs bound to the α5 subunit. These define the molecular basis of binding and α5 selectivity of the β-carboline, methyl 6,7-dimethoxy-4-ethyl-β-carboline-3-carboxylate (DMCM), type II benzodiazepine NAMs, and a series of isoxazole NAMs and PAMs. For the isoxazole series, each molecule appears as an ‘upper’ and ‘lower’ moiety in the pocket. Structural data and radioligand binding data reveal a positional displacement of the upper moiety containing the isoxazole between the NAMs and PAMs. Using a hybrid molecule we directly measure the functional contribution of the upper moiety to NAM versus PAM activity. Overall, these structures provide a framework by which to understand distinct modulator binding modes and their basis of α5-subtype selectivity, appreciate structure–activity relationships, and empower future structure-based drug design campaigns.