The use of Unmanned Aircraft Systems (UAS) for commercial applications is seeing steady growth in numerous sectors as the technology improves and more applications are realised. Whilst research into human response to UAS noise is a relatively new field, there is a far greater understanding of the perception and impacts of conventional aircraft noise. Therefore, a listening experiment was designed to investigate the relationship between perception of conventional aircraft and UAS flyover noise. The experiment used mono ground plate recordings of flyovers from several different aircraft and UAS, these recordings were re-spatialised to simulate flyovers over a multi-channel loudspeaker system. Participants were then asked to rate the annoyance of the sounds through a series of A/B comparisons. Results of the experiment demonstrate a clear positive correlation between Loudness and annoyance. However, Sound Quality Metric analysis also suggests that Sharpness of the UAS sound and Tonality and/or Roughness of aircraft sounds can also be a significant factor for annoyance. The finding of this experiment may contribute to the understanding and development of exposure-response relationships for UAS noise, building upon the extensive existing evidence on human response to conventional aircraft noise.