Rotor broadband noise spectra are typically analyzed over time scales on the order of one or more rotor periods. However, modulation of the broadband noise spectrum with the blade passage frequency (BPF) has been shown to be significant for noise levels and perception of wind turbines and helicopters. In contrast, time-varying broadband noise has not been extensively studied for aircraft with many rotors, such as unmanned aerial vehicles (UAVs) or advanced air mobility aircraft. In this work, significant broadband noise modulation was measured in flight and anechoic chamber tests of hexacopter UAVs at various observer angles. This modulation is aperiodic with the BPF such that the modulation amplitude varies substantially between blade passages, even when the BPFs are controlled to be nearly constant between all rotors at all times. Furthermore, the azimuthal phasing between rotors greatly affects the measured modulation, such that the modulation of multiple rotors may be less than or greater than for a single rotor, depending on the phase offsets. The effects of phase variations on acoustic interactions between rotors is studied by comparing the sum of the modulation of individual rotors to the modulation of those rotors operating simultaneously. This is done not only using measurements, but also noise predictions made using PSU-WOPWOP. These results contribute understanding to how the noise modulation of rotors sum together, including the resulting directivity and aperiodicity.