Spin-wave technology (magnonics) has the potential to further reduce the size and energy consumption of information-processing devices. In the submicrometer regime (exchange spin waves), topological defects such as domain walls may constitute active elements to manipulate spin waves and perform logic operations. We predict that spin waves that pass through a domain wall in an ultrathin perpendicular-anisotropy film experience a phase shift that depends on the orientation of the domain wall (chirality). The effect, which is absent in bulk materials, originates from the interfacial Dzyaloshinskii-Moriya interaction and can be interpreted as a geometric phase. We demonstrate analytically and by means of micromagnetic simulations that the phase shift is strong enough to switch between constructive and destructive interference. The two chirality states of the domain wall may serve as a memory bit or spin-wave switch in magnonic devices.