Sediment (S) and particulate phosphorus (PP) inputs from soil erosion of arable land into surface waters contribute substantially to water pollution. We have mapped erosion and S input to water bodies after each erosive rainfall event on 203 arable fields in the Swiss Midlands over 20 years (=4060 field years) and quantified soil loss as well as S and PP input to water bodies. On average, 21% of the total soil loss reached water bodies. A total of 353 (9%) field years provided S and PP inputs. The mean rates of S and PP input were 0.10 t ha−1 yr−1 and 0.14 kg ha−1 yr−1, and maximum values for single events were 10.71 t ha−1 and 12.96 kg ha−1, respectively. Of all investigated fields, 61% were connected to water bodies, but 88% of the total S and PP input occurred indirectly via drainage pipes with inlet shafts from roads and farm tracks. The temporal and spatial variability of S and PP input to water bodies was high. Only 7 (3%) out of 203 fields contributed to more than 50% of the total S input, and the largest 10% of field year magnitudes contributed to 73% of the total observed S input. Linear structures, such as thalwegs, traffic lines, tramlines, furrows or compacted headlands, were important pathways. More than 90% of the S and PP input resulted from linear erosion features. A wider application of conservation tillage practices in the second 10-year phase of the study led to a 5–6 fold reduction in S and PP inputs into water bodies in comparison to the first 10-year phase. With the presented unique data set, a sound basis for modelling and model validation is available. Moreover, such long-term studies are suitable to evaluate the effectiveness of mitigation measures and programmes.