Due to the recent high speed of railway, noise and vibration generated during operation has emerged as a major social problem. Specially, the internal noise of railway varies depending on various operating conditions such as speed, vehicle crossings, and tunnel entry, and has a great effect on the riding comfort of passengers. In this study, an evaluation method for binaural noise of train pass-by noise and a simple wave field model which can reflect the Doppler effect are examined. To reflect the binaural characteristics of train pass-by noise, binaural measurements were conducted for several types of train, different measurement distances and various measurement angles. The annoyances of measured binaural noises were analyzed using conventional sound pressure level, psychoacoustic metrics, and their rate of change. Also, the noise emitted from high-speed railways was measured and characterized. By its analysis result, moving sounds were synthesized considering an initial frequency and a frequency shift. The synthesized sounds were convolved to reflect binaural condition. Auditory experiments were conducted using the synthesized moving sound stimuli, and the effects of the Doppler frequency shift on annoyance of moving sound sources were obtained.