The prediction of annoyance due to railway transportation in residential areas is crucial to correctly dimension noise and vibration reduction solutions. In situ annoyance assessment for multiple exposition scenarios requires to convince many residents to take annoyance tests combined with noise and vibration measurements inside their homes. Such approach is not adapted to large-scale experiments. Therefore, the VibAcouPhysio project aims at studying annoyance due to railway noise and vibration in laboratory conditions. A vibrating platform combined with a high-quality sound reproduction system has been designed for this project to reproduce the noise and vibrations of a train pass-by. Annoyance is estimated by multiple means including questionnaires, cognitive tasks performance assessment and various physiological measurements. Twenty participants were exposed to various train pass-by signals as if they were seating at a table inside their housing. The results show a significant correlation between physiological metrics, the annoyance evaluated in the questionnaires and the physical indicators derived from the noise and vibration signals. The main purpose of the VibAcouPhysio project is to rank various physical indicators describing noise and vibration signals according to their correlation to the perceived or measured annoyance.