Binaural synthesis with head tracking has become a highly important tool in creating an auditory experience for users that is as similar as possible to their perception of sound in the real world. This is crucial in many different applications that rely on accurate representation of real or virtual sound environments, ranging from scientific research to gaming applications. The quality of the binaural synthesis process and the accuracy and naturalness of the produced result depend on numerous factors, ranging from the selection of suitable IMU sensor technology used for head tracking to the choice of methods for implementing binaural synthesis using generic or individual sets of Head-Related Transfer Functions (HRTF). This is especially significant in Virtual Reality (VR) systems with three or six degrees of freedom that fully immerse the users in the virtual world through visual and auditory stimuli. This paper presents the authors' experiences in addressing a variety of challenges that arose during the design of such spatial audio systems and the testing of sound source localization quality. Strong emphasis is put on the testing of sound source localization quality, where the position of the source dynamically changes in relation to the listener.