This study aims to assess (i) which acoustic characteristics of snoring sounds are associated with the annoying nature of snoring sounds; (ii) whether listeners' heart rates correlate with their perceived annoyance; and (iii) whether perceived annoyance is different between listeners with different experiences with their bedpartners' snoring sounds. Six snoring epochs with distinct acoustic characteristics (viz., reference, high pitch, high intensity, short interval, irregular intensity, and irregular intervals) were collected from snoring patients. Twenty physicians and technicians were involved in the healthcare of snoring patients, and were divided into three groups based on personal experience with their bedpartners' snoring sounds (viz., non-snoring, snoring but not annoying, and snoring and annoying). The test subjects listened to each epoch and rated its level of annoyance. Listeners' heart rates were also recorded during the test using a finger plethysmograph. Within the limitations of this study, it was found that, compared with other snoring sounds, snoring sounds with high intensity and irregularity were associated with higher perceived annoyance. However, higher perceived annoyance of snoring sound was not reflected in heart rate-related parameters. In addition, listeners' personal experiences do not seem to affect their perceived annoyance.