While it is well known that humans are highly responsive to musical rhythm, the factors that influence our innate ability to synchronize remain unclear. In the current study, we examined how stimulus complexity and modality, along with the synchronizer's level of musicality, impacted sustained sensorimotor synchronization (SMS). Utilizing a finger-tapping task and three sensory modalities (visual, auditory, and tactile), we manipulated rhythmic complexity by varying the location and availability of temporal cues across four conditions. Additionally, to determine our participants' (n = 30) musical experience and aptitude, we administered the Goldsmiths Musical Sophistication Index (Gold-MSI) questionnaire. We found that SMS to external rhythmic stimuli was significantly more precise for auditory and tactile than for visual sequences. Further, we found SMS precision significantly decreased in all modalities as rhythmic complexity increased, suggesting rhythmic complexity directly relates to SMS difficulty. Moreover, a significant correlation was found between Gold-MSI scores and SMS accuracy in the most rhythmically complex condition, such that the higher one's musicality score, the greater one's accuracy. This held for all three modalities. Combined, these findings suggest that rhythmic synchronization performance is affected not only by the modality and complexity of the rhythmic stimuli but also by the musicality of the synchronizer.