Sound categorization is automatic, yet very little is known about how this process works. Physical sound sources such as musical instruments generate sounds that carry timbral information about two mechanical components: The excitation sets into vibration the resonator, which acts as a filter to amplify, suppress, and radiate sound components. Given that excitation–resonator interactions are quite limited in the physical world, Modalys, a digital, physically inspired modeling platform, was utilized to simulate the combinations of three excitations (bowing, blowing, striking) and three resonators (string, air column, plate). This formed nine types of interactions, which are either typical (e.g., struck string) or atypical (e.g., blown plate). In two separate categorization tasks, participants chose either the excitation or resonator they thought produced each interaction. For the typical interactions, participants accurately categorized their excitations and resonators. Atypical interactions were assimilated to typical ones and listeners identified either the correct excitation or the correct resonator but not both. Hierarchical clustering revealed that interactions were perceived differently depending on the categorization task. These findings suggest that unfamiliar sound sources are interpreted as conforming to familiar sound sources for which mental models exist. These studies consequently exemplify the role of timbre in sound source recognition.