A Mathematical Approach to Correlating Objective Spectro-Temporal Features of Non-linguistic Sounds With Their Subjective Perceptions in Humans.

Thomas Burns,Ramesh Rajan

doi:10.3389/fnins.2019.00794

Abstract

Non-linguistic sounds (NLSs) are a core feature of our everyday life and many evoke powerful cognitive and emotional outcomes. The subjective perception of NLSs by humans has occasionally been defined for single percepts, e.g., their pleasantness, whereas many NLSs evoke multiple perceptions. There has also been very limited attempt to determine if NLS perceptions are predicted from objective spectro-temporal features. We therefore examined three human perceptions well-established in previous NLS studies (“Complexity,” “Pleasantness,” and “Familiarity”), and the accuracy of identification, for a large NLS database and related these four measures to objective spectro-temporal NLS features, defined using rigorous mathematical descriptors including stimulus entropic and algorithmic complexity measures, peaks-related measures, fractal dimension estimates, and various spectral measures (mean spectral centroid, power in discrete frequency ranges, harmonicity, spectral flatness, and spectral structure). We mapped the perceptions to the spectro-temporal measures individually and in combinations, using complex multivariate analyses including principal component analyses and agglomerative hierarchical clustering.

Highlights

The objective features of sensory stimuli form a large part of our subjective perceptions, e.g., a chemical’s structure relates to our perception of its odor (Castro et al, 2013) and the wavelength of light being reflected from an object influences our perception of its color (Solomon and Lennie, 2007)
Accuracy of Naming appeared to depend on Complexity, it was retained in further analysis, given the novelty and non-intuitive nature of this relationship
Results for the agglomerative hierarchical clustering (AHC) analysis using principal component (PC) derived from the spectral principal component analysis (PCA) were similar to those from the raw spectral AHC – there were differences at multiple cluster levels for Accuracy of Naming and Complexity, but not for Pleasantness

Summary

Introduction

The objective features of sensory stimuli form a large part of our subjective perceptions, e.g., a chemical’s structure relates to our perception of its odor (Castro et al, 2013) and the wavelength of light being reflected from an object influences our perception of its color (Solomon and Lennie, 2007). Almost all of these studies focused on only a single percept and outside of special sets or precepts of sounds such as musical timbre (Grey, 1977; Grey and Gordon, 1978), urgency (Momtahan, 1991; Hellier et al, 1993; Burt et al, 1995; Edworthy et al, 1995; Haas and Edworthy, 1996; Graham, 1999), and identification of materials, e.g., the length of a material being struck and whether it is made of metal or wood (Warren and Verbrugge, 1984; Lakatos et al, 1997), tones (Pollack and Ficks, 1954), or subjects, e.g., the gender of a human walker (Li et al, 1991), little is known generally about the perceptual mappings between complex auditory stimuli and their objective features

Methods

Results

Discussion

Conclusion