Instrument Sounds Research Articles

Liszt’s Representation of Instrumental Sounds on the Piano: Colors in Black and White by Hyun Joo Kim

Liszt’s Representation of Instrumental Sounds on the Piano: Colors in Black and White by Hyun Joo Kim

Use of Speaker Recognition Approaches for Learning and Evaluating Embedding Representations of Musical Instrument Sounds

Constructing an embedding space for musical instrument sounds that can meaningfully represent new and unseen instruments is important for downstream music generation tasks such as multi-instrument synthesis and timbre transfer. The framework of Automatic Speaker Verification (ASV) provides us with architectures and evaluation methodologies for verifying the identities of unseen speakers, and these can be repurposed for the task of learning and evaluating a musical instrument sound embedding space that can support unseen instruments. Borrowing from state-of-the-art ASV techniques, we construct a musical instrument recognition model that uses a SincNet front-end, a ResNet architecture, and an angular softmax objective function. Experiments on the NSynth and RWC datasets show our model’s effectiveness in terms of equal error rate (EER) for unseen instruments, and ablation studies show the importance of data augmentation and the angular softmax objective. Experiments also show the benefit of using a CQT-based filterbank for initializing SincNet over a Mel filterbank initialization. Further complementary analysis of the learned embedding space is conducted with t-SNE visualizations and probing classification tasks, which show that including instrument family labels as a multi-task learning target can help to regularize the embedding space and incorporate useful structure, and that meaningful information such as playing style, which was not included during training, is contained in the embeddings of unseen instruments.

Tiny, Tiny Copyright: An Examination of the Copyrightability of Sampler Instruments and Its Impact on Derivative Works

Tiny, Tiny Copyright: An Examination of the Copyrightability of Sampler Instruments and Its Impact on Derivative Works

Sonic Visualiser: Visualisation, Analysis, and Annotation of Music Audio Recordings

Sonic Visualiser: Visualisation, Analysis, and Annotation of Music Audio Recordings

Linguistic roles of surrogate language in the select literary texts

Language has verbal and nonverbal components and both of them complement each other in our daily communication. The former exists in spoken form and the latter uses body movement, musical instruments and sign language to convey meaning. Surrogate language is synonymous with nonverbal communication. This paper investigates the linguistic roles of surrogate language in the select literary texts. The study explores different media of surrogate language and states the roles of surrogate language in literary texts. This study adopts Katz, Blumler and Gurevitch’s Uses and Gratifications Theory. This theory states that audience use the mass media for their own purposes such as information, entertainment, messages, announcements and advertisements. The researcher selected seven literary texts using non-probability sampling method with particular reference to purposive sampling. The researcher made use of both primary and secondary sources such as textbooks, journal articles, theses and dictionaries. The author read, selected and jotted down surrogate language for easy presentation and analysis. It has been found out that in surrogate language, blind people can hear the sounds of the musical instruments and understand, deaf people can see the eye contacts of the speakers without bothering about the speech.

Ordinal scaling of temporal audio descriptors and perceptual significance of attack temporal centroid in timbre spaces.

Temporal audio features play an important role in timbre perception and sound identification. An experiment was conducted to test whether listeners are able to rank order synthesized stimuli over a wide range of feature values restricted within the range of instrument sounds. The following audio descriptors were tested: attack and decay time, temporal centroid with fixed attack and decay time, and inharmonicity. The results indicate that these descriptors are susceptible to ordinal scaling. The spectral envelope played an important role when ordering stimuli with various inharmonicity levels, whereas the shape of the amplitude envelope was an important parameter when ordering stimuli with different attack and decay times. Linear amplitude envelopes made the ordering of attack times easier and caused the least amount of confusion among listeners, whereas exponential envelopes were more effective when ordering decay times. Although there were many confusions in ordering short attack and decay times, listeners performed well in ordering temporal centroids even at very short attack and decay times. A meta-analysis of six timbre spaces was therefore conducted to test the explanatory power of attack time versus the attack temporal centroid along a perceptual dimension. The results indicate that attack temporal centroid has greater overall explanatory power than attack time itself.

Blind Source Separation in Polyphonic Music Recordings Using Deep Neural Networks Trained via Policy Gradients

We propose a method for the blind separation of sounds of musical instruments in audio signals. We describe the individual tones via a parametric model, training a dictionary to capture the relative amplitudes of the harmonics. The model parameters are predicted via a U-Net, which is a type of deep neural network. The network is trained without ground truth information, based on the difference between the model prediction and the individual time frames of the short-time Fourier transform. Since some of the model parameters do not yield a useful backpropagation gradient, we model them stochastically and employ the policy gradient instead. To provide phase information and account for inaccuracies in the dictionary-based representation, we also let the network output a direct prediction, which we then use to resynthesize the audio signals for the individual instruments. Due to the flexibility of the neural network, inharmonicity can be incorporated seamlessly and no preprocessing of the input spectra is required. Our algorithm yields high-quality separation results with particularly low interference on a variety of different audio samples, both acoustic and synthetic, provided that the sample contains enough data for the training and that the spectral characteristics of the musical instruments are sufficiently stable to be approximated by the dictionary.

Developing Trends of Music in the Vedic and Mythological Eras

The Vedas are religious texts which inform the religion of Hinduism also known as Sanatan Dharma; meaning eternal order or eternal Path. The Vedic – mythological period is considered to be the golden era in the history of world literature. Not only did the philosophy of the age reach a new pinnacle; but even aspects of music, art, culture, literature, sculpture, religion, and spiritualism were extended to their highest point. Amongst these cultural instruments, Music represents vocal and instrumental sounds combined in such a way as to produce beauty of form, harmony and expression of emotions. During this era music – vocal and instrumental were held in high respect in society. Music had both ritual and secular aspects. Sāmaveda is considered as the root of Vedic music as well as the root of today's south Asian classical music. Sāmagāna was considered as the sound of inspiration for the people of that age. This paper attempts to discuss the musical situations in Vedic and Mythological periods regarding its origin, development, extension and practices in ancient south-eastern i.e. Hindu civilization. The paper also points out why the need and importance of Vedic music in present day society is much more; especially in regards to the adoption of lessons and ethics from Sanatan Hinduism.

Probing associations between interbrain synchronization and interpersonal action coordination during guitar playing

Playing music in an ensemble is a highly coordinated human action involving musicians' brain-body interactions at different levels of physical and cortical organization in time and space. It has been suggested that interbrain phase synchronization plays an essential role in musical interaction. In this study, we aimed to explore associations between interbrain synchronization and interpersonal action coordination, using electroencephalographic recordings of the brain activity of guitarists playing in a duet as well as the acoustic recordings of their music. By applying phase synchronization algorithms to the musicians' brain activities and the sounds produced during guitar playing, we show that synchronous brain activity is strongly related to instrument sounds and behavioral play-onset synchrony, as indicated by phase alignment in relation to the time differences in play onsets and an angular-linear correlation between phase and time differences across trials and guitarist pairs. Interestingly, this correlation was especially strong in the first part of the music piece, when the guitarists seem to adjust their coordinated play onsets and brain rhythms actively. This suggests that the methods capturing intra- and interbrain synchronization and its relations to coordinated playing provide crucial information about the underlying mechanisms of interpersonal action coordination.

Shift of musical hallucinations to visual hallucinations after correction of the hearing deficit in a patient with Lewy body dementia: a case report

BackgroundMusical hallucinations are a particular type of auditory hallucination in which the patient perceives instrumental music, musical sounds, or songs. Musical hallucinations are associated with acquired hearing loss, particularly within the elderly. Under conditions of reduced auditory sensory input, perception-bearing circuits are disinhibited and perceptual traces released, implying an interaction between peripheral sensory deficits and central factors related to brain dysfunction.Case presentationA 71-year-old Caucasian man with hearing loss complained of memory difficulties and resting tremor of the right upper limb in the previous 2 years. He already had difficulties in instrumental activities of daily life. Neurological examination showed Parkinsonian signs and hypoacusia. Neuropsychological examination identified deficits in executive functions and memory tests. Brain computerized tomography and nuclear magnetic resonance scans showed mild cortical and subcortical atrophy. The clinical diagnosis of possible dementia with Lewy bodies was established. Five years later, the patient began complaining of musical hallucinations. There had been no previous change in medication. An otorhinolaryngologist diagnosed age-related hearing loss and prescribed bilateral hearing aids. After using the hearing aids, the patient did not hear the songs any longer, only some tinnitus, described as a whistle. However, at the same time, the patient started experiencing visual hallucinations he never had before.DiscussionTo our knowledge, the immediate shift of hallucinations from one sensory modality to another sensory modality when perception is improved has not been previously described. This report emphasizes the interaction between brain pathology and sensory deficits for the genesis of hallucinations, and reinforces the theory that attention and control networks must couple properly to the default mode network, as well as integrate and select adequately peripheral signals to the somatosensory cortices, in order to keep a clear state of mind.ConclusionThe clinician should bear in mind and let the patient know that improving one sensory modality to ameliorate hallucinations may sometimes paradoxically lead to hallucinations in a different sensory modality.

Silence is Golden: Auditory Preferences in Zoo-housed Gorillas

ABSTRACT Enrichment is presented to improve the welfare of captive animals but sound is frequently presented with the assumption that it is enriching without assessing individuals’ preferences. Typically, presented sounds are unnatural and animals are unable to choose which sounds they can listen to or escape them. We examined preferences of three zoo-housed gorillas for six categories of sound. The gorillas selected unique icons on a computer touchscreen that initiated brief samples of silence, white noise, nature, animal, percussion, and electronic instrumental sounds. Following training, gorillas selected each sound paired with silence (Phase 2), each sound paired with each other sound (Phase 3), and one sound among all other sound categories (Phase 4). Initially, a single sound was associated with each icon, but additional exemplars of the category were added in phases 5–8. Preferences were generally stable and one gorilla showed a strong preference for silence. Although there were individual differences, a surprising general preference for unnatural over natural sounds was revealed. These results indicate the importance of assessing preferences for individuals before introducing auditory stimulation in captive habitats.

Improving speech characteristics of young adults with congenital dysarthria: An exploratory study comparing articulation training and the Beatalk method

IntroductionThis exploratory study compared the effects of two speech therapy approaches on speech characteristics of young adults with congenital dysarthria resulting from various etiologies: a) articulation training focusing on consonant articulation exercises at various levels (isolation, syllables, and words), and b) the Beatalk method, based on human beatboxing, i.e., producing various instrumental sounds in an a-cappella musical context. Both interventions were designed to increase participants’ speech intelligibility. MethodsTwelve adults with congenital dysarthria and reduced speech intelligibility participated in treatment groups for eight weeks. Six participants were assigned to the articulation training group, and six to the Beatalk group. Intelligibility of single words and continuous speech, voice measures, and oral-diadochokinesis rates were measured before and after the treatment. ResultsThe results showed that the Beatalk intervention yielded a significant overall pre- to post-treatment effect. Specifically, it resulted in gains in articulatory accuracy and intelligibility for single words. Improvements were not noted following articulation training. ConclusionsThe results present initial evidence of the positive effect of the Beatalk method as an intervention tool for adults with congenital dysarthria. This relatively easy-to-learn technique shows promise, as it involves intense and repetitive production of speech sounds while controlling rhythm and breathing in an enjoyable context.

Making media for learning musical instruments using the Scratch application

The challenge for music educators in distance learning is the inability to practice teaching musical instruments directly to students. This article aims to bridge music educators so that they can still teach musical instruments to students and the students can continue to learn music even though they are learning from home. The solution offered is learning music using the Scratch application on a laptop or desktop computer as the controller. There are five steps to make music learning media using the Scratch application: (1) downloading and installing the Scratch Desktop application, (2) making designs or icons for each music scale, (3) recording or inputting sound musical instruments, (4) setting the buttons for each musical scale, and (5) playing the musical compositions. The Scratch application was effective in making learning media for learning musical instruments, learning to recognize the sounds of musical instruments, and learning to memorize song melodies. In conclusion, the Scratch application can be a solution for creating computer-based learning media for musical instruments.

The Performative Dimension of a Literary Text-Concert

Using the premises of literary intermediality theory, the paper dwells on the particular performability of a literary text, in which a concert is not only described but also played out as a distinct socially significant phenomenon of contemporary culture. It is because of the performative dimension, which is an essential medium between actual musical events and their verbal rendition, between the embodiment of music in movements, gestures, sounds of instruments, program pieces, and its presentation in interpretations that are historically determined and motivated by personal poetologies of authors, that such texts could be treated as texts-concerts. It is stated that a literary text-concert thematizes and describes a concert event using specific names of musical pieces and their performers, drawing attention to the performing style and individual manner of expression used by musicians as well as the sound of musical instruments while interpreting specific musical compositions. The text itself is conceptualized as a concert, i.e., the plot may be framed by a concert event or be presented as a special concert event with a detailed description of the conditions of the performance (acoustic space, the drama, timing, atmosphere, acoustic background, relevant auditory habits, and performative and ritualistic elements of the concert) and their impact on the listener. Due to particular structural technologies aimed at the most adequate and vocal rendering of the musical performance, a text of this type is played out as a concert that exceeds the limits of representation and displays such dimensions as embodiment, mediality, or eventfulness. In a metaphorical sense, the text is a stage on which the musical performance plays out, which is often combined with the discourse on the author’s creative becoming. Since a text-concert is constituted through intermedial links with music, it has the potential of revealing its own mediality, showing the things it talks about, and in such a way transcending the limits of its content. The analysis is based on Julio Cortazar’s essay Around the Piano with Thelonius Monk and Friedrich Christian Delius’s The Future of Beauty. 

Spectral envelope position and shape in sustained musical instrument sounds.

It has been argued that the relative position of spectral envelopes along the frequency axis serves as a cue for musical instrument size (e.g., violin vs viola) and that the shape of the spectral envelope encodes family identity (violin vs flute). It is further known that fundamental frequency (F0), F0-register for specific instruments, and dynamic level strongly affect spectral properties of acoustical instrument sounds. However, the associations between these factors have not been rigorously quantified for a representative set of musical instruments. Here, we analyzed 5640 sounds from 50 sustained orchestral instruments sampled across their entire range of F0s at three dynamic levels. Regression of spectral centroid (SC) values that index envelope position indicated that smaller instruments possessed higher SC values for a majority of instrument classes (families), but SC also correlated with F0 and was strongly and consistently affected by the dynamic level. Instrument classification using relatively low-dimensional cepstral audio descriptors allowed for discrimination between instrument classes with accuracies beyond 80%. Envelope shape became much less indicative of instrument class whenever the classification problem involved generalization to different dynamic levels or F0-registers. These analyses confirm that spectral envelopes encode information about instrument size and family identity and highlight their dependence on F0(-register) and dynamic level.

Perceiving Sound Objects in the Musique Concrète.

In the late 1940s and early 1950s, there emerged a radically new kind of music based on recorded environmental sounds instead of sounds of traditional Western musical instruments. Centered in Paris around the composer, music theorist, engineer, and writer Pierre Schaeffer, this became known as musique concrète because of its use of concrete recorded sound fragments, manifesting a departure from the abstract concepts and representations of Western music notation. Furthermore, the term sound object was used to denote our perceptual images of such fragments. Sound objects and their features became the focus of an extensive research effort on the perception and cognition of music in general, remarkably anticipating topics of more recent music psychology research. This sound object theory makes extensive use of metaphors, often related to motion shapes, something that can provide holistic representations of perceptually salient, but temporally distributed, features in different kinds of music.

Analyzing spectral envelope position and shape in a large set of sustained musical instrument sounds

It has been argued that the relative position of spectral envelopes along the frequency axis serves as a cue for musical instrument size (e.g., violin versus viola) and that the shape of the spectral envelope encodes family identity (violin versus flute). However, the extent to which spectral envelope position and shape are affected by fundamental frequency (F0), F0- register within instruments, and dynamic level has never been rigorously quantified across a representative set of musical instrument sounds. Here, we analyzed sounds from 50 sustained orchestral instruments sampled across their entire range of F0s at three dynamic levels. Linear regression of spectral centroid (SC) values that index envelope position indicated that smaller instruments possessed higher SC values in most instrument classes, but SC also correlated with F0 and was strongly and consistently affected by the dynamic level. Instrument classification using relatively low-dimensional cepstral audio descriptors allowed for discrimination between instrument classes with accuracies beyond 80%. Envelope shape became much less indicative of instrument class whenever the classification problem involved generalization across dynamic levels or F0-registers. These analyses confirm that the spectral envelope encodes information about instrument size and family identity and highlight its strong dependence on F0 (-register) and dynamic level.

A model of cortical timbre classification using spike latency coding

Earlier modeling studies of spiking neural networks with conduction delays have shown that temporally precise, distributed spike patterns (polychronous groups) contain more information about timbral distinctions than firing rates. Additionally, experimental work in auditory cortex has suggested that the initial onset of firing activity is especially important for encoding timbre, an observation that is in accordance with psychoacoustical evidence that the onset characteristics of a stimulus can have a profound effect on the perception of timbre. These observations are combined in a model that uses spike latency to make timbral classifications. A simplified spiking model of the subcortical auditory system is connected to a reservoir of spiking point neurons with cortex-like connectivity statistics and conduction delays. Stimuli are presented to the model in an unsupervised manner, and synaptic strengths are adapted via a spike-timing-dependent plasticity mechanism. After training, a stochastic search algorithm is used to find stimulus-triggered firing patterns generated in the reservoir, the firing latency of which can be used to make accurate timbral categorizations of novel stimuli. The stimuli consist of FM-synthesized instrument sounds (two categories) with varying pitches, demonstrating the model’s ability to learn timbral distinctions in a manner invariant with respect to changes of pitch.

Characterizing prototypical musical instrument timbres with timbre trait profiles

This paper offers a series of characterizations of prototypical musical timbres, called Timbre Trait Profiles, for 34 musical instruments common in Western orchestras and wind ensembles. These profiles represent the results of a study in which 243 musician participants imagined the sounds of various instruments and used the 20-dimensional model of musical instrument timbre qualia proposed by Reymore and Huron (2020) to rate their auditory image of each instrument. The rating means are visualized through radar plots, which provide timbral-linguistic thumbprints, and are summarized through snapshot profiles, which catalog the six highest- and three lowest-rated descriptors. The Euclidean distances among instruments offer a quantitative operationalization of semantic distances; these distances are illustrated through hierarchical clustering and multidimensional scaling. Exploratory Factor Analysis is used to analyze the latent structure of the rating data. Finally, results are used to assess Reymore and Huron’s 20-dimensional timbre qualia model, suggesting that the model is highly reliable. It is anticipated that the Timbre Trait Profiles can be applied in future perceptual/cognitive research on timbre and orchestration, in music theoretical analysis for both close readings and corpus studies, and in orchestration pedagogy.

Sparse pursuit and dictionary learning for blind source separation in polyphonic music recordings

We propose an algorithm for the blind separation of single-channel audio signals. It is based on a parametric model that describes the spectral properties of the sounds of musical instruments independently of pitch. We develop a novel sparse pursuit algorithm that can match the discrete frequency spectra from the recorded signal with the continuous spectra delivered by the model. We first use this algorithm to convert an STFT spectrogram from the recording into a novel form of log-frequency spectrogram whose resolution exceeds that of the mel spectrogram. We then make use of the pitch-invariant properties of that representation in order to identify the sounds of the instruments via the same sparse pursuit method. As the model parameters which characterize the musical instruments are not known beforehand, we train a dictionary that contains them, using a modified version of Adam. Applying the algorithm on various audio samples, we find that it is capable of producing high-quality separation results when the model assumptions are satisfied and the instruments are clearly distinguishable, but combinations of instruments with similar spectral characteristics pose a conceptual difficulty. While a key feature of the model is that it explicitly models inharmonicity, its presence can also still impede performance of the sparse pursuit algorithm. In general, due to its pitch-invariance, our method is especially suitable for dealing with spectra from acoustic instruments, requiring only a minimal number of hyperparameters to be preset. Additionally, we demonstrate that the dictionary that is constructed for one recording can be applied to a different recording with similar instruments without additional training.