Musical Instrument Sounds Research Articles

The Creative of Isan Music in Soundscape, Battle of Naga

Objectives: This article is part of the research with the objective of creating Isan music in the soundscape, "Battle of Naga." Methods: The researcher used a creative process in music creation, consisting of two parts. The first part is song design, where the researcher took five parts from the story of Sinxay on the Battle of Naga to use as the song structure. Isan melody was used to reinterpret and give new meaning. The melodies included Lai Thang San, Lai Thang Yao, Lai Ka Ten Gon, Lai An Nangseu, Phaya poem reading, and Sinxay melody. The second part consisted of finding the soundscape from Isan musical instruments through the process of playing and adjusting the sounds using technology. This involved capturing sounds of Naga, water, wind, atmosphere, and various situational sounds from the story, and projecting the sounds of Isan musical instruments into water, mixing them with actual instrument sounds. Results: The sounds included the sound of Naga, water, wind, atmosphere, the sounds of situations happening in the story, and the sound projection of Isan musical instruments into the water mixed with the actual sounds of the instruments. At the same time, the sound was mixed with the blowing of wind into the microphone to create the sound of the Garuda’s wings. Additionally, the researcher used the microphone to rub against a fishtank to create sound effects for accompanying the music, such as in the battle scene. Conclusion: The creative process used in this research successfully blended traditional Isan melodies with innovative soundscapes, creating a unique auditory experience for the "Battle of Naga." This approach not only preserved the cultural essence of Isan music but also introduced new dimensions through the use of technology, enhancing the storytelling and emotional impact of the music.

Visually Guided Sound Source Separation With Audio-Visual Predictive Coding.

The framework of visually guided sound source separation generally consists of three parts: visual feature extraction, multimodal feature fusion, and sound signal processing. An ongoing trend in this field has been to tailor involved visual feature extractor for informative visual guidance and separately devise module for feature fusion, while utilizing U-Net by default for sound analysis. However, such a divide-and-conquer paradigm is parameter-inefficient and, meanwhile, may obtain suboptimal performance as jointly optimizing and harmonizing various model components is challengeable. By contrast, this article presents a novel approach, dubbed audio-visual predictive coding (AVPC), to tackle this task in a parameter-efficient and more effective manner. The network of AVPC features a simple ResNet-based video analysis network for deriving semantic visual features, and a predictive coding (PC)-based sound separation network that can extract audio features, fuse multimodal information, and predict sound separation masks in the same architecture. By iteratively minimizing the prediction error between features, AVPC integrates audio and visual information recursively, leading to progressively improved performance. In addition, we develop a valid self-supervised learning strategy for AVPC via copredicting two audio-visual representations of the same sound source. Extensive evaluations demonstrate that AVPC outperforms several baselines in separating musical instrument sounds, while reducing the model size significantly. Code is available at: https://github.com/zjsong/Audio-Visual-Predictive-Coding.

Exploring the relation between fundamental frequency and spectral envelope in the perception of musical instrument sounds

Pitch and timbre perception centrally depend on fundamental frequency (F0) and spectral envelopes (SE), respectively. However, timbre perception as a function of F0 is less explored in the literature. An analysis-synthesis approach was used to compare sounds with congruent and incongruent F0 and SE parameters via sound pleasantness, brightness, and plausibility ratings. Analyzing 1900 musical instruments and voice sounds, a two-dimensional synthesis space was derived. The space was sampled in its inner and outer regions, reflecting high and low similarity to existing acoustical instrument sounds. Sounds with congruent and incongruent F0-SE relations were synthesized, defining congruency as the co-occurrence of F0 and SE in the analyzed set of sounds. In Experiment 1, inner regions of the space were rated as more pleasant compared to outer regions. Further, sounds with incongruent F0-SE relation were rated as less pleasant only for F0 discrepancies above two octaves. Finally, the first dimension of the space strongly reflected differences in brightness when F0 was controlled for. In Experiment 2, a spectrally more fine-grained resynthesis approach revealed effects of congruency on pleasantness and plausibility for smaller F0 separations. These results provide evidence for a co-dependence of F0 and SE properties in the perception of harmonic sounds.

The preservation of Isan folk music with digital sound technology

The preservation of Isan Folk Music is of the utmost significance for preserving the rich cultural heritage of the region and nurturing artistic innovation. Digital sound technology has emerged as a potent instrument for this endeavor, allowing for the construction of an exhaustive and diverse digital sound library. In this study, traditional Isan Folk Music instruments were meticulously recorded using the cutting-edge Logic Pro X software, capturing their nuances in volume and dynamics. The authenticity of each instrument's timbre and unique characteristics were preserved through the use of sound adjustment and editing techniques. The appropriate organization and note nomenclature within the Kontakt software contributed to the library's well-structured and user-friendly design. The process of importing the sounds of musical instruments into Kontakt was seamless, involving selection, customization, and modification, resulting in a vast repository of sounds that are available for exploration. A user-friendly interface was thoughtfully designed to enhance the user's experience exploring the captivating sounds of traditional Isan Folk Music instruments. This digital sound library stands as a testament to the preservation and celebration of Isan Folk Music, providing artists and enthusiasts alike with a resourceful outlet for artistic expression and promoting global appreciation for this distinctive musical tradition, demonstrating the important role of music in cultural preservation and human evolution.

Reassessing the anomalous low frequencies in the Japanese koto

A previous study reported on a finite element model of an indigenous Japanese 13-stringed zither (the koto) based on a CT scan. Frequencies above 100 Hz were relatively easy to account for, while those below 100 Hz were more problematic. Low frequencies have also been reported for the piano, another zither of similar length. The model has now been updated in a range of critical areas, including damping. The updates have resulted in a more precise resolution of the model. It allows us to reassess the anomalous low frequencies to determine if they have other rational explanations. This work continues to expand knowledge of the koto’s complex resonances and has broader applications for understanding the characteristic sounds of other wooden indigenous musical instruments.

Piano etudes by Lo Maisho: trends in the development of the genre

In the modern professional piano art of China, the genre of piano etude is an important part of musical culture and occupies a special place in the history of Chinese music. The first samples of etudes for piano by Chinese composers date back to the 20s of the last century and were created as pieces for the development of the pianist's technical skills. During the development of the genre, the artistic value of the sketches has increased: from a play of a technical nature, the sketches have become independent, dramaturgically constructed compositions. The composers rely on the intonation features of national music, use the melodies of traditional Chinese songs, and resort to modern writing techniques. However, innovation is usually limited to this. The young Chinese composer Luo Maisho created a series of etudes in which, on the one hand, he inherits traditional methods of composition, on the other, he violates the established foundations. For the first time, the metrorhythm region becomes a donor for structural transformations. The author of the article introduces the work of the young composer, offers an excursion into the history of the development of the piano etude genre in China, provides an analysis of the metrorhythmic organization of Lo Maisho's etudes. Thus, the purpose of the study is to analyze new trends in the development of the piano etude genre in China, the subject is the metrorhythmic organization of the musical fabric. In piano studies, Lo Maisho reaches a new level in the progression of the genre. He identifies new trends in Chinese piano music, introducing elements of folk songs, imitation of the sounds of traditional musical instruments, reflecting the imagery and various cultural characteristics of the peoples of various regions of China, introduces new techniques of metrorhythmic organization. The composer does not limit himself to the use of pentatonics, but resolutely strives for innovations in the genre of etude. It is noteworthy that the author departs from the established tradition, which is based on the use of Western European techniques combined with the intonation culture of Chinese melodies, he offers new forms of compositional solutions.

The Effects of Convolution Reverberation on the Emotional Characteristics of Musical Instrument Sounds

Abstract Previous work has shown that the emotional characteristics of musical instrument sounds are significantly changed with parametric reverberation. But do the parametric reverberation results also apply to real concert hall reverberation? This article considers the effects of reverberation time on the emotional characteristics of instrument sounds with convolution reverberation. We compared eight musical instruments and ten emotional characteristics over five hall impulse responses ranging from the one-second Royal National Theatre to the five-second King's College Chapel. The results showed that convolution reverberation had more pronounced effects on emotional characteristics compared with parametric reverberation. This makes sense, since convolution reverberation is often regarded as warmer, more natural, and smoother than parametric reverberation. The latter is often regarded as blander by comparison. Halls with shorter reverberation times emphasized the emotional characteristics Angry and Comic, whereas medium reverberation times emphasized the characteristics Happy, Heroic, and Shy, and longer reverberation times emphasized the characteristics Calm, Mysterious, Romantic, Sad, and Scary. Although the results were more pronounced for convolution reverberation compared with parametric reverberation, there was also a striking agreement in their results, and the correlation coefficient between them was 0.654 over all emotional characteristics. This strong correlation indicates that reverberation time has a remarkably consistent effect on the emotional characteristics regardless of whether using convolution or parametric reverberation—a reflection of their deep underlying functional similarities despite their fundamentally different implementations.

Timbral cues for learning to generalize musical instrument identity across pitch register.

Timbre provides an important cue to identify musical instruments. Many timbral attributes covary with other parameters like pitch. This study explores listeners' ability to construct categories of instrumental sound sources from sounds that vary in pitch. Nonmusicians identified 11 instruments from the woodwind, brass, percussion, and plucked and bowed string families. In experiment 1, they were trained to identify instruments playing a pitch of C4, and in experiments 2 and 3, they were trained with a five-tone sequence (F#3-F#4), exposing them to the way timbre varies with pitch. Participants were required to reach a threshold of 75% correct identification in training. In the testing phase, successful listeners heard single tones (experiments 1 and 2) or three-tone sequences from (A3-D#4) (experiment 3) across each instrument's full pitch range to test their ability to generalize identification from the learned sound(s). Identification generalization over pitch varies a great deal across instruments. No significant differences were found between single-pitch and multi-pitch training or testing conditions. Identification rates can be predicted moderately well by spectrograms or modulation spectra. These results suggest that listeners use the most relevant acoustical invariance to identify musical instrument sounds, also using previous experience with the tested instruments.

Эмоциональная оценка звуков окружающей среды студентами творческих профессий

Background. The present-day mass media and cinema specialists do not pay much attention to the environmental sounds for the effect of creation and putting the viewers and radio listeners into the appropriate emotional atmosphere during the film dubbing and cultural events scoring. Each acoustic event has a specific range and type of emotional impact on people, i.e. each separate sound has a different sort of influence on human beings and brings out specific emotions. Therefore, we come up with an idea that each acoustic event can be classified from the point of view of its emotional impact on human beings by the type and the kind of sentiment. Objective. The aim is to study and analyze the features of the affective evaluation of the environmental sounds by the students of the creative professions. Sample. The pilot study involved 69 students of the Russian Institute of Theatre Arts — GITIS (acting faculty, ballet master faculty and musical theater faculty), of which 26 (38%) were boys and 43 (62%) were girls. 710 verbal units have been encrypted. Methods. The study participants have recorded the environmental sounds that have brought out different kinds of basic emotions in themselves such as guilt, anger, interest, disgust, sadness, scorn, joy, fear, shame, surprise. The audio-record also included the participant’s comments concerning the date and time, sound source and the reasons for the emotions’ uprise. Later on the received data were transformed into text and processed with the procedures of inductive and frequency analysis. Results. The results of the frequency analysis revealed the groups of the environmental sounds bringing out the basic emotions from the point of view of the students of the creative professions. Guilt was brought out by the sounds expressing the complaints against you from other people; anger and scorn were evoked by the sounds made by the person who intentionally breaks the rules of social behavior; interest and sadness were aroused with the sounds of musical instruments, music and speech; disgust was brought out by the physiological sounds of other people, and shame was evoked by the sounds of one’s own physiological sounds; fear arose with sudden shrilling sounds; surprise was brought out by the sounds giving us a notice about some new piece of information. Conclusion. The results of the frequency analysis confirm that the environmental sounds can help in modeling different kinds of emotional impact.

Cultural relationship between rural soundscape and space in Hmong villages in Guizhou

The Hmong is one of the oldest ethnic groups in south-western China, and sound plays an important role in their culture. According to the general classification of landscape science, the rural soundscape of Hmong villages is divided into three types: a point soundscape dominated by a single sound source, a line soundscape with a typical Hmong rural village alley space as the sound field interface, and an area soundscape with the Bronze-drum Square, the only open public space of Hmong nationality, as the field. Combined with a sound collection and field test during field investigation, we obtained the characteristics of various rural soundscapes and evaluated the acoustic environment. The results showed that the “point” soundscape of medium- and low-frequency powder shotgun and wooden drum sounds in the unclosed Hmong villages exceed the transmission distance and sound durability. The “line” soundscape utilizes the turning point and envelope of different spaces to increase the acoustic reflection surface and create reverberation, increasing the propagation time and distance of the sound, and emphasizing the directivity of the sound through a long and narrow space. The “surface” soundscape includes the richest medium- and high-frequency-based human voice and music sounds and medium- and low-frequency musical instrument sounds, among others. The values of T30, EDT, and C80 in the sound field, all changed significantly because of the lack of acoustic reflection from the ceiling, through the top opening. Finally, the study reveals the cultural association between rural soundscape and space, which is reflected in the Hmong's “defensive” function and “group” culture. Thus, the Bronze-drum Square sound field does not meet the requirements of the current specification of cultural interpretation, and it is in the participatory Hmong performance mode, centered on drum club group structure.

Musical audio samples generated from joint text embeddings

The field of machine learning has benefited from the appearance of diffusion-based generative models for images and audio. While text-to-image models have become increasingly prevalent, text-to-audio generative models are currently an active area of research. We present work on generating short samples of musical instrument sounds generated by a model which was conditioned on text descriptions and the file structure labels of large sample libraries. Preliminary findings indicate that generation of wide-spectrum sounds such as percussion are not difficult, while the generation of harmonic musical sounds presents challenges for audio diffusion models.

Use of Speaker Recognition Approaches for Learning and Evaluating Embedding Representations of Musical Instrument Sounds

Constructing an embedding space for musical instrument sounds that can meaningfully represent new and unseen instruments is important for downstream music generation tasks such as multi-instrument synthesis and timbre transfer. The framework of Automatic Speaker Verification (ASV) provides us with architectures and evaluation methodologies for verifying the identities of unseen speakers, and these can be repurposed for the task of learning and evaluating a musical instrument sound embedding space that can support unseen instruments. Borrowing from state-of-the-art ASV techniques, we construct a musical instrument recognition model that uses a SincNet front-end, a ResNet architecture, and an angular softmax objective function. Experiments on the NSynth and RWC datasets show our model’s effectiveness in terms of equal error rate (EER) for unseen instruments, and ablation studies show the importance of data augmentation and the angular softmax objective. Experiments also show the benefit of using a CQT-based filterbank for initializing SincNet over a Mel filterbank initialization. Further complementary analysis of the learned embedding space is conducted with t-SNE visualizations and probing classification tasks, which show that including instrument family labels as a multi-task learning target can help to regularize the embedding space and incorporate useful structure, and that meaningful information such as playing style, which was not included during training, is contained in the embeddings of unseen instruments.

Blind Source Separation in Polyphonic Music Recordings Using Deep Neural Networks Trained via Policy Gradients

We propose a method for the blind separation of sounds of musical instruments in audio signals. We describe the individual tones via a parametric model, training a dictionary to capture the relative amplitudes of the harmonics. The model parameters are predicted via a U-Net, which is a type of deep neural network. The network is trained without ground truth information, based on the difference between the model prediction and the individual time frames of the short-time Fourier transform. Since some of the model parameters do not yield a useful backpropagation gradient, we model them stochastically and employ the policy gradient instead. To provide phase information and account for inaccuracies in the dictionary-based representation, we also let the network output a direct prediction, which we then use to resynthesize the audio signals for the individual instruments. Due to the flexibility of the neural network, inharmonicity can be incorporated seamlessly and no preprocessing of the input spectra is required. Our algorithm yields high-quality separation results with particularly low interference on a variety of different audio samples, both acoustic and synthetic, provided that the sample contains enough data for the training and that the spectral characteristics of the musical instruments are sufficiently stable to be approximated by the dictionary.

Making media for learning musical instruments using the Scratch application

The challenge for music educators in distance learning is the inability to practice teaching musical instruments directly to students. This article aims to bridge music educators so that they can still teach musical instruments to students and the students can continue to learn music even though they are learning from home. The solution offered is learning music using the Scratch application on a laptop or desktop computer as the controller. There are five steps to make music learning media using the Scratch application: (1) downloading and installing the Scratch Desktop application, (2) making designs or icons for each music scale, (3) recording or inputting sound musical instruments, (4) setting the buttons for each musical scale, and (5) playing the musical compositions. The Scratch application was effective in making learning media for learning musical instruments, learning to recognize the sounds of musical instruments, and learning to memorize song melodies. In conclusion, the Scratch application can be a solution for creating computer-based learning media for musical instruments.

Spectral envelope position and shape in sustained musical instrument sounds.

It has been argued that the relative position of spectral envelopes along the frequency axis serves as a cue for musical instrument size (e.g., violin vs viola) and that the shape of the spectral envelope encodes family identity (violin vs flute). It is further known that fundamental frequency (F0), F0-register for specific instruments, and dynamic level strongly affect spectral properties of acoustical instrument sounds. However, the associations between these factors have not been rigorously quantified for a representative set of musical instruments. Here, we analyzed 5640 sounds from 50 sustained orchestral instruments sampled across their entire range of F0s at three dynamic levels. Regression of spectral centroid (SC) values that index envelope position indicated that smaller instruments possessed higher SC values for a majority of instrument classes (families), but SC also correlated with F0 and was strongly and consistently affected by the dynamic level. Instrument classification using relatively low-dimensional cepstral audio descriptors allowed for discrimination between instrument classes with accuracies beyond 80%. Envelope shape became much less indicative of instrument class whenever the classification problem involved generalization to different dynamic levels or F0-registers. These analyses confirm that spectral envelopes encode information about instrument size and family identity and highlight their dependence on F0(-register) and dynamic level.

Perceiving Sound Objects in the Musique Concrète.

In the late 1940s and early 1950s, there emerged a radically new kind of music based on recorded environmental sounds instead of sounds of traditional Western musical instruments. Centered in Paris around the composer, music theorist, engineer, and writer Pierre Schaeffer, this became known as musique concrète because of its use of concrete recorded sound fragments, manifesting a departure from the abstract concepts and representations of Western music notation. Furthermore, the term sound object was used to denote our perceptual images of such fragments. Sound objects and their features became the focus of an extensive research effort on the perception and cognition of music in general, remarkably anticipating topics of more recent music psychology research. This sound object theory makes extensive use of metaphors, often related to motion shapes, something that can provide holistic representations of perceptually salient, but temporally distributed, features in different kinds of music.

Analyzing spectral envelope position and shape in a large set of sustained musical instrument sounds

It has been argued that the relative position of spectral envelopes along the frequency axis serves as a cue for musical instrument size (e.g., violin versus viola) and that the shape of the spectral envelope encodes family identity (violin versus flute). However, the extent to which spectral envelope position and shape are affected by fundamental frequency (F0), F0- register within instruments, and dynamic level has never been rigorously quantified across a representative set of musical instrument sounds. Here, we analyzed sounds from 50 sustained orchestral instruments sampled across their entire range of F0s at three dynamic levels. Linear regression of spectral centroid (SC) values that index envelope position indicated that smaller instruments possessed higher SC values in most instrument classes, but SC also correlated with F0 and was strongly and consistently affected by the dynamic level. Instrument classification using relatively low-dimensional cepstral audio descriptors allowed for discrimination between instrument classes with accuracies beyond 80%. Envelope shape became much less indicative of instrument class whenever the classification problem involved generalization across dynamic levels or F0-registers. These analyses confirm that the spectral envelope encodes information about instrument size and family identity and highlight its strong dependence on F0 (-register) and dynamic level.

Sparse pursuit and dictionary learning for blind source separation in polyphonic music recordings

We propose an algorithm for the blind separation of single-channel audio signals. It is based on a parametric model that describes the spectral properties of the sounds of musical instruments independently of pitch. We develop a novel sparse pursuit algorithm that can match the discrete frequency spectra from the recorded signal with the continuous spectra delivered by the model. We first use this algorithm to convert an STFT spectrogram from the recording into a novel form of log-frequency spectrogram whose resolution exceeds that of the mel spectrogram. We then make use of the pitch-invariant properties of that representation in order to identify the sounds of the instruments via the same sparse pursuit method. As the model parameters which characterize the musical instruments are not known beforehand, we train a dictionary that contains them, using a modified version of Adam. Applying the algorithm on various audio samples, we find that it is capable of producing high-quality separation results when the model assumptions are satisfied and the instruments are clearly distinguishable, but combinations of instruments with similar spectral characteristics pose a conceptual difficulty. While a key feature of the model is that it explicitly models inharmonicity, its presence can also still impede performance of the sparse pursuit algorithm. In general, due to its pitch-invariance, our method is especially suitable for dealing with spectra from acoustic instruments, requiring only a minimal number of hyperparameters to be preset. Additionally, we demonstrate that the dictionary that is constructed for one recording can be applied to a different recording with similar instruments without additional training.

Guided Generative Adversarial Neural Network for Representation Learning and Audio Generation Using Fewer Labelled Audio Data

The Generation power of Generative Adversarial Neural Networks (GANs) has shown great promise to learn representations from unlabelled data while guided by a small amount of labelled data. We aim to utilise the generation power of GANs to learn Audio Representations. Most existing studies are, however, focused on images. Some studies use GANs for speech generation, but they are conditioned on text or acoustic features, limiting their use for other audio, such as instruments, and even for speech where transcripts are limited. This paper proposes a novel GAN-based model that we named Guided Generative Adversarial Neural Network (GGAN), which can learn powerful representations and generate good-quality samples using a small amount of labelled data as guidance. Experimental results based on a speech [Speech Command Dataset (S09)] and a non-speech [Musical Instrument Sound dataset (Nsyth)] dataset demonstrate that using only 5% of labelled data as guidance, GGAN learns significantly better representations than the state-of-the-art models.

Computational phonogram archiving

A general approach to analyzing audio files makes it possible to re-create the sound of ancient instruments, identify cross-cultural musical properties, and more.