Sound Synthesis Method Research Articles

Synthesis of natural sound based on Nyquist

In recent years, sound synthesis has become an important part of daily life with the rapid development of the technology. There are many methods of natural sound synthesis, but each of them has its disadvantages. This paper explored a novel method of natural sound synthesis based on Nyquist. This method has significant advantages in high speed, convenience, flexibility, randomness, and small file size. The research investigates the effectiveness of this method by synthesizing four distinct natural sounds - rain, wind, thunder, and a combination of all three - using unit generators in Nyquist. To further evaluate the result, Mel spectrograms and a questionnaire are applied. The evaluation shows that the synthesized sounds meet expectations and are relatively real and natural. This method can be combined with other technology, such as the large language model, to provide better accessibility. This method can generate natural sounds for many tasks, especially those requiring high speed.

Estimation of acoustical materials sound absorption coefficient under oblique incidence plane wave and diffuse field via Allard model inversion

A technique for estimating the sound absorption of materials under oblique incidence plane wave and diffuse field excitations is proposed. It requires a mobile loudspeaker and a pair of fixed microphones above a layer of absorbing material. Starting from sound pressure measurements made above the material surface for multiple source positions and by inverting Allard’s propagation model, the proposed method allows for the identification of two effective (or equivalent) parameters, namely, the complex effective density and the complex wave number. Obtaining these parameters makes it possible to estimate the sound absorption coefficient for a plane wave with any angle of incidence or under a diffuse acoustic field by summation over the angles. The results are compared to theoretical values calculated using the Johnson-Champoux-Allard model and to reference measurements obtained using an impedance tube, a small cabin, a large reverberant room, and a sound field synthesis method. One of the limitations of this method lies in the assumptions associated with the Allard model to describe the sound field above the material (assumed to be isotropic, homogeneous, of constant thickness, and to behave like an equivalent fluid). The main advantages are (1) that the sound absorption coefficient can be estimated under oblique incidence plane wave and diffuse acoustic field with values that are always in a physical range (between 0 and 1), and (2) that samples of the order of the square meter without specific preparation can be tested which reduces the constraints associated with impedance tube or reverberant room measurements.

Electronic wind instruments for mobility-restricted musicians

One of the earliest applications of analog music synthesizers was the simulation of orchestral instruments. Electronic wind instrument sounds were played through piano-like keyboards or interfaces with breath controllers that mimicked physical wind instruments. Synthesizers are also often the best alternative for artists with disabilities to play a musical instrument. This talk focuses on methods and experiences the authors were involved with to design electronic alternatives to wind instruments for people who lost control of their arms. The Jamboxx is a USB controller that was specifically designed as a head-only interface using a breath controller to simulate airflow but also operates as a suck/puff device to activate menus and buttons, a slider to manipulate pitch, and a tilt-sensor to control expression parameters. The Jamboxx also addressed musicians seeking a harmonica-type interface to augment their guitar performance. The AUMI instrument is a touchless device based on camera tracking. This talk will address different sound synthesis methods to achieve realistic wind instruments, the ability to integrate these instruments with other off-the-shelf products (e.g., VST plugins), and special requirements for artists with special needs to play these instruments without too much fatigue. [Work supported by the Craig H. Neilsen Foundation.]

Anti-cancer effects of green synthesized gold nanoparticles using leaf extract of Annona muricata. L against squamous cell carcinoma cell line 15 through apoptotic pathway

Background: Oral cancer remains one of the most dreadful diseases in developing nations. Currently, there has been a rise in the prevalence of tongue squamous cell carcinoma (SCC), with a poor prognosis. The use of standard treatment approaches against oral cancer patients brings about several side effects. In recent years, nanomedicine has provided a versatile platform for developing new targeted therapeutic modalities. However, safety remains a concern in the synthesis of nanoparticles (NPs). Therefore, the present study aims to synthesize safer phytoconstituent-mediated gold NPs (AuNPs) utilizing leaf extracts of Annona muricata, where the biochemical components of the plant leaf act as the reducing and capping agents in the synthesis of NPs, and to evaluate its anti-cancer activity against SCC. Materials and Methods: In this in vitro experimental study, AuNPs were synthesized through an effective, simple, and ecologically sound green synthesis method. After characterization of these synthesized AuNPs, in vitro assays such as 3-(4, 5-dimethylthiazole2-yl)-2, 5-biphenyl tetrazolium bromide, wound healing, and clonogenic assays were carried out to investigate the anti-cancer potential of green synthesized AuNPs in the human tongue SCC cell line (SCC-15), and the possible mechanism of action was evaluated through gene and protein expression analysis of Bax, Bcl-2, and p53 genes. The results were expressed as mean ± standard deviation using Statistical Package for Social Sciences (SPSS) 20.0 software and Student's t-test was performed for experimental data. P ≤0.05 were considered statistically significant. Results: The in vitro assays demonstrated that the synthesized AuNPs are exhibiting anti-cancer activity by apoptosis of SCC-15 cells in a dose-dependent manner. Further, it also revealed a highly significant decrease in anti-apoptotic Bcl-2 gene expression, whereas pro-apoptotic genes p53 and Bax revealed a highly significant increase, which is statistically significant compared to the control (P < 0.05). Conclusion: Our findings demonstrated that the AuNPs synthesized from A. muricata leaf extract could act as a novel anticancer agent, particularly against SCC, after further scrutiny.

An Overview of Visual Sound Synthesis Generation Tasks Based on Deep Learning Networks

Visual sound synthesis (which refers to the process of recreating, as realistically as possible, the sound produced by the movements and actions of objects within a video, given specific conditions such as video content and accompanying text) is an important part of the composition of high-quality films at present. Most traditional methods of sound synthesis are based on the artificial creation of simulated props for sound effects synthesis, which is achieved by using various existing props and constructed scenes. However, traditional methods cannot meet specific conditions for sound effect synthesis and require large amounts of participant, material resources and time. It can take nearly ten hours to simulate realistic sound effects in a minute-long video. In this paper, we systematically summarize and consolidate current advances in deep learning in the field of visual sound synthesis, based on existing related papers. We focus on the exploration and development history of deep learning models for the task of visual sound synthesis, and classify detailed research methods and related dataset information based on their development characteristics. By analyzing the technical differences among various model approaches, we can summarize potential research directions in the field, thereby further promoting the rapid development and practical implementation of deep learning models in the video domain.

RESEARCH OF THE PROCESS OF VISUAL ART TRANSMISSION IN MUSIC AND THE CREATION OF COLLECTIONS FOR PEOPLE WITH VISUAL IMPAIRMENTS

This article explores the creation of music through the automated generation of sounds from images. The developed automatic image sound generation method is based on the joint use of neural networks and light-music theory. Translating visual art into music using machine learning models can be used to make extensive museum collections accessible to the visually impaired by translating artworks from an inaccessible sensory modality (sight) to an accessible one (hearing). Studies of other audio-visual models have shown that previous research has focused on improving model performance with multimodal information, as well as improving the accessibility of visual information through audio presentation, so the work process consists of two parts. The result of the work of the first part of the algorithm for determining the tonality of a piece is a graphic annotation of the transformation of the graphic image into a musical series using all colour characteristics, which is transmitted to the input of the neural network. While researching sound synthesis methods, we considered and analysed the most popular ones: additive synthesis, FM synthesis, phase modulation, sampling, table-wave synthesis, linear-arithmetic synthesis, subtractive synthesis, and vector synthesis. Sampling was chosen to implement the system. This method gives the most realistic sound of instruments, which is an important characteristic. The second task of generating music from an image is performed by a recurrent neural network with a two-layer batch LSTM network with 512 hidden units in each LSTM cell, which assembles spectrograms from the input line of the image and converts it into an audio clip. Twenty-nine compositions of modern music were used to train the network. To test the network, we compiled a set of ten test images of different types (abstract images, landscapes, cities, and people) on which the original musical compositions were obtained and stored. In conclusion, it should be noted that the composition generated from abstract images is more pleasant to the ear than the generation from landscapes. In general, the overall impression of the generated compositions is positive. Keywords: recurrent neural network, light music theory, spectrogram, generation of compositions.

Modeling and reproduction of sound field by moving complex sound source

Previously, Wave Field Synthesis (WFS), Near-field Compensated Higher-Order Ambisonics (NFC-HOA), Boundary Surface Control (BoSC), and the Spectral Division Method (SDM) have been proposed as sound field synthesis methods. These methods are based on matching the acoustic physical quantities at the boundary of the reproduced sound field with those of the desired sound field. Sound field synthesis techniques can be classified into data-based and model-based methods. The former aim at reproducing sound fields captured by microphone arrays, while the latter aim at synthesizing sound fields based on mathematical models. Most existing sound field reproduction methods only consider simple sound source models, such as static point sources, instead of sound sources with complex radiation properties. To generate a more realistic sound image, this study investigated the formulation of a sound field generated by a sound source with complex radiation properties moving along arbitrary trajectories in a spherical harmonic domain, and reproduced the sound field using loudspeaker arrays.

Sound field synthesis for psychoacoustic research: In situ evaluation of auralized sound pressure levela).

The use of virtual acoustic environments has become a key element in psychoacoustic and audiologic research, as loudspeaker-based reproduction offers many advantages over headphones. However, sound field synthesis methods have mostly been evaluated numerically or perceptually in the center, yielding little insight into the achievable accuracy of the reproduced sound field over a wider reproduction area with loudspeakers in a physical, laboratory-standard reproduction setup. Deviations from the ideal free-field and point-source concepts, such as non-ideal frequency response, non-omnidirectional directivity, acoustic reflections, and diffraction on the necessary hardware, impact the generated sound field. We evaluate reproduction accuracy in a 61-loudspeaker setup, the Simulated Open Field Environment, installed in an anechoic chamber. A first measurement following the ISO 8253-2:2009 standard for free-field audiology shows that the required accuracy is reached with critical-band-wide noise. A second measurement characterizes the sound pressure reproduced with the higher-order Ambisonics basic decoder, with and without max rE weighting, vector base amplitude panning, and nearest loudspeaker mapping on a 187 cm × 187 cm reproduction area. We show that the sweet-spot size observed in measured sound fields follows the rule kr≤N/2 rather than kr≤N but is still large enough to avoid compromising psychoacoustic experiments.

Agent-Based Music Live Coding: Sonic adventures in 2D

This article describes agent-based music live coding, an approach for music performance and composition based on programming a set of agents in a 2D plane. This style of programming draws from the tradition of agent-based models and facilitates interactive algorithmic control of data-driven sound synthesis methods such as wave terrain synthesis or corpus-based concatenative synthesis. The main elements are a ‘terrain’, which may be used to access different types of data, a set of agents and their trajectories, and a set of synthesis functions associated to agents. An implementation using the SuperCollider language is demonstrated.

Audio Augmented Reality in Smart Learning Environments

Image processing-based augmented reality (AR) is widely used in several fields. However, audio content is also crucial in certain cases, for example, focusing on appreciating artwork in a museum, rather than a virtually synthesized image. In this study, an attempt was made to provide ubiquitous learning (u-learning) services by using audio AR for smart cities where people do not need smartphones, tablets, or wearable devices for learning. The concept of “unmanned stores” was introduced in the proposed learning environments. A 360° camera was mounted on the ceiling of an exhibition room. An algorithm was used to detect moving objects and locate a visitor, from whose position an information-processing platform searched the showpiece nearest to the visitor and selected the corresponding audio guide. For multiple visitors, infrared (IR) sensors were used to count the visitors inside the room. A loudspeaker array was built to emit directional sounds for eliminating interference between multiple sound sources. Along with the proposed algorithm for increasing the accuracy while sensing multiple movements, the conventional sound field synthesis method was modified to improve the listening experience and learning quality. Image-based sensor technology, IR detection, and an audio signal-processing platform were integrated into a smart environment in which people can conduct u-learning without personal devices. The method can provide immersive audio tours with virtual narrators and peer-helpers in schools, museums, and art galleries. Qualitative and quantitative analyses indicated that effective hands-free and eyes-free learning was achieved.

성덕대왕신종(聖德大王神鍾)의 음향학적 에뮬레이션 연구

The purpose of this study is to analyze the possibility of using the “Sacred Bell of Great King Seongdeok” which is designated as National Treasure No. 29, as a musical resource through acoustic analysis and sound synthesis method. Based on the acoustically analyzed data of the Bell, it was tried to synthesized with frequency and fluctuation of partial sound, the beat cycle and envelop application by the additive synthesis among the method of sound synthesis The spatial sounds were modelled by an artificial neural network to estimate the exact reverberation of the source. The final result was produced with Native Instruments Reactor6, which can be easily used not only by sound designers and multimedia artists but also by many musicians, to preserve the history and identity of Sacred Bell of Great King Seongdeok, and to confirm its potential as a medium to be fused to new media.

Research on the trend and innovation of electronic music production

Nowadays, the level of science and technology in our country gradually develops, and the corresponding electronic music production is experiencing rapid progress. At present, people's demand for the quality of electronic music production is higher and higher. It should not only give people a good sense of hearing, but also be able to combine with high and new technology for application. Throughout the evolution of Chinese electronic music production, it is not difficult to find that it has experienced a process of gradual improvement of technology and is developing towards the direction of computerization. This paper will briefly introduce the evolution process of electronic music production and traditional technologies such as wave table synthesis, addition synthesis, subtraction synthesis, sampling synthesis, modulation synthesis and other sound synthesis methods, and briefly describe the application of the new ROLI Seaboard technology in electronic music production.

Measurement of the diffuse field sound absorption using a sound field synthesis method

A method for measuring the diffuse field sound absorption coefficient of a material using sound field synthesis is proposed. A planar loudspeaker array is first used to generate acoustic plane waves with variable incidence angle on the surface of a material under test. Using a two-microphone probe positioned closely to the sample’s surface, the angle-dependent sound absorption coefficients are then estimated. Finally, the diffuse field absorption coefficient is computed following Paris formula. Numerical simulations are used to evaluate the respective effects of the maximum incidence angle value and the number of individual incidence angles that are required for a robust calculation of the diffuse sound field absorption. Measurements are conducted on three different materials and compared with either simulation results obtained using the Johnson-Champoux-Allard theory, or with measurement results obtained using the standard reverberation chamber method. For all considered materials and over a wide frequency range, the proposed method leads to results that are in better agreement with theoretical predictions than those obtained using standardized methods.

Towards Maximizing a Perceptual Sweet Spot for Spatial Sound With Loudspeakers

The <i>sweet spot</i> can be interpreted as the region where acoustic sources create a spatial auditory illusion. We study the problem of maximizing this sweet spot when reproducing a desired sound wave using an array of loudspeakers. To achieve this, we introduce a theoretical framework for spatial sound perception that can be used to define a sweet spot, and we develop a method that aims to generate a sound wave that directly maximizes the sweet spot defined by a model within this framework. Our method aims to incorporate perceptual principles from the onset and it is flexible: it imposes little to no constraints on the regions of interest, the arrangement of loudspeakers or their radiation pattern. However, the perceptual models must satisfy a convexity condition, which is fulfilled by state-of-the-art monaural perceptual models, but not by binaural ones. Proof-of-concept experiments show that our method, when implemented with van de Par&#x0027;s monaural model, outperforms state-of-the-art sound field synthesis methods in terms of their binaural azimuth localization and binaural coloration properties.

SYNTHESIATORS. SOUND SYNTHESIS METHODS

SYNTHESIATORS. SOUND SYNTHESIS METHODS

Application of a Musical Robot for Adjusting Guitar String Re-Excitation Parameters in Sound Synthesis

Sound synthesis methods based on physical modelling of acoustic instruments depend on data that require measurements and recordings. If a musical instrument is operated by a human, a difficulty in filtering out variability is introduced due to a lack of repeatability in excitation parameters, or in varying physical contact between a musician and an instrument, resulting in the damping of vibrating elements. Musical robots can solve this problem. Their repeatability and controllability allows studying even subtle phenomena. This paper presents an application of a robot in studying the re-excitation of a string in an acoustic guitar. The obtained results are used to improve a simple synthesis model of a vibrating string, based on the finite difference method. The improved model reproduced the observed phenomena, such as the alteration of the signal spectrum, damping, and ringing, all of which can be perceived by a human, and add up to the final sound of an instrument. Moreover, as it was demonstrated by using two different string plucking mechanisms, musical robots can be redesigned to study other sound production phenomena and, thus, to further improve the behaviours of and sounds produced by models applied in sound synthesis.

Automatic sound synthesis using the fly algorithm

Our study is demonstrated a new type of evolutionary sound synthesis method. This work based on the fly algorithm, a cooperative co-evolution algorithm; it is derived from the Parisian evolution approach. The algorithm has relatively amended the position of individuals (the Flies) represented by 3-D points. The fly algorithm has successfully investigated in different applications, starting with a real-time stereo vision for robotics. Also, the algorithm shows promising results in tomography to reconstruct 3-D images. The final application of the fly algorithm was generating artistic images, such as digital mosaics. In all these applications, the flies’representation started for simple, 3-D points, to complex one, the structure of 9-elements. Our method follows evolutionary digital art with the fly algorithm in representing the pattern of the flies. They represented in a way of having their structure. This structure includes position, colour, rotation angle, and size. Our algorithm has the benefit of graphics processing units (GPUs) to generate the sound waveform using the modern OpenGL shading language.

Perception of atomic speech

Natural speech signals are continuous and have great redundancy in time, which is learned by human beings and guarantees speech comprehension even in challenging circumstances. This work develops a sound synthesis method to study some time-related characteristics of the human auditory system. The temporal information within the individual frequency band of a speech is down-sampled using Gaussian-shaped pulses and then recombined into a new sound which may have some remaining intelligibility or feasible phonetic features. The “atomic” sound is coined to the sound with an extremely spectral-temporally sparse spectrogram generated using the method. A battery of speech perception tests was administered in normal-hearing listeners. Results show that (1) atomic sounds from clear speech can be understood as speech, although the listeners often reported a feeling of water sound textures; (2) the temporal and spectral resolution could be traded off in the atomic speech comprehension; and (3) only one maximum envelope value preserved among a 32-channel filter-bank at a rate of 400 Hz was surprisingly adequate for speech understanding, which indicates that the brain can organize the only spectral peak within each short duration no longer than 2.5-ms into sentence understanding without either explicit or implicit encoding of the first three formants.

Animating explosion with exploding sound and rigid‐body sound

AbstractVisual simulation of explosion animation can be found many applications in virtual reality. In addition to visual simulation, sound synthesis of explosion scenes is also indispensable. However, little attention has been paid to this research area. To this end, this paper proposes an automatic sound synthesis method for explosion scenes. Explosion animation consists of two major events, namely explosive event and interactive rigid‐body fracture event. Correspondingly, the sound for an explosion scene can be viewed as the combination of two components, that is, the exploding sound and the rigid‐body sound. Among them, the exploding sound often shows similarity in terms of characteristics in frequency domain, while the rigid‐body sound is rich in variations due to the unpredictable motion state and different materials of debris. As a result, for the synthesis of the rigid‐body sound, firstly, we extract the rigid‐body sounds from recording examples by the empirical mode decomposition method. On this basis, a new spectral flux based approach is developed to segment the extracted sounds into a grain dictionary. Then, a cascade sound synthesis method based on impulse response is designed to synchronize the rigid‐body sound for fracture events. For the synthesis of exploding sound, due to its similar characteristics in frequency domain, an example‐based automatic sound synthesis method is adapted in this paper. Finally, we propose a priority‐aware sound blending method to merge these two types of completely different sounds reasonably. Various experimental results demonstrated the effectiveness of our new method.

Sound field synthesis of arbitrary moving sources using spectral division method.

The spectral division method (SDM) is one of several sound field synthesis techniques of spatial sound reproduction. The synthesis accuracy of SDM is usually higher than that of wave field synthesis (WFS). However, existing SDM-based sound field synthesis methods are developed for virtual sources moving uniformly along a straight trajectory. This Letter proposes a 2.5-dimensional SDM driving function for arbitrary moving sources. Numerical simulations have validated the driving function. The results indicate that the synthesis error of the SDM with the proposed driving function is lower than that of WFS, especially when the virtual source is close to the secondary sources.