Sound Effects Research Articles

Hierarchical-Concatenate Fusion TDNN for sound event classification.

Semantic feature combination/parsing issue is one of the key problems in sound event classification for acoustic scene analysis, environmental sound monitoring, and urban soundscape analysis. The input audio signal in the acoustic scene classification is composed of multiple acoustic events, which usually leads to low recognition rate in complex environments. To address this issue, this paper proposes the Hierarchical-Concatenate Fusion(HCF)-TDNN model by adding HCF Module to ECAPA-TDNN model for sound event classification. In the HCF module, firstly, the audio signal is converted into two-dimensional time-frequency features for segmentation. Then, the segmented features are convolved one by one for improving the small receptive field in perceiving details. Finally, after the convolution is completed, the two adjacent parts are combined before proceeding with the next convolution for enlarging the receptive field in capturing large targets. Therefore, the improved model further enhances the scalability by emphasizing channel attention and efficient propagation and aggregation of feature information. The proposed model is trained and validated on the Urbansound8K dataset. The experimental results show that the proposed model can achieve the best classification accuracy of 95.83%, which is an approximate improvement of 5% (relatively) over the ECAPA-TDNN model.

The cybercultural characteristics present in the digital story The Boat (2015)

The digital story The Boat (2015) by Matt Huynh revolves around the Vietnamese teenager Mai who had to emigrate from Vietnam to Australia in a boat. Mai had to endure poor living conditions such as hunger, thirst, and emotional distress caused by the war. In this context, the digital story narrates those hardships and provides an immersive, interactive reading experience by incorporating cybercultural features into the narrative. Despite the prevalence of cyberculture in the narrative, recent studies about The Boat (2015) focus either on movement-related features (Lehman, 2020) or utilize the story as an emotional healing, and pedagogic online tool (Kwak, 2021; Miles & Boltin, 2024). Therefore, our study aimed to identify and analyze the cybercultural characteristics incorporated in The Boat (2015) by proposing an analysis of selected scenes after applying meta-analysis and systematic methods. We have identified three dominant characteristics of The Boat (2015): immersion, interaction, and platform. Those characteristics are often presented in overlapping layers since the cyberculture features such as anchor texts, GIFs, and sound effects require a platform and create a propitious environment for the audience to interact and immerse themselves in Mai’s story.

Aurally enhanced media promotion: Music and sound in Greek television commercials

The sonic dimensions of modern media—including speech, sound effects, and music—play a significant role in communicating signs, meanings, and messages through multimodal channels of audiovisual connotations. Each feature can be regarded as a discrete unit; however, the overall framework is established through an orchestrated “interaction” of these components. Receiving information through televised advertisements is a cognitive process that requires critical analysis. This study examines the functions of music and sound in television advertising, emphasizing the collaboration between visual and audio material. Our objective is to equally investigate optical and musical conventions in selected TV commercials. We will also analyze how meaning emerges through visuals, sounds, and music in correlation to the product, branding, and the purpose of advertisements as short, concise audiovisual narratives. The data for this study were collected from a series of food and drink ads shown on Greek television between 2015 and 2020. The research draws upon contemporary culturally informed, musicological, semiotic, and interpretive methodologies, applying both textual and contextual analyses to examine how these techniques captivate audiences and effectively communicate messages. Understanding music’s psychological and pragmatic effects in advertising is crucial for recognizing its role as a powerful tool that influences consumer behavior and perception.

Exploring sex differences in auditory saliency: the role of acoustic characteristics in bottom-up attention

BackgroundSeveral cognitive functions are related to sex. However, the relationship between auditory attention and sex remains unclear. The present study aimed to explore sex differences in auditory saliency judgments, with a particular focus on bottom-up type auditory attention.MethodsForty-five typical adults (mean age: 21.5 ± 0.64 years) with no known hearing deficits, intelligence abnormalities, or attention deficits were enrolled in this study. They were tasked with annotating attention capturing sounds from five audio clips played in a soundproof room. Each stimulus contained ten salient sounds randomly placed within a 1-min natural soundscape. We conducted a generalized linear mixed model (GLMM) analysis using the number of responses to salient sounds as the dependent variable, sex as the between-subjects factor, duration, maximum loudness, and maximum spectrum of each sound as the within-subjects factor, and each sound event and participant as the variable effect.ResultsNo significant differences were found between male and female groups in age, hearing threshold, intellectual function, and attention function (all p > 0.05). Analysis confirmed 77 distinct sound events, with individual response rates of 4.0–100%. In a GLMM analysis, the main effect of sex was not statistically significant (p = 0.458). Duration and spectrum had a significant effect on response rate (p = 0.006 and p < 0.001). The effect of loudness was not statistically significant (p = 0.13).ConclusionsThe results suggest that male and female listeners do not differ significantly in their auditory saliency judgments based on the acoustic characteristics studied. This finding challenges the notion of inherent sex differences in bottom-up auditory attention and highlights the need for further research to explore other potential factors or conditions under which such differences might emerge.

Equipment Sounds’ Event Localization and Detection Using Synthetic Multi-Channel Audio Signal to Support Collision Hazard Prevention

Construction workplaces often face unforeseen collision hazards due to a decline in auditory situational awareness among on-foot workers, leading to severe injuries and fatalities. Previous studies that used auditory signals to prevent collision hazards focused on employing a classical beamforming approach to determine equipment sounds’ Direction of Arrival (DOA). No existing frameworks implement a neural network-based approach for both equipment sound classification and localization. This paper presents an innovative framework for sound classification and localization using multichannel sound datasets artificially synthesized in a virtual three-dimensional space. The simulation synthesized 10,000 multi-channel datasets using just fourteen single sound source audiotapes. This training includes a two-staged convolutional recurrent neural network (CRNN), where the first stage learns multi-label sound event classes followed by the second stage to estimate their DOA. The proposed framework achieves a low average DOA error of 30 degrees and a high F-score of 0.98, demonstrating accurate localization and classification of equipment near workers’ positions on the site.

The Effect of Competition Sounds on Balance Performance in Football Players

Abstract Objective This study aimed to investigate the impact of competition environment sounds on the static and dynamic balance of football players. Materials and Methods Thirty-two football players with a mean age of 21.12±2.99 years and a body mass index of 21.98±2.58 kg/m² were included in this study. The initial assessment condition (whether in a quiet environment or with competition sounds) was determined by simple randomization. Forty-eight hours later, at the same time of day, the tests were repeated with the environmental conditions reversed. Balance performance was assessed using the Tecnobody device for both static and dynamic balance. Results In the static balance test, it was found that the surface area covered significantly decreased in the environment where competition sounds were played (p=0.014), while there was no significant change in overall sway (p=0.313), anteroposterior sway (p=0.852), or mediolateral sway (p=0.104). In the dynamic balance test, no significant change was observed in anteroposterior sway (p=0.411), but mediolateral sway (p=0.025) and overall sway (p=0.042) significantly decreased. Additionally, there was a significant improvement in the athletes’ score for reaching the target (p=0.003). The perceived difficulty levels during the tests were found to be significantly lower in both the static (p=0.048) and dynamic (p=0.001) balance tests in the environment where competition sounds were played. Conclusion In conclusion, competition environment sounds were found to positively affect the balance performance of football players and reduce the perceived difficulty level. Incorporating balance training with competition sounds during athlete rehabilitation may contribute to improved balance recovery.

Evidence of a metacognitive illusion in stimulus-specific prospective judgments of distraction by background speech

Two experiments served to examine how people arrive at stimulus-specific prospective judgments about the distracting effects of speech on cognitive performance. The direct-access account implies that people have direct metacognitive access to the cognitive effects of sounds that determine distraction. The processing-fluency account implies that people rely on the processing-fluency heuristic to predict the distracting effects of sounds on cognitive performance. To test these accounts against each other, we manipulated the processing fluency of speech by playing speech forward or backward and by playing speech in the participants’ native or a foreign language. Forward speech and native speech disrupted serial recall to the same degree as backward speech and foreign speech, respectively. However, the more fluently experienced forward speech and native speech were incorrectly predicted to be less distracting than backward speech and foreign speech. This provides evidence of a metacognitive illusion in stimulus-specific prospective judgments of distraction by speech, supporting the processing-fluency account over the direct-access account. The difference between more and less fluently experienced speech was largely absent in the participants’ global retrospective judgments of distraction, suggesting that people gain access to comparatively valid cues when experiencing the distracting effects of speech on their serial-recall performance firsthand.

Using mobile EEG to study auditory work strain during simulated surgical procedures

Surgical personnel face various stressors in the workplace, including environmental sounds. Mobile electroencephalography (EEG) offers a promising approach for objectively measuring how individuals perceive sounds. Because surgical performance does not necessarily decrease with higher levels of distraction, EEG could help guide noise reduction strategies that are independent of performance measures. In this study, we utilized mobile EEG to explore how a realistic soundscape is perceived during simulated laparoscopic surgery. To examine the varying demands placed on personnel in different situations, we manipulated the cognitive demand during the surgical task, using a memory task. To assess responses to the soundscape, we calculated event-related potentials for distinct sound events and temporal response functions for the ongoing soundscape. Although participants reported varying degrees of demand under different conditions, no significant effects were observed on surgical task performance or EEG parameters. However, changes in surgical task performance and EEG parameters over time were noted, while subjective results remained consistent over time. These findings highlight the importance of using multiple measures to fully understand the complex relationship between sound processing and cognitive demand. Furthermore, in the context of combined EEG and audio recordings in real-life scenarios, a sparse representation of the soundscape has the advantage that it can be recorded in a data-protected way compared to more detailed representations. However, it is unclear whether information get lost with sparse representations. Our results indicate that sparse and detailed representations are equally effective in eliciting neural responses. Overall, this study marks a significant step towards objectively investigating sound processing in applied settings.

Auditory, Visual, or Both? Comparing Visual and Auditory Representations of Game Elements in a Gamified Image-Tagging Task

In gamification users show increased motivation and engagement towards tasks. So far, this effect has mostly been empirically tested based on the visual depiction of game elements, while research on the use and addition of auditory aspects is sparse. In this work we investigate the effect of different modalities of game elements (auditory, visual, audiovisual) on user experience, perception and performance in a gamified image-tagging task. We approached this via an online validation survey to find suitable sound effects (N=50), a main quantitative study (N=109) and a qualitative semi-structured interview (N=9). Our results show that while visual gamification increases performance, auditory and audiovisual gamification had no effect. However, they were shown to have an influence on the user's flow state. Our qualitative follow-up study shed light on underlying reasons and revealed each modality has its own drawbacks and advantages and that combining both visual and auditory aspects was preferred by participants.

«ОПТИКА» БӨЛІМІ БОЙЫНША ЭЛЕКТРОНДЫҚ ОҚУЛЫҚТАРДЫҢ ЗАМАНУИ БІЛІМ БЕРУ ҚҰРАЛЫ РЕТІНДЕ РӨЛІ

The article examines the effectiveness and role of the use of electronic textbooks in the education system. And, including in the section «Optics», optical phenomena and patterns are taken as a basis. The creation of electronic textbooks is one of the most pressing problems of the educational process today. The expansion of the use of modern information technologies has led to the need to develop electronic learning tools that increase the effectiveness of self-learning and the quality of learning for students. The use of electronic means in the learning process is effective for both the teacher and the student. The use of electronic learning tools in the educational process does not contradict the traditional teaching method, but rather complements the use of its achievements. Comparison parameters are also considered: page and rotation; reader's movement through the book; plane and depth; book block; personal appearance of the book; text; illustration; tactile sensations; reliability of information storage and accessible information update; one can say about the pros and cons of this and other types of books. The advantages of an electronic textbook in terms of printing are accompanied only by sound effects associated with its large information capacity and ease of use. With all the convenience of the e-book format, the reader may want to see it directly on the site if he does not want to download it, or, as in a bookstore, first view the publication and then decide whether to download it or not. The main disadvantages of the electronic textbook are due to its loss of personal artistic appearance. Currently, there are many tools for creating electronic textbooks. For example, the possibilities of the program SunRav BookEditor, SunRav BookReader, TurboSite, Flipsnack, EdApp, Canva, etc. are considered. In the article, we will define the role of analyzing the digital literacy of students using an electronic textbook. Examples of electronic forms of a physics textbook for grades 7-9 can be given.

Shooting Simulator «Inhibitor»: Mathematical Support of Shot Special Effects

The algorithms for preparing shooters to perform an exercise, conducting its course with real-time informing, as well as the implementation of mathematical support of shot special effects (sounds, tracers, explosions, hits, and misses) for the optical-electronic shooting simulator 'Inhibitor' developed at Institute of Mechanics UdmFRC UB RAS and at facilities of Computer Department of Kalashnikov ISTU jointly with JSC «Kalashnikov» Concern» are described. A tactical and technical task is given for the visual and sound effects of a shot, such as tracers, grenade ruptures, misses, return fire, and controlled flight of PTURS. In addition, the manager must monitor the weapon simulator sensors during the preparation of the exercise (magazine, fire switch, cartridge inside the barrel and sight readings) and conduct the course of the exercise with real-time viewing of both targets and shots and altering the ammunition number and type (all ammunition can be made tracer ones). The arrows are selected from the department database for convenience. Viewing shooting errors and aiming along with hit points allows correction of trainees’ actions by means of commands and quick display service information for trainee independent classes on the projection screen. The study the factors affecting ballistic dispersion was carried out and the recommendations for its modeling were formulated. The literature review confirms the potential of further research and development of electronic rifle simulators via improvement of computational tools and the development of software libraries in order to increase the accuracy of shot process simulation and flexible display adjustment of current training exercise results. It is necessary to expand realistic opportunities of shot special effects and reduce the cost constantly, which means to increase the competitiveness of electronic shooting simulators.

Partial equivalent sound pressure level as an approach to manage irrelevant sounds in environmental noise measurements

Residual sound and unwanted events are known factors affecting the accuracy of environmental noise measurements. ISO 1996-2 provides methods to manage these irrelevant sounds, for example by applying correction factors, but the methods require a degree of knowledge of the irrelevant sounds that is not always practical or possible to obtain when performing long-term measurements with unattended monitoring stations in complex urban soundscapes. While automatic detection of irrelevant sounds provides the required information, it also allows for new approaches not described in ISO 1996-2. In this paper, we discuss a metric called partial equivalent sound pressure level (Partial Leq), calculated after data samples with residual sound and unwanted events are identified automatically and replaced by project-specific values. Our hypothesis is that better estimates of equivalent sound pressure levels (Leq) can be achieved with this metric, compared with the general methods given in ISO 1996-2. Comparisons of results are presented, together with a discussion of applicability of the Partial Leq metric and experiences gathered from its use.

Instrumentation and automatic evaluation of event indicators alongside railway lines: a first attempt to comply with article 90 of the French mobility legislation on transport

According to the Article 90 of the French Mobility Orientation Law (LOM) No. 2019-1428 (December 24, 2019), the Testing and Measurement Laboratory (LEM) of the Régie Autonome Des Transports Parisiens (RATP) 1 has been at the forefront in determining and evaluating indicators of annoyance due to event-related noise from railway transport infrastructure. Six measurement points located near tramway, metro, and suburban trains transport infrastructures were equipped with specific instrumentation allowing for measuring and automatically detecting railway and non-railway sound events (road traffic, alert sirens, and other sound activities), this multiplicity of noise sources necessitated a comprehensive approach to data collection and analysis, ensuring accurate assessment of the impact of railway transport infrastructure on surrounding area. Measurements were carried out continuously for 15 days. A post-processing system was also developed to analyze over a dozen noise indicators for each category of sound events. This article presents the technical solutions developed and deployed by the laboratory in a few months to address the regulatory challenges posed by the experimental phase of Article 90 of the LOM law.

Noise annoyance and application of the Harmonica index at short time intervals

The Harmonica index (HRM) is aimed to quantify the perceived noisiness of environmental sounds on an hourly basis by a score on a linear scale from quiet to very noisy. The index is computed as the sum of the background noise (BGN) and the sound events (EVT) emerging from BGN and has been developed to facilitate communication with the public and to help local authorities in the decision-making process. In this paper, the HRM index has been applied to short time intervals, providing an increase in the time resolution of sound level acquisition from 1 s to 0.1 s. Six sound recordings were considered, taken in different urban contexts (i.e., parks, road traffic noise, construction sites), each lasting two minutes and divided into four sequential soundtracks of 30 s each. The tracks were played back randomly in laboratory listening tests to collect ratings of perceived annoyance given by participants after each soundtrack and, at the end of the four soundtrack sequences, the annoyance rating of the whole sound. The results show that the short-time application of the index is sensitive to the qualitative characteristics of the background and events, and needs improving before being used.

Advanced concept for noise monitoring in smart cities through wireless sensor units with AI classification technologies

The rapid urbanization in the cities has indicated the importance of sustainable development for creating smart cities with high quality of life. Environmental noise as one of the main concerns has to be addressed according to International Directives and Legislation. Traditional noise estimation methods are costly and time-consuming, but the emerging technologies like low-cost wireless sensor units (WSUs) offer more granular data collection and analysis. Requirements for more accurate assessment of the noise pollution by detecting the dominant noise sources impose the need to apply novel approaches in the technical systems for noise monitoring. To this aim, this paper investigates the advanced methods for noise monitoring through WSUs with AI classification technologies. The proposed concept of the device not only quantitively describes the noise pollution, but also tends to recognize the class of the disturbing sound events. The accuracy of the WSUs are configured by comparing the results with professional hand-held analyzer, showing powerful and affordable low-cost units able to publish results through cloud and fog computing based on the IoT and smart city technologies. The paper discusses the challenges for merging noise measurement technology and the AI algorithms to classify and quantify the different classes of sound events.

Towards an open digital catalogue of sound signatures from the underwater soundscape

Passive acoustic monitoring plays a crucial role to study the underwater soundscape. Identifying the sound signatures of specific marine species and anthropogenic contributions is a key step in assessing the impact of human activities on the acoustic environment. The project COSS (Catalogue of Sound Signatures) aims to build an open library of sound signatures from the coastal seascape of European waters. In this framework, a data management workflow is developed within FAIR principles from the storage of the metadata, ensuring integrity and traceability of the recorded datasets, to the analysis and the annotation method, providing interoperability classification of sound signatures. An interactive map provides an overview of the database. Sound events are analyzed using a home-made annotation software. Labeling is based on a tree-structure ontology with respect to the biological species, human activities and geophony. This labels standardization makes it possible to extract information at several levels of precision. In particular, this library will serve as a learning and testing base for open source AI algorithms for the automatic classification of signals of anthropogenic origin. A long-term objective is to provide a means of determining the acoustic impact of human activities on the marine environment.

Study on frequency dependent convolution methods for sound event detection

Sound event detection (SED) aims to classify target sound events from a given audio clip with the corresponding onset and offset time localization. SED, a core research field in machine listening, has been rapidly growing by adopting the methods by applying deep learning methods in image recognition fields. One representative example is 2D convolution which is applied to 2D time-frequency audio features. However, 2D audio data exhibits physical discrepancies from 2D image data, one of which is that frequency dimension of 2D audio data is translational-variant while position dimension of 2D image data is translational-invariant. Therefore, to make 2D convolution more consistent to 2D audio data, we need to make 2D convolution dependent to frequency dimension. Previously, to address this problem, Frequency Dynamic Convolution (FDYConv) was proposed to replaces 2D convolution. FDYConv is dependent on frequency dimensions since it applies different kernels on frequency dimension. While it has proven its outstanding performance on SED, there are several more alternative frequency dependent convolution methods which are yet to be explored, In this work, we propose other simpler frequency dependent convolution methods and compare them with FDYConv in order to further investigate the effect of frequency dependence of 2D convolution methods.

Enhanced railway sound event detection using YAMNet and classification criteria

The segmentation-by-classification method has become a popular way to detect sound events. It uses neural networks, trained to detect sound sources on short signals (one tenth of a second to a second), or long-term signals divided into a succession of short segments. In this paper, we focus on detecting train pass-bys from long term signals, where it is assumed that railway noise is higher than ambient noise. Using the neural network YAMNet, we show that a processing stage is necessary to improve the segmentation of such events and reduce the high false positive rate. The applied criteria are related to the nature of a train pass-by, lasting several seconds and with broad frequency band. Around 90% of events are detected with proper boundaries. However, we observe that YAMNet is not designed to distinguish railway vehicles from other vehicles. False positive rate remains high whenever other vehicles travel near the measurement location. Additional AI models are proposed to perform this specific distinction. Their efficiency depends on train type, as recent passenger train noise resembles roadway noise. Otherwise, false positive rate decreases below 10%, and the railway noise contribution estimate aligns with the reference level, within a 0.5 dB(A) margin on average.

Sound imaging for car/mobility sounds by generative artificial intelligence

Use of generative artificial intelligence becomes popular in recent days. Especially the area of producing image, chatting and sound effects are commonly recognized to use. However it is not so much extended into product design or combinations of image and sounds. In this paper, application examples to design car sound with generative artificial intelligence are presented. These approaches are mixed usage with image generative AI, sound generative AI and sound modification, it leads creative way of producing sound examples for future car and mobility. Lastly, with respect to theme of inter-noise 2024, special sound design for Jules Vernes world will be presented.

Assessing biochar, clinoptilolite zeolite and zeo-char loaded nano-nitrogen for boosting growth performance and biochemical ingredients of peace lily (Spathiphyllum Wallisii) plant under water shortage

BackgroundPeace lily (Spathiphyllum wallisii Regel) is an ornamental indoor plant with promising cut flower market, as well as antiviral, pharmacological and ecological potentials. Water deficiency can have sound effects on the growth performance and aesthetic quality of such plant. The aim of this study was to investigate the consequences of zeolite, biochar, and zeo-char loaded nano-nitrogen application on the growth performance and biochemical components of peace lily under water shortage conditions. An experiment was conducted over two consecutive seasons (2021–2022) at the experimental nursery of Ornamental Horticulture Department, Faculty of Agriculture, Cairo University, Giza, Egypt. Soil amendments; zeolite, biochar, and zeo-char loaded nano-nitrogen were prepared and applied to soil before cultivation.ResultsOur results revealed that the new combination treatment (zeo-char loaded nano-N) had an exceeding significant effect on most of the studied parameters. Vegetative traits such as plant height (35.7 and 35.9%), leaf number per plant (73.3 and 52.6%), leaf area (40.2 and 36.4%), stem diameter (28.7 and 27.1%), root number (100 and 43.5%) and length (105.7 and 101.9%) per plant, and fresh weight of leaves (23.2 and 21.6%) were significantly higher than control (commercially recommended dose of NPK) with the application of zeo-char loaded nano-N during the two growing seasons, respectively. Similar significant increments were obtained for some macro- (N, P, K, Mg, Ca) and micro- (Fe, Zn, Mn) elements with the same treatment relative to control. Chlorophyll (18.4%) and total carotenoids (82.9 and 32.6%), total carbohydrates (53.3 and 37.4%), phenolics (54.4 and 86.9%), flavonoids (31.7% and 41.8%) and tannins (69.2 and 50%), in addition to the phytohormone gibberellic acid (GA3) followed the same trend with the application of zeo-char loaded nano-N, increasing significantly over control. Leaf histological parameters and anatomical structure were enhanced with the new combination treatment in comparison with control. Antioxidant enzymes (catalase and peroxidase), proline and abscisic acid (ABA) exhibited significant declines with zeo-char loaded nano-N treatment relative to control.ConclusionThese findings suggest that incorporating soil amendments with nano- nutrients could provide a promising approach towards improving growth performance and quality of ornamental, medicinal and aromatic species under water deficiency conditions.