Sound Mixing Research Articles

Development of automatic audio panning system for immersive sound through stage size-aware model and object-tracking

We investigated a novel AI-assisted automated 'immersive' audio panning system designed to track audio-related objects within a video clip. This system comprises four sequential steps: Object-Tracking, Stage dimension Estimation, XY-Coordinate Calculation, and Object Audio Rendering. The system is designed to overcome existing challenges arising from the rapid and frequent movement of target objects by employing a pre-trained object-tracking model and integrating depth information to ensure stability in subsequent tasks. Additionally, we introduce a stage size-aware model to extrapolate stage dimension using our manually collected dataset, formatted as (Image, Width, Depth), which facilitates model training. Consequently, the system calculates XY-Coordinate pairs, serving as panning values for conventional audio mixers or decoders to enable immersive audio reproduction. We anticipate that this video- and space-aware automatic panning system will be valuable for the rapid production of new media.

Designing audio processing strategies to enhance cochlear implant users' music enjoyment

Cochlear implants (CIs) provide hundreds of thousands of users with increased access to sound, particularly speech, but experiences of music are more varied. Can greater engagement by CI users in the music mixing process mutually benefit them as well as audio engineers? This paper presents an exploratory mixed-methods study that leverages insights from CI users and professional audio engineers (AEs) in the investigation of techniques to enhance CI user music enjoyment and promote empathetic practices in AEs. Analysis of data collected over a multi-step process of surveys, interviews, and iterative cycles of sound mixing and feedback revealed two mixing strategies—namely essentializing and exaggeration of musical elements—common among mixes preferred by CI users. Participant responses also highlighted systemic ableism and audism in the music industry—an industry from which CI users report feeling excluded. Specifically, AEs felt inadequately trained around aural diversity considerations and experienced anxiety around their own aural diversity. In sum, this work contributes to insights into CI user music listening preferences; how AEs approach mixing for an aurally diverse audience; and discussion around the efficacy of CI simulation, user feedback, and AE peer feedback on mix enjoyment by CI users. Based on these findings, we offer several design insights that emphasize the need for customizable music listening tools centered around user agency and enjoyment.

Introduction to Foley sound

Foley sound effects – those sounds that are created in post-production through live performance and then added to the sound mix of films, television, video games and animation – are ubiquitous. But effective Foley should not be heard. It is one of those creative labours in the making of a film that, to be considered successful, must erase itself. While film studies has begun to redress theory and practice’s bias towards the visual, film sound studies has yet to sufficiently acknowledge this hidden body in the sound process and fully reckon with the individual contribution made by the Foley artist to cinematic expression and cinema’s capacity to represent the world in motion and the exertion of bodies on-screen. To the extent we consider sound cinematic, what does the Foley artist contribute when they supplement images of a rock, a chair, a phone with sound or pay attention to the way paper brushes wood, metal rattles against metal, sand pushes back against a finger? This introduction establishes how refocusing attention to Foley sound extends our understanding of cinema as a medium, as well as addressing the current cultural and scholarly ‘visibility’ of Foley sound. It outlines the way the issue seeks to open up new theoretical perspectives on Foley sound and explore its interdisciplinary potential for theories of the soundtrack and film theory more broadly.

Advanced emotion analysis: harnessing facial image processing and speech recognition through deep learning

The human face hides many secrets and is one of the most expressive human features. Human faces even contain hidden information about a person's personality. Considering the fundamental role of the human face, it is necessary to prepare appropriate deep-learning solutions that analyze human face data. This technology is becoming increasingly common in many industries, such as online retail, advertising testing, virtual makeovers, etc. For example, facial analysis technology now allows online shoppers to virtually apply makeup and try on jewelry or new glasses to get an accurate picture of what these products will look like. The human sense of hearing is a treasure trove of information about the current environment and the location and properties of sound-producing objects. For instance, we effortlessly absorb the sounds of birds singing outside the window, traffic passing in the distance, or the lyrics of a song on the radio. The human auditory system can process the intricate mix of sounds reaching our ears and create high-level abstractions of the environment by analyzing and grouping measured sensory signals. The process of obtaining segregation and identifying sources of a received complex acoustic signal, known as sound scene analysis, is a domain where the power of deep learning shines. The machine implementation of this functionality (separation and classification of sound sources) is pivotal in applications such as speech recognition in noise, automatic music transcription, searching and retrieving multimedia data, or recognizing emotions in statements.

Tunable active noise control circuit topology for multiple-feature applications

A tunable active-noise-control (ANC) circuit topology for headsets used in different applications is presented in this article. In the current consumer headset market, ANC is a mature technology that is commonly applied to wireless headsets connected to smartphones for listening to music and making phone calls. The development of ANC headsets has resulted in low-cost and simple devices due to the built-in ANC registers in the digital circuit. Digital circuit implementation is important to provide ultra-low latency processing for different algorithms compared with using digital signal processors (DSPs). However, a limitation associated with these built-in ANC filters is that the ANC digital circuit has been designed only for the wireless headset consumer market. Consequently, ANC headsets can only be used for designated features. ANC tuned for planes by eliminating low-frequency engine noise has been commonly used in the past twenty years. Pass-through (PT) amplification is also commonly tuned to allow users to hear external sounds and have conversations without having to remove their headset. In addition, ANC and PT amplification are also not allowed to operate simultaneously on the existing chipsets. In this paper, an ANC circuit topology with ultra-low latency processing is presented, where the ANC response of the headset can be tuned without any restriction imposed by the built-in ANC filters. The proposed ANC circuit topology consists of a commercial ANC chipset with an external audio codec to provide tunable characteristics. It is achieved by fixed maximum noise cancellation of ANC chipset being modified by summing with the signal from audio-codec path via an audio mixer. Acoustic performance for several headset applications implemented using the proposed circuit is validated. This simple circuit topology can be easily developed and adapted to the current headset consumer market for consumer-selective ANC in different environments.

Bad Bridget—An unexplored aspect of the Irish migration story

AbstractThis paper examines how museums can address and raise awareness of histories of incarceration through a case study—the Bad Bridget exhibition, which opened at the Ulster American Folk Park in Omagh, Northern Ireland, in April 2022. Originating as an academic research project (2015–2019) led by Dr Leanne McCormick (Ulster University) and Dr Elaine Farrell (Queen's University Belfast), it tells the stories of women who left Ireland for North America between 1838 and 1918 and ended up in trouble with the authorities for one reason or another. A unique mix of sights, sounds, and smells, further enhanced by first‐person narration, is used within the exhibition to encourage visitors to emotionally engage with the experiences of these women. For much of the nineteenth century, Irish‐born migrants were the biggest group in American prisons, and there were disproportionate numbers of Irish girls and women in the justice system, court, and prison (Farrell & McCormick, ‘Bad Bridgets’: The criminal and deviant Irish women convicted in America. Irish Times, 2019). Bad Bridget has provided us with a platform to reveal this previously unexplored aspect of the Irish migration story at the museum, contrary to the “American Dream.”

Digital Media Convergence Of Broadcast Radio

The existence of media convergence is an effort to adapt radio to the digital era. Although many people think that radio will no longer survive in the midst of technological and information developments, in fact, many radios still survive and have loyal listeners and partnerships to use radio advertising services. This form of research aims to find out the application of media convergence carried out by Shelter 95.6 FM Radio's public relations to maintain its existence amid the presence of various new media as competitors and the development of technology and information. This research uses qualitative research methods and data collection by interviewing 2 sources with public relations and broadcasters and descriptive observations and data collection techniques through previous literature studies, strengthened by the Media Convergence theory developed by Henry Jenkins through the 3C approach (computing, communication & content). Some of the applications made by Shelter FM to compete with other media are the application of computing convergence media Shelter FM has adapted by broadcasting Shelter FM outside with only a laptop, microphone, and audio mixer and the application of communication convergence media live radio broadcasts and also streaming Shelter FM radio in the Shelter FM application and content convergence media by delivering information and broadcasting using digital technology, namely social media to expand networks and audiences.

The Shaping of Sound Focus in Film and Television Sound Mixing

Sound Focus aims to adopt the concept of "focus" as the core reference of sound creation. Solve the problem of how art and technology are combined in sound production. Sound focus can "break the surface with the point" in the process of sound optimization to achieve a clear artistic expression. From the perspective of film and television sound mixing creation, this paper explains the means of displaying sound focus in the process of mixing and shrinking in the later stage of the film. And achieve the goal of improving the overall sound production effect.

The Impact of Audio Voice Characteristics on Audience Positive Emotions in Podcasting Analysis

In this study, the researchers directed their attention towards the phenomenon of sound. Specifically, they sought to examine the emotional responses of the participants by analyzing electrodermal responses (EDRs). The study involved the comparison of two distinct sound conditions. Initially, it was observed that there was no discernible variation in the Electrodermal Response (EDR) pattern across different conditions. However, upon careful examination of the peak EDR outputs observed in various simulations, it became evident that the emotional content had a discernible impact on the spatial conditions. Furthermore, it was particularly noteworthy that the interaction between sound and the aforementioned spatial conditions played a significant role. It was observed that no distinct individuals stimulators could be identified, although there were notable trends and variations in Emotional Dysregulation Rating (EDR) under specific circumstances. In addition, it is worth noting that the participants in the study recognized the most prevalent instances of positive emotions across all experimental conditions. Furthermore, the utilization of a combination of arousal sound design and 3D sound mix elicited the greatest number of emotional responses. Conversely, the omission of data regarding participants' educational level, cultural background, and gender prevents a comprehensive exploration of these factors, thereby limiting the potential for generating more nuanced insights.

Improving learning-based birdsong classification by utilizing combined audio augmentation strategies

In ecology, changes in environmental conditions are often closely linked to shifts in species diversity. This relationship can be investigated by analyzing avian vocalizations, which are robust indicators of trends in biodiversity. Within this contribution, we explored various data augmentation techniques and deep learning strategies for the classification of birdsong within natural soundscapes. For this purpose, we employed three fundamental deep neural network architectures, such as vision transformers, to classify 397 different bird species. To improve both the accuracy and generalizability of our models, we incorporated up to 19 well-established data augmentation techniques commonly used in audio classification. This included an iterative selection process where only augmentations that enhanced classification performance were selected. The primary augmentation technique involved the integration of various noise samples and non-bird audio elements, which significantly improved model performance as assessed on the BirdCLEF 2021 data set. Individual augmentations achieved F1-scores from 48.0 % (vertical flip) to 72.6 % (primary background noise soundscapes). Through the strategic combination of key techniques – namely simulated pink noise, interspecies sound mixing, and loudness normalization – we achieved a top F1-score of 73.7%. Depending on the selected classification model, this corresponds to an improvement by 4.81 % to 10.5 %. Improvements and deteriorations of all applied augmentation techniques appeared to be robust across our three evaluated models. Therefore, our approach highlights the potential of sophisticated audio augmentations in refining the accuracy and robustness of birdsong classification models.

Trainer Instalasi Audio Three Way Pada Sound System Untuk Media Pembelajaran Sistem Audio

In the world of education, practical activities are very important. This activity requires students to carry out observations, experiments and testing after obtaining the theories that have been taught. This addie method is used to determine the suitability of trainers with the help of a questionnaire. The research is in the form of an audio trainer system consisting of a tone control as a tone controller, with an OCL amplifier for the final connection, as well as a preamp for the initial connection, a mixer, and an output in the form of a speaker as well as troubleshooting for error simulation. Exercise book module to make it easier to use the Audio System Trainer. This test produced several data, namely the gain voltage on the preamp was 16.8 times at a frequency of 1 KHz, the audio mixer was 15.7 times at a frequency of 1 KHz and the OCL amplifier was 18.6 times at a frequency of 1 KHz. Audio System Trainer has a learning media appropriateness level of 92% according to experts and 88% according to students. This shows that this trainer can be applied in Audio Systems learning.

What does the fox say? Arctic fox vocalization and associated den behaviours

Foxes (Vulpes spp.) are small, solitary canids with relatively low social complexity compared to more gregarious canids, such as wolves and dogs. They are, therefore, expected to have a relatively simple vocal repertoire, with limited low-intensity sounds for close communication and many high-intensity sounds for long-distance communication. Arctic foxes (Vulpes lagopus), like many other foxes, are largely solitary outside of the breeding season. However, they have the largest litter size in the order Carnivora and may experience enhanced social complexity during the breeding season. In this study, we document the vocal repertoire of the Arctic fox during the breeding season, and how it changes before and after the emergence of pups. We also describe the relationship between vocalizations and other denning behaviours. Camera-traps captured six distinct sounds produced by breeding pairs of Arctic foxes and their young at dens: territorial barks, warning barks, alarm calls, cooing, whines and growling. Our study shows that although high-intensity sounds, such as territorial barks, are an important form of long-distance communication among Arctic foxes, low-intensity sounds and sound mixing are used on their dens following pup emergence. Thus, Arctic fox vocalization may be more complex than previously documented.

Composer Kajiura Yuki and neo-medieval anime soundtracks

This study semiotically interrogates the historical imaginary evinced in the neo-medievalist musical topoi found in Kajiura Yuki’s distinctive music for anime, which is easily recognised by its eclectic mix of sounds and styles gathered from across the globe. Her early scores employed a compositional method practised in Japanese popular music since the 1990s, which treated the creative act as a process of musical curation. This technique is evident in Kajiura’s handling of medieval Gregorian chant, which - as she has explained in interviews - she did not learn from studies in music history but rather from the German band Enigma and their hit album MCMXC a.D. from 1990, where samples of chant were mixed with Euro dance pop and French rap. The anime Noir from 2001 contains an excellent example of her approach, combining chant-based vocal tracks with energetic dance rhythms. Enigma used chant to call on modern neo-medieval tropes that highlight the pleasures of mysticism, religious devotion, and sexuality freed from morality, and Kajiura has replicated this imagery in Noir, making chant the symbol of an ancient criminal order that both worships and overtly sexualises femininity as embodied by the main female characters. Kajiura’s later style expands the technique of the Gregorian chant-influenced sound she developed in her earlier works: shifting away from Latin lyrics to her invented nonsense language of “Kajiurago” (literally “the language of Kajiura” in Japanese), with her ethereal chant delivered primarily by female voices. This shift is partially due to her collaboration with FictionJunction and Kalafina and is also a reflection of the strong female anime protagonists. Her signature sound enlivens the soundscapes of both Fate/Zero (2011) and the blockbuster anime, Demon Slayer (2019). The track “Brace up and run!” opens each episode of the latter, highlighting female voices chanting Kajiurago as a musically haunting reminder of the otherness of the past.

Adapted Original Music as an Environmental Enrichment in an Intensive Pig Production System Reduced Aggression in Weaned Pigs during Regrouping

In intensive swine production systems,, the practice of regrouping unfamiliar pigs is common, often leading to aggressive behavior. Although the effect of different musical genres composed for humans has been evaluated in pigs to mitigate aggression, there have been few attempts to create music specifically for pigs. Here, we assess whether sensory stimulation through music, created by adapting the acoustic parameters in the sound mix, induces changes in the aggressive behaviors of pigs during regrouping. Six litters of 10-week-old piglets were randomly selected and assigned to different treatments. The control group (Group A) received no intervention, while Group B was exposed to music for two continuous hours in the morning and afternoon from the time of regrouping. Group C received musical stimulation for one continuous hour in the morning following regrouping. A significant reduction in the frequency and duration of aggressive behaviors was observed in the groups that received musical stimulation during regrouping. Additionally, social, and individual play behaviors showed a decrease in the musical stimulation groups. These findings provide evidence for the effectiveness of created music as a strategy in reducing aggressive behavior during pig regrouping, which can enhance the welfare of pigs and offer a practical solution for pig producers to minimize aggression and its associated negative impacts.

Reduksi Buta Aksara melalui aplikasi Magguru Mabbaca pada kelompok remaja masjid di Desa Pationgi

Illiteracy is a crucial issue today, especially in Indonesia. The high illiteracy rate in Indonesia reaches 3.56% of the population. One of the most significant illiteracy issues is in Pationgi Village, Patimpeng District. Illiteracy is experienced by many teenagers, especially those who are not in school. The culture of the community also contributes to the high illiteracy rate in Pationgi Village. Most parents have never been to school or tried to get an education, which makes kids reluctant to go to school, leading many people who cannot read or write. The purpose of this community service is to reduce illiteracy through community service activities and the assistance of the Magguru Mabbaca application. Magguru Mabbaca is a mix of data, video, audio, text, and sounds that can be seen and heard on a smartphone. Magguru Mabbaca was developed to allow partners to understand and recognize letters and numbers quickly. Integrating the Bugis language with Magguru Mabbaca will assist the Pationgi Village community, which is still illiterate. Counseling at partner locations, training with two stages of training beginning with the introduction and operation of applications, and assistance to see an increase in partners' knowledge and understanding of reading and arithmetic are all part of the implementation methods. The result of this community service is that partners experience a significant increase in the knowledge component and application of technology, with an increase of 85% and 100%, respectively. This shows that the Magguru Mabbaca application helped partners improve their reading and math skills in a big way. Therefore, Magguru Mabbaca can be a solution for people in South Sulawesi who still have a high quantity of illiterates, which can be reduced by introducing holistic applications.

Modern movie sound: reality and simulated reality

Modern movie sound mixing utilizes impulse responses and convolution to provide a usable reproduction of dialogue and other sounds to align the visual and sonic impression while keeping the viewer engaged in the story. This paper will outline the process by which movie audio is processed to get to the final product, and provide and overview of playback platforms. Furthermore short clips of film with audio will be presented to compare the actual in situ recording to the studio mixed product, and other experimental/virtual approaches to creating impulse responses for use in the studio mix approach.

Unsupervised detection and classification of heartbeats using the dissimilarity matrix in PCG signals

Background and objective: Auscultation is the first technique applied to the early diagnose of any cardiovascular disease (CVD) in rural areas and poor-resources countries because of its low cost and non-invasiveness. However, it highly depends on the physician’s expertise to recognize specific heart sounds heard through the stethoscope. The analysis of phonocardiogram (PCG) signals attempts to segment each cardiac cycle into the four cardiac states (S1, systole, S2 and diastole) in order to develop automatic systems applied to an efficient and reliable detection and classification of heartbeats. In this work, we propose an unsupervised approach, based on time-frequency characteristics shown by cardiac sounds, to detect and classify heartbeats S1 and S2.Methods: The proposed system consists of a two-stage cascade. The first stage performs a rough heartbeat detection while the second stage refines the previous one, improving the temporal localization and also classifying the heartbeats into types S1 and S2. The first contribution is a novel approach that combines the dissimilarity matrix with the frame-level spectral divergence to locate heartbeats using the repetitiveness shown by the heart sounds and the temporal relationships between the intervals defined by the events S1/S2 and non-S1/S2 (systole and diastole). The second contribution is a verification-correction-classification process based on a sliding window that allows the preservation of the temporal structure of the cardiac cycle in order to be applied in the heart sound classification. The proposed method has been assessed using the open access databases PASCAL, CirCor DigiScope Phonocardiogram and an additional sound mixing procedure considering both Additive White Gaussian Noise (AWGN) and different kinds of clinical ambient noises from a commercial database.Results: The proposed method outperforms the detection and classification performance of other recent state-of-the-art methods. Although our proposal achieves the best average accuracy for PCG signals without cardiac abnormalities, 99.4% in heartbeat detection and 97.2% in heartbeat classification, its worst average accuracy is always above 92% for PCG signals with cardiac abnormalities, signifying an improvement in heartbeat detection/classification above 10% compared to the other state-of-the-art methods evaluated.Conclusions: The proposed method provides the best detection/classification performance in realistic scenarios where the presence of cardiac anomalies as well as different types of clinical environmental noises are active in the PCG signal. Of note, the promising modelling of the temporal structures of the heart provided by the dissimilarity matrix together with the frame-level spectral divergence, as well as the removal of a significant number of spurious heart events and recovery of missing heart events, both corrected by the proposed verification-correction-classification algorithm, suggest that our proposal is a successful tool to be applied in heart segmentation.

Gagaku, Ensemble OnoTeruhisa Fukuda, maître de shakuhachi. Offrande musicale

Gagaku, Ensemble OnoTeruhisa Fukuda, maître de shakuhachi. Offrande musicale

Masculinity and underground music scene participation across time: A case study from Indonesia

The Ujung Berung (Bandung, Indonesia) death metal scene arguably peaked in the years 2009–13. Based on personal interviews and conversations with musicians, we identify four factors adversely affecting the scene at present: (1) decline in community nongkrong/relocation of Serak store at Jalan Cihampelas; (2) those left behind – the disillusionment of the older second-division bands; (3) the new generation’s attitude and perspective and (4) the turn towards religion. We also explore whether Indonesian masculinity is in crisis, because of young Indonesian men’s subordinated place within global capitalism and the threat that capitalism poses to traditional hierarchies. Hypermasculine death metal culture inspires because it is western in origin but not bourgeois – it is a mysterious mix of signs, symbols, sounds and mythologies, which attracts by its very obscurity; it quietly challenges, in its indirect way, the power of Indonesian hegemonic identities such as ‘pious Islamic family-man’ and ‘businessman’.

Section Meetings

On Tuesday, 23 November, John Gallagher moderated a panel discussion about the making of the film <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Apollo 11</i> , specifically the IMAX version. The panel consisted of the key production crew involved in the film, including director Todd Douglas Miller, Final Frame chief executive officer and colorist Will Cox, and sound mixer Eric Milano. The meeting was held virtually with 49 participants.