Sound Design Research Articles

Composing perception: Hildegard Westerkamp meets Gus Van Sant (a video essay)

This video essay explores the relationship between the soundscape compositions of Hildegard Westerkamp, the films of Gus Van Sant, and the writing of Danijela Kulezic-Wilson. The piece presents Westerkamp’s composition “Doors of Perception” in its entirety with accompanying sequences from the Van Sant films that have used her work as part of their sound design, along with other of his films that do not. The sequences from Elephant and Last Days are positioned according to the timing of the excerpts used in the films, only now it is the films that are integrated into the soundscape composition rather than the other way around. For the rest of the composition, I speculate on what it might have been like to hear “Doors of Perception” in other films by Van Sant, excerpting a few sequences from across his filmography loosely synchronised around a few key points that suggest correlation between sound and image in similar ways to how Van Sant mapped the composition into Elephant and Last Days. These thematic connections support Danijela Kulezic-Wilson’s arguments about the new perceptual engagements opened up under Van Sant’s influence by soundwalking and music concrète within his filmmaking practice.

On Drill Team and the musicality of videographic criticism

An article comprising the videographic study Drill Team alongside a written framing of its film studies origins and comparative methodology. The video attempts a performative audiovisual adaptation of a chapter section from Danijela Kulezic-Wilson’s 2020 book Sound Design is the New Score: Theory, Aesthetics, and Erotics of the Integrated Soundtrack. Drill Team explores both Kulezic-Wilson’s specific (if uncertain) argument about the relationship between two films, Beau Travail (Claire Denis, 1999) and The Fits (Anna Rose Holmer, 2015), and her general argument in the chapter about the “composed” nature of the sound (and music) design in these and other modern or contemporary films, as well as about the capacity of these films, and film generally, to use modes of embodied experience as the substance of their language.

Music in “reticent” cinema

Expanding on ideas presented by Danijela Kulezic-Wilson in a chapter of her 2020 book Sound Design Is the New Score, this article explores the nature of “reticent” films both old and new, and it suggests that often it is because their soundtracks have so little to “say” that these films communicate so very, very much.

A full-scale complementary dynamic comparator

Abstract This paper presents a dynamic comparator that achieves precise comparison across the entire full-scale range. The comparator, an essential component module in analog-to-digital converters (ADCs), is influenced by its input signal range, speed, and offset voltage, all of which impact the ADC’s performance. In the proposed comparator, a pre-amplification stage using PMOS as the input is incorporated alongside the comparator utilizing NMOS as the input, creating a complementary structure. This design ensures that at least one pre-amplification stage operates effectively, thereby maintaining the accuracy of the comparator as the input signal varies across the entire full-scale voltage range (0 to VDD). Through sound circuit design, the proposed comparator effectively avoids introducing additional distortion and noise. The performance of the comparator is verified under the 180 nm process. The comparator achieves accurate comparison across the entire full-scale range, with a standard deviation σ of the input offset voltage of 6 mV, noise of 450 uV, and a layout area of 467.36 um2. Simulation results demonstrate that the comparator exhibits excellent performance, providing stability assurance for the ADC and extending the ADC’s input signal range.

Spatial sound and emotions: A literature survey on the relationship between spatially rendered audio and listeners’ affective responses

With the development of the entertainment industry, the need for immersive and emotionally impactful sound design has emerged. Utilization of spatial sound is potentially the next step to improve the audio experiences for listeners in terms of their emotional engagement. Hence, the relationship between spatial audio characteristics and emotional responses of the listeners has been the main focus of several recent studies. This paper provides a systematic overview of the above reports, including the analysis of commonly utilized methodology and technology. The survey was undertaken using four literature repositories, namely, Google Scholar, Scopus, IEEE Xplore, and AES E-Library. The overviewed papers were selected according to the empirical validity and quality of the reported studies. According to the survey outcomes, there is growing evidence of a positive influence of the selected spatial audio characteristics on the listeners’ affective responses. However, more data is required to build reliable, universal, and useful models explaining the above relationship. Furthermore, the two research trends on this topic were identified. Namely, the studies undertaken so far can be classified as either technology-oriented or technology-agnostic, depending on the research questions or experimental factors examined. Prospective future research directions regarding this topic are identified and discussed. They include better utilization of scene-based paradigms, affective computing techniques, and exploring the emotional effects of dynamic changes in spatial audio scenes.

A Natural Sonification Mapping for Handwriting

The sonification of handwriting has been shown effective in various learning tasks. In this paper, the authors investigate the sound design used for handwriting interaction based on a simple and cost-efficient prototype. The authentic interaction sound is compared with physically informed sonification designs that employ either natural or inverted mapping. In an experiment, participants copied text and drawings. The authors found simple measures of the structure-borne audio signal that showed how participants were affected in their movements, but only when drawing. In contrast, participants rated the sound features differently only for writing. The authentic interaction sound generally scored best, followed by a natural sonification mapping.

Connecting Sound to Data: Sonification Workshop Methods With Expert and Non-Expert Participants

Sonification and sonic interaction design aim to create meaningful displays and digital interactions using data and information from the most disparate fields (astronomy, finance, health, and security, for example) as the basis of the design. To date, there are no standards and conventions on how to meaningfully link data to sound; therefore, designers develop these connections on a case-by-case basis. Participatory workshops that target end users and domain experts are a way for sound designers to find meaningful connections between data and sounds at the start of the design process so that final outcomes are more likely to be effective and accepted by users. In this paper, the authors present and discuss the participatory workshop methods they have developed within the Sound for Energy project. In particular, they will highlight the aspects that can be easily transferable to other target domains. With this, the authors contribute to the effort of making sonification and sonic interaction design a more viable and accepted alternative to traditional, usually visual, displays.

Sound Terminology in Sonification

Sonification research is intrinsically interdisciplinary. Consequently, a proper documentation of and interdisciplinary discourse about a sonification is often hindered by terminology discrepancies between involved disciplines, i.e., the lack of a common sound terminology in sonification research. Without a common ground, a researcher from one discipline may have trouble understanding the implementation and imagining the resulting sound perception of a sonification, if the sonification is described by a researcher from another discipline. To find a common ground, the author consulted literature on interdisciplinary research and discourse, identified problems that occur in sonification, and applied the recommended solutions. As a result, the author recommends considering three aspects of sonification individually, namely 1) Sound Design Concept, 2) Objective, and 3) Evaluation, clarifying which discipline is involved in which aspect and sticking to this discipline’s terminology. As two requirements of sonifications are that they are a) reproducible and b) interpretable, the author recommends documenting and discussing every sonification design once using audio engineering terminology and once using psychoacoustic terminology. The appendixes provide comprehensive lists of sound terms from both disciplines, together with relevant literature and a clarification of often misunderstood and misused terms.

Mobile application-based phonetic training facilitates Chinese-English learners’ learning of L2

BackgroundNumerous studies have claimed that high variability phonetic training (HVPT) has significant efficacy in improving learners’ phonological decoding ability. Although mobile learning (ML) based HVPT program integrated with sound instructional design also demonstrated the effectiveness, its reliability and validity need to be further verified. AimsThis study evaluated the training's efficacy in improving Chinese learners' perception and production of English words by comparing learners' brain activation changes and network connectivity changes because of the training. SamplesParticipants were 40 undergraduate students. MethodsThis study developed a Wechat application to implement a self-regulated high-variability-phonetic-training (HVPT) for 20 university students; employed a modified phonics screening test (MPST) as a benchmark test to assess learners’ phonetic decoding ability before and after the training; designed a quasi-experiment (control/experimental design) for 40 participants; most importantly, this study used a functional near infrared spectroscopy (fNIRS) device to monitor brain activation and networks synchronisation for participants in both groups before and after training. ResultsThe results showed that learners' performance on 5-phoneme pseudowords was significantly improved. The training inhibited learners’ right fusiform gyrus. A significant correlation was observed between enhanced RMTG and increased testing scores. ConclusionThe transfer effect of Mobile MVPT on long pseudowords is significant, suggesting its efficacy in helping learners apply the phonetic patterns perceived from the training to segmentally analyse new difficult words.

Exploring Therapeutic Potential of Catalase: Strategies in Disease Prevention and Management.

The antioxidant defense mechanisms play a critical role in mitigating the deleterious effects of reactive oxygen species (ROS). Catalase stands out as a paramount enzymatic antioxidant. It efficiently catalyzes the decomposition of hydrogen peroxide (H2O2) into water and oxygen, a potentially harmful byproduct of cellular metabolism. This reaction detoxifies H2O2 and prevents oxidative damage. Catalase has been extensively studied as a therapeutic antioxidant. Its applications range from direct supplementation in conditions characterized by oxidative stress to gene therapy approaches to enhance endogenous catalase activity. The enzyme's stability, bioavailability, and the specificity of its delivery to target tissues are significant hurdles. Furthermore, studies employing conventional catalase formulations often face issues related to enzyme purity, activity, and longevity in the biological milieu. Addressing these challenges necessitates rigorous scientific inquiry and well-designed clinical trials. Such trials must be underpinned by sound experimental designs, incorporating advanced catalase formulations or novel delivery systems that can overcome existing limitations. Enhancing catalase's stability, specificity, and longevity in vivo could unlock its full therapeutic potential. It is necessary to understand the role of catalase in disease-specific contexts, paving the way for precision antioxidant therapy that could significantly impact the treatment of diseases associated with oxidative stress.

The Effect of an Emotionalizing Sound Design on the Driver’s Choice of Headway in a Driving Simulator

This study investigates the impact of emotionalizing sound design on driving behaviour, focusing on the effect of an acoustic stimulus that varies from positive to negative/threatening based on the vehicle’s time headway (THW). Our primary goal was to explore how this sound influences driving durations within specific THW ranges and the mean THW itself. The experiment utilized a control group and a within-participant setting across simulated driving scenarios. The statistical analysis showed mixed results. While participants in the control group setup did not demonstrate significant reductions in the durations of driving in lower THW ranges, a modest but significant increase in mean THW was observed when the emotionalizing sound was active. However, within-participant comparisons showed both a significant decrease in the duration of driving at lower THWs and an increase in mean THW when the negative stimulus was active, suggesting the stimulus’ effectiveness in promoting safer driving habits. These findings highlight the potential of emotionalizing sound design to influence driver behaviour towards maintaining safer distances, although the impact appears to diminish at higher THW ranges. Future research should further investigate the characteristics of sounds that effectively modify driving behaviour, aiming for broader applications in traffic safety.

Hydrogen-assisted recycling of Nd-Fe-B magnets from the end-of-life audio products

The transition towards green and sustainable technologies has significantly increased the demand for a subset of raw materials known as critical elements. Rare earth elements with their main use in Neodymium-Iron-Boron magnets (NdFeB) play a key role in clean energy technologies including electric vehicles and wind turbine generators. They are also a key component in electronic devices including mobile phones, hard disk drives, and loudspeakers. Here, we build on previous knowledge of the recycling of NdFeB magnets from hard disk drives, using a hydrogen process called HPMS (Hydrogen Processing of Magnet Scrap). This study investigates a new scrap stream from loudspeakers, specifically from vehicles and flat-screen TVs. The extracted magnets from loudspeakers typically had grades of N30/N33 with relatively low coercivity values. The interactions of the extracted magnets with hydrogen were monitored by gravimetric and thermal analyses. The thermodynamics and kinetics of the hydrogenation reaction appear to differ significantly based on their chemical composition, chemical homogeneity, coating state, and surface condition of the magnets. Overall, the HPMS process was demonstrated to successfully produce demagnetised hydrogen decrepitated powder at room temperature under 2 bar of hydrogen, in a commercially viable timeframe (approx. 4 h). Zinc was found to be the dominant coating material which peeled away during the HPMS process. The zinc content of the NdFeB alloy powder could be reduced after the hydrogen decrepitation by sieving. The remaining level of zinc had no noticeable impact on the magnetic properties of the recycled magnets. The particle size of the recycled alloy powder could be reduced to below 7 µm by ball milling and knife milling but with an increase in the oxygen level, limiting its sintering ability. Sintered magnet from the recycled alloy powder demonstrated similar magnetic properties (Br: 1.06 T, HcJ: 1240 kA/m, and (BH)max: 220 kJ/m3) after blending with 5 wt% of NdH3, making vehicles and flat-screen TV speakers ideal sources for a recycled magnet.

Connected product lifetime extension tokens (CPLET): Eliciting longevity concepts of connected audio products

Designing longer-lasting connected products is a complex problem, as they include multiple components such as connectivity chipsets, batteries etc. Discussing longevity concepts with industry experts can therefore be challenging due to a lack of framework and ambiguous terminology. In this paper, we are proposing a tangible token kit called Connected Product Lifetime Extension Tokens (CPLET) that allows us to elicit data in interviews. CPLET includes longevity concepts derived from Design for sustainability and Sustainable ICT literature. We utilized CPLET as conversation enablers during semi-structured interviews with 16 participants from nine different companies in the audio industry. This paper introduces CPLET, and an overview of the longevity concepts currently applied to improve the longevity of connected sound products. Utilising such a complementary method contributed to richness of the data, simulated further reflections. guided and broadened the discussions and mitigated terminology confusion.

USE OF AUDIOMARKETING TO PROMOTE THE SERVICES OF THE TOURISM INDUSTRY

The aim of the study is to substantiate the importance and peculiarities of using audio marketing tools to promote tourism services in the tourism industry. The article proves that the use of neuroimaging methods in marketing is gaining popularity, as it provides marketers with information that cannot be obtained using conventional marketing methods. Musical content is increasingly integrated into the marketing communications system, covers the points of interaction with tourists, and contributes to the personalization of audiocommunication. It has been determined that audio marketing is an important element of modern marketing strategies, and its tools not only influence the behavior and psycho-emotional state of consumers of tourism services, but also significantly enhance the effectiveness of other marketing tools aimed at obtaining the expected result. Well-chosen music helps to improve interaction and form a positive perception of tourism products and services among tourists, creates sound comfort, ensures a good mood, enhances positive impressions, builds commitment, increases loyalty, extends the time of stay, and increases both planned and spontaneous purchases. The article substantiates that the choice of background music is of great importance in the formation of audio design, which provides tourists with positive emotions, creates an appropriate atmosphere and a pleasant overall impression. It is proved that in order to achieve an effective result of music's influence on consumers of tourist services, producers of tourist services need to constantly conduct marketing research on the needs and behavior of tourists; monitor and analyze the impact of music on tourist behavior; and conduct training and professional development of marketers.

Scratch and Mark: Affective Intermediality Towards Mathilde

Claire Denis’ cinema relies on in-betweenness of the film medium, combining it with other art forms and media through sensations. Towards Mathilde (2005) documents the correspondence between the director’s own work and that of the famous choreographer, Mathilde Monnier, reflecting on artistic expression, movement, and embodiment with the senses. This article explores the intricate texture of affective intermediality within the film. Interweaving the theories of affect and intermediality within film studies, this analysis delves into the ways in which Denis embodies a wide range of intermedial elements to inspire emotional responses. Through a detailed examination of the use of cinematography, sound design, and music, it uncovers how Denis engages in different arts and media to create a deeply affective cinematic experience. By examining the interplay between visual, auditory, and sense in dance and cinema as two distinct but intertwined art forms, this article aims to illuminate the affective intermediality at work in the film, offering insights into the complex ways in which cinema and dance can move and communicate emotion.

Race, romance, and Hollywood: Black women filmmakers and the cultural production of Black love

ABSTRACT Hollywood has been instrumental in the construction of Blackness on screen, often perpetuating myths, stereotypes, and negative images about the Black experience, showcasing Black people as aggressive, unloving, and incapable of maintaining a healthy romantic relationship. With more diversity efforts in Hollywood, Black women have become more empowered to reshape those narratives. So, what happens when these women intentionally provide a diverse representation of Blackness? What depictions of Blackness and love are prevalent when Black women drive the story? Using Black Feminist Thought as a theoretical framework, this essay examines the narratives surrounding love in the films The Photograph and Really Love (2020), both written and directed by Black women. Using thematic analysis of the film’s narratives, cinematography, casting, sound design, and character development, this study found the films Really Love (2020) and The Photograph (2020) offer a new conception of Black Love in Hollywood. These deliberate considerations demonstrate the intention Black women film writers take in depicting a full expression of Blackness, including romantic and intimate relationships.

Synthesis of Non-Linguistic Utterances for Sound Design Support Using a Genetic Algorithm

As social robots become more prevalent, they often employ non-speech sounds, in addition to other modes of communication, to communicate emotion and intention in an increasingly complex visual and audio environment. These non-speech sounds are usually tailor-made, and research into the generation of non-speech sounds that can convey emotions has been limited. To enable social robots to use a large amount of non-speech sounds in a natural and dynamic way, while expressing a wide range of emotions effectively, this work proposes an automatic method of sound generation using a genetic algorithm, coupled with a random forest model trained on representative non-speech sounds to validate each produced sound’s ability to express emotion. The sounds were tested in an experiment wherein subjects rated the perceived valence and arousal. Statistically significant clusters of sounds in the valence arousal space corresponded to different emotions, showing that the proposed method generates sounds that can readily be used in social robots.

The Role of Sound in the Immersive Experience

To create an immersive experience, it is not important to have a plethora of stimuli for different sensors, but to have their coherence, or in other words, multi-sensory integration. This coherence becomes the necessary basis for immersing oneself in a story, a game, a movie, or any other experience. Sound plays a key role in creating such an experience, both by accompanying the action and emphasizing the narrative flow, and as an independent actor that takes a central role and directs attention. The aim of the paper and poster is to explore the means by which sound can influence immersion in computer-generated virtual environments and the psychological mechanisms involved in the musical induction of emotions. The main question to be answered is how these techniques and models can be used to create a sense of immersion. The first part of the article examines immersiveness itself. The second part is devoted to the practical aspects of sound design for immersive technologies, describing the tools and techniques used to create realistic and compelling soundscapes. The third part touches briefly on the psychological mechanisms behind the induction of musical emotions. Taking a holistic view of these components and using them to reveal each other is how sound can create a sense of presence and place.

Review of Underwater Anechoic Coating Technology Under Hydrostatic Pressure

AbstractThe underwater anechoic coating technology, which considers pressure resistance and low-frequency broadband sound absorption, has become a research hotspot in underwater acoustics and has received wide attention to address the increasingly advanced low-frequency sonar detection technology and adapt to the working environment of underwater vehicles in deep submergence. One the one hand, controlling low-frequency sound waves in water is more challenging than in air. On the other hand, in addition to initiating structural deformation, hydrostatic pressure also changes material parameters, both of which have a major effect on the sound absorption performance of the anechoic coating. Therefore, resolving the pressure resistance and acoustic performance of underwater acoustic coatings is difficult. Particularly, a bottleneck problem that must be addressed in this field is the acoustic structure design with low-frequency broadband sound absorption under high hydrostatic pressure. Based on the influence of hydrostatic pressure on underwater anechoic coatings, the research status of underwater acoustic structures under hydrostatic pressure from the aspects of sound absorption mechanisms, analysis methods, and structural designs is reviewed in this paper. Finally, the challenges and research trends encountered by underwater anechoic coating technology under hydrostatic pressure are summarized, providing a reference for the design and research of low-frequency broadband anechoic coating.

EEG measurements-based study for evaluating acoustic human perception: A pilot study

Sound quality analysis and sound design are well-known human-centered strategies to evaluate the subjective perception of noise and to design machines and environments with pleasant and comfortable acoustic signatures. The subjective acoustic perception is conventionally measured by means of sound quality metrics determined through a correlation process with jury test results. The exploitation of electroencephalogram (EEG) measurements during the jury test for the registration of the brain activity in response to the acoustic stimuli presented to the jurors can allow us to estimate the jurors’ perception directly from their physiological response. This study presents results from the application of an EEG wearable device to investigate changes in the EEG frequency domain at different acoustic stimuli. Forty-three participants were recruited, and the EEG signals were recorded using the wearable sensor. The analysis of power spectral densities (PSDs) was performed to investigate features correlated to acoustic sensation induced by audio stimuli. Statistically significant differences were found between the three audio stimuli. The results bring to the conclusion that wearable sensors could be used for EEG acquisition applied to acoustic perception evaluation.