Music Visualization Research Articles

A Study of Using Visual Arts in Music Classroom

This study explored the integration of visual arts elements into music education and its impact on students' musical understanding, creativity, and artistic literacy. The study used a mixed-methods approach that included classroom observations, analysis of student work, and interviews to explore interdisciplinary arts education practices. Findings highlight several key benefits of this interdisciplinary approach: enhancing conceptual understanding through music visualization, increasing student engagement, fostering creativity, supporting diverse learning needs, and making meaningful interdisciplinary connections. Students' ability to visually present musical concepts, to create art across the senses, and to solve problems were all enhanced. In addition, the study explored the potential for integrated instructional design that incorporates art museum and concert hall experiences. While acknowledging the challenges of implementation, the study highlights the great potential of this approach to enrich students' learning experiences and promote a more holistic understanding of the arts. This study contributes to the growing body of knowledge supporting interdisciplinary approaches to arts education and suggests areas for future research, including long-term implications for artistic development, cognitive skills, and overall academic achievement.

Genesis of Music Video Clip as a Popular Form of Modern Audiovisual Culture: Experiments with Music Visualisation

The purpose of the article is to reveal the peculiarities of the development of the music video clip in the context of the historical interaction of musical and audiovisual culture. Research methodology. General scientific methods of cognition were applied, in particular analysis and synthesis, systematisation of theoretical data, the method of structural-functional analysis, which made it possible to reveal specific aspects of the interaction of music and various forms of artistic practices in the space of audiovisual culture; a typological method, with the help of which various features of the music video clip in the structure of modern audiovisual culture are revealed. Scientific novelty. The genesis of the music video clip as a popular form of modern audiovisual culture is studied; the problem of integration of visual images and music as part of artistic practices in historical retrospect is considered. Little-known works of foreign scientists, the problems of which are correlated with the subject area of research were introduced into scientific circulation. Conclusions. The genesis of the music video clip testifies to the significant role of artistic practices in the process of image and music interaction at the early stages, which contributed to the invention of new creative ways of representing the imaginary – animation and montage proposed by avant-garde artists; videos and video installations created by mastering the television image; music videos that opened a new dimension for experimentation, becoming an influential form of modern culture. Artistic history of analogue interactions between cinema and music during the 20th century demonstrates the antecedent conditions of production, perception, and socio-political context that were necessary for the mass popularization of the music video as an established form. Thanks to the early experiments in the formation of visualised music, the music video as a new form of audiovisual art gained cultural value, which allowed it to play a significant role at the modern stage. The music video played and continues to play a key role in the socio-cultural space. The technological evolution from analogue to digital means of production and display has accelerated the development of both the form and the aesthetics of the music video, which emphasises one of the most important roles of art in society – to provoke thought and encourage a rethinking of social, political and cultural conditions and surroundings, using this is the main means of communication, including popular music.

Further Study of Music Visualization: Chladni's Figure

By applying the principle of standing waves, the various scales produced by a piano within music software can be transformed into Chladni figures. These figures, which visually represent the vibration patterns of a surface, can be captured using photographic equipment. By carefully selecting the most accurate patterns and organizing them into a comprehensive table, it is possible to create a visual representation of Chladni figures corresponding to each musical syllable. This method provides a unique way to visualize the acoustic characteristics of different musical notes.

Practical Application and Case Analysis of Computer Image Vision Technology in Music Education and Teaching

Exploring how to utilize images to enrich music teaching content and provide a more visually impactful learning experience is an important topic. Therefore, this paper introduces a convolutional neural network-based algorithm for extracting audio features to construct a music visualization model. By identifying features such as note pitches, it enhances pitch recognition and integrates with CNN algorithms for audio information visualization. Experimental results demonstrate an accuracy rate exceeding 97%, showcasing the significant advantage of this method in visualizing audio information in music multimedia classrooms. It provides technical support for bringing a new visual experience to music education.

Analysis of the implementation methods for emotion recognition in music based on machine learning

Music is a carrier of emotions, capable of triggering, expressing, and conveying people's innermost emotional experiences. With the development of digital technology, the recognition of emotions in music has been the subject of extensive research. From the viewpoint of application, MER is of great significance in music personalized recommendation services, psychotherapy, music creation, and music visualization. Currently, MER implementation methods based on machine learning are gradually becoming mainstream. This study starts from the basic knowledge of MER, first introduces the relevant research directions and research background of MER, and then proposes a three-part research framework combining MER and MEC. Based on this research framework, the machine learning algorithms and specific applications involved in the framework are introduced. Finally, the challenges and future paths facing the implementation of MER technology are proposed. Provide a reference for improving the accuracy and quality of using artificial intelligence methods to capture music characteristics and expressiveness.

Seeing the piano sound Exploration on utilizing finite element analysis to visualize piano sound

The study aims to provide a way for impaired people to learn music better through music visualization. We established a relationship between piano key frequencies and Chladni patterns. Firstly, we performed modal analysis simulations using ANSYS, covering the full range of excitable modes in the frequency range of the piano keys. However, since the natural frequencies of the simulation results were slightly different from experimental results, we used the hammering method to validate the simulation results and to demonstrate that the finite element modal analysis was able to simulate the dynamic properties of the circular plate well. Subsequent studies have found that different excitation frequencies in the lower frequency range may also affect the generation of Chladni patterns. Using harmonic analysis in ANSYS, we confirmed that tones of similar frequency on the same plate can produce different Chladni patterns, thus greatly simplifying the previous problem of having to use multiple plates of different shapes and sizes in order to visualize most of the piano keys. Based on the graphical simulation of a single tone, we have successfully extended it to a melody. This result provides strong support for effectively helping the hearing impaired to learn to play musical instruments, makes it possible to present music in a visual form, and opens up new possibilities for the wider use of music education tools in the hearing-impaired community.

Hearing with the eyes: modulating lyrics typography for music visualization

Hearing with the eyes: modulating lyrics typography for music visualization

Research on the Integration and Innovative Expression Methods of Visual Elements of Popular Music in Multimedia Era

Abstract Visual art and music art are two typical media, and the progress of multimedia technology has prompted the cross-border integration between them. This paper tries to visualize the visual translation of music acoustic information and emotional experiences through audiovisual mapping design. The extraction and analysis of musical and visual elements in the music visual mapping model are summarized and analyzed. Combined with the visual expression method, a mapping pattern framework that can be reused in music visualization design is proposed, and two types of mapping patterns are classified on top of it. Based on the four basic attributes of pitch, chord, tone, and timbre of popular music, corresponding visual elements are designed. Finally, the impact of the two mapping modes on the functional and aesthetic effects of music visualization is experimentally demonstrated and evaluated, and it is found that the mean value of the scaling index of the original HCR sequence of the pop music clip is close to 0.864, while the mean value of the disturbed sequence is 0.504, which is a referable evaluation result for determining the appropriate mapping type when designing music visualization.

A Study on the Construction of a Smart Music Education Classroom and the Utility of Music Teaching Reforms

Abstract This study explores the integration of information technology tools, such as audio signal visualization, score recommendation algorithms based on difficulty features, and sight-singing assessment algorithms, into the development of an intelligent music classroom. It evaluates the impact of these technologies on teaching quality and efficiency, emphasizing their role in enhancing music sight-singing and classroom visualization. Findings reveal that innovative music education significantly improves teaching outcomes. While over half of the students responded positively to the creative teaching approaches, a segment found the material presented through music visualization challenging. However, introducing an intelligent music education classroom markedly benefits students’ musical learning.

Electronic music composition thinking using visual learning and visual sensing technology

AbstractThis exploration aims to provide a kind of interactive electronic music composition thinking based on visual learning and visual sensing technology to enhance the thinking process of electronic music composition. Initially, it summarizes the functionalities of the physics components in Unity and analyzes the principles of visual interaction implementation using Virtual Reality (VR) devices and Leap Motion in Unity. Subsequently, the composition of interactive electronic music works is designed and implemented based on ultrasonic sensor technology. Lastly, this exploration focuses on the complete visual design of the audio‐visual integration of Musical Instrument Digital Interface (MIDI) music to represent Cymatics' images of the emotional content of the music. MIDI data can be assigned to provide various mapping possibilities between images and content in music visualization composition. This exploration also designs experiments based on sensory aftereffects and audio‐visual synesthesia to further determine the counterpoint law between the shape of dynamic Cymatics' images and the timbre's audio‐visual synesthetic results. The results indicate that Bass and Kick are distributed in the mid‐low frequency and sub‐low frequency range (20–160 Hz), Vocal and Lead are distributed in the mid‐high frequency range (1280–2560 Hz), and Hihat is distributed in the high‐frequency range (2560–5120 Hz). This exploration utilizes computer technology to create a music visualization method that conforms to the visual expression and aesthetic style of a multi‐sensory experience.

Computer Aided Music Visualization Teaching Model Based on Emotional Feature Detection

Computer Aided Music Visualization Teaching Model Based on Emotional Feature Detection

Glancing back at your hearing: Generating emotional calligraphy typography from musical rhythm

Glancing back at your hearing: Generating emotional calligraphy typography from musical rhythm

MUSE: Visual Analysis of Musical Semantic Sequence.

Visualization has the capacity of converting auditory perceptions of music into visual perceptions, which consequently opens the door to music visualization (e.g., exploring group style transitions and analyzing performance details). Current research either focuses on low-level analysis without constructing and comparing music group characteristics, or concentrates on high-level group analysis without analyzing and exploring detailed information. To fill this gap, integrating the high-level group analysis and low-level details exploration of music, we design a musical semantic sequence visualization analytics prototype system (MUSE) that mainly combines a distribution view and a semantic detail view, assisting analysts in obtaining the group characteristics and detailed interpretation. In the MUSE, we decompose the music into note sequences for modeling and abstracting music into three progressively fine-grained pieces of information (i.e., genres, instruments and notes). The distribution view integrates a new density contour, which considers sequence distance and semantic similarity, and helps analysts quickly identify the distribution features of the music group. The semantic detail view displays the music note sequences and combines the window moving to avoid visual clutter while ensuring the presentation of complete semantic details. To prove the usefulness and effectiveness of MUSE, we perform two case studies based on real-world music MIDI data. In addition, we conduct a quantitative user study and an expert evaluation.

Predicting Perceived Music Emotions with Respect to Instrument Combinations

Music Emotion Recognition has attracted a lot of academic research work in recent years because it has a wide range of applications, including song recommendation and music visualization. As music is a way for humans to express emotion, there is a need for a machine to automatically infer the perceived emotion of pieces of music. In this paper, we compare the accuracy difference between music emotion recognition models given music pieces as a whole versus music pieces separated by instruments. To compare the models' emotion predictions, which are distributions over valence and arousal values, we provide a metric that compares two distribution curves. Using this metric, we provide empirical evidence that training Random Forest and Convolution Recurrent Neural Network with mixed instrumental music data conveys a better understanding of emotion than training the same models with music that are separated into each instrumental source.

Music-to-Facial Expressions: Emotion-Based Music Visualization for the Hearing Impaired

While music is made to convey messages and emotions, auditory music is not equally accessible to everyone. Music visualization is a common approach to augment the listening experiences of the hearing users and to provide music experiences for the hearing-impaired. In this paper, we present a music visualization system that can turn the input of a piece of music into a series of facial expressions representative of the continuously changing sentiments in the music. The resulting facial expressions, recorded as action units, can later animate a static virtual avatar to be emotive synchronously with the music.

The History and Problems of Experimental Digital Pinball and Developing a Pinball Video Game Concept Based on Music

Pinball games may have lost their popularity over time; however, developers revived their features inside other video games or completely renovated them. While arcade pinball machines offer limited gameplay, digital pinball can be much more flexible. Nevertheless, according to the author’s preliminary investigation, academic research is lacking in pinball gaming. This artistic paper attempts to close this gap by shedding light on experimental digital pinball games. Moreover, the author focuses on music visualization features of digital pinball games, and he developed an experimental pinball video game concept based on synaesthesia by answering a simple question, “How does a classical music-based pinball video game would look like?”.

AI Color Organ: Piano Music Visualization using Onset Detection and HistoGAN

The music visualization algorithm described in this study allows users to construct piano audio files using imported image files. This paper contributes to previous studies and designs of sonification by highlighting the effectiveness of utilizing onset detection in creating intuitive sonic changes. The audio-visual correspondences employed in this study could be expanded to many other syntheses and sample manipulation techniques. Translating visual information into sonic changes could yield many creative applications in music production, as it offers musicians a simultaneously optical and auditory production experience. This approach to audio manipulation also increases the unpredictability of the sound output, which could be appealing to experimental musicians seeking to control sounds with the visual structure of artworks that they enjoy, as opposed to precise parameters. It is looking forward to seeing creative implementations of the techniques in audio-visual artworks, music production tools, and interactive multimedia systems. These results shed light on guiding further exploration of AI composing.

Research on Music Visualization in New Media

With the current development of science and technology, technology provides a new carrier for music, and the form and quantity of music are constantly expanding. The way people come into contact with music is no longer just radio, but also TV, mobile phones, computers, etc. The number of music and video bundling is increasing, and the amount of music in independent art forms is decreasing. The integration between industries has made the combination of vision and hearing closer, music visualization has become more mature, and the art emerging in this form has also become the mainstream of music art. This paper studies music visualization through the case study method, content analysis method and literature analysis method to analyze the current development status of music visualization from two aspects: streaming media and games. On this basis, this paper studies the development trend and prospects of the future music and music industry, providing reference suggestions for the future development of the music industry.

A review: Music-emotion recognition and analysis based on EEG signals.

Music plays an essential role in human life and can act as an expression to evoke human emotions. The diversity of music makes the listener's experience of music appear diverse. Different music can induce various emotions, and the same theme can also generate other feelings related to the listener's current psychological state. Music emotion recognition (MER) has recently attracted widespread attention in academics and industry. With the development of brain science, MER has been widely used in different fields, e.g., recommendation systems, automatic music composing, psychotherapy, and music visualization. Especially with the rapid development of artificial intelligence, deep learning-based music emotion recognition is gradually becoming mainstream. Besides, electroencephalography (EEG) enables external devices to sense neurophysiological signals in the brain without surgery. This non-invasive brain-computer signal has been used to explore emotions. This paper surveys EEG music emotional analysis, involving the analysis process focused on the music emotion analysis method, e.g., data processing, emotion model, and feature extraction. Then, challenging problems and development trends of EEG-based music emotion recognition is proposed. Finally, the whole paper is summarized.

Practices of Visual Arts in the Music of the Twentieth and Early Twenty-First Centuries

"The study of transformational processes in contemporary music art under the influence of the fine arts practices is one of the important problems of musical culturology and aesthetics. The study is topical because of the new efforts aimed at understanding and arrangement of the evolution of established aesthetic systems and experiments in music and fine arts in the 20th century and at the beginning of the 21st century, as well as their influence on modern practices of introduction of meaningful elements of painting, graphics, architecture, and multimedia technologies into music creation processes. The aim of the research was to identify current trends in music visualization through the arrangement and generalization of the experience of artists of the 20th — early 21st centuries, who used expressive arts, synthesizing one art into another in their musical compositions, as well as to identify the background of those synthetic ideas and ways to implement them. The scientific research established that the development of synthetic art has its roots in the disposition towards merging arts and versatility. The fine arts practices applied in music has contributed to the expansion and enrichment of artistic means and techniques and has entailed further complication of sound and visual components in the overall concept of the work. It is concluded that the individual creative and personal traits of the artist, who seeks to fully convey the multidimensional figurativeness of the work determine a significant impact on implementing fine arts practices in music. Prospects for further research of the evolution of fine arts practices and audio-visual media in music involve the development of description techniques in the context of the dominance of the visualization factor over music one in artistic perception, as well as the study of the impact of cultural technologization on contemporary music art. Keywords: music, visualization, fine art, artist, interconnection of the arts, synthesis, intermediality "