Piano Melodies Research Articles

Melody Generation using Deep Learning: Unleashing the Power of RNN and LSTM

This project aims to develop a novel approach for piano melody generation using Recurrent Neural Networks (RNN) and Long Short-Term Memory (LSTM) models in deep learning.The suggested models will be trained on a dataset of MIDI files with piano melodies to use sequential learning capabilities and capture the complex patterns and relationships present in musical compositions. [1] The project aims to gen- erate a variety of melodies that are both musically coherent and diverse by experimenting with various network designs, hyperparameters, and training procedures. The developed tunes will be evaluated primarily on their originality, conformity to stylistic elements, and general quality. The results of this study could lead to new developments in AI-driven music composition as well as opportunities for computational creativity in the music industry.

Evidence of a metacognitive illusion in judgments about the effects of music on cognitive performance

Two experiments serve to examine how people make metacognitive judgments about the effects of task-irrelevant sounds on cognitive performance. According to the direct-access account, people have direct access to the processes causing auditory distraction. According to the processing-fluency account, people rely on the feeling of processing fluency to make heuristic metacognitive judgments about the distracting effects of sounds. To manipulate the processing fluency of simple piano melodies and segments of Mozart’s sonata K. 448, the audio files of the music were either left in their original forward direction or reversed. The results favor the processing-fluency account over the direct-access account: Even though, objectively, forward and backward music had the same distracting effect on serial recall, stimulus-specific prospective metacognitive judgments showed that participants incorrectly predicted only backward music but not forward music to be distracting. The difference between forward and backward music was reduced but not eliminated in global retrospective metacognitive judgments that participants provided after having experienced the distracting effect of the music first-hand. The results thus provide evidence of a metacognitive illusion in people’s judgments about the effects of music on cognitive performance.

Influence of overtones and undertones on melody recognition with a cochlear implant with SSD

Many cochlear implant (CI) users have difficulties recognising pitches and melodies because pitch transmission is blurred and shifted. This study investigates whether postlingually deafened adult CI users recognize melodies better when overtones are removed or undertones are added.Fifteen unilaterally postlingually deafened CI users (single sided deafness = SSD) were included aged 22 to 73 years (MW 52, SD 11.6) with CI hearing experience between 3 and 75 months (MW 33, SD 21.0) with varying MED-EL devices. Three short piano melodies were presented to them firstly to the normal-hearing ear and then in modified overtone or undertone variants and the original variant to the CI ear. These variants should be identified as one of the three original melodies. In addition, musical experience and ability were assessed by the Munich Music Questionnaire and the MiniPROMS music tests.The CI users showed the best melody recognition in the fundamental frequency variant. The overtone variant with the third overtone was as good as the original variant with all overtones with regard to melody recognition (p=1). However, the undertone variant with the first undertone was recognised significantly worse than the fundamental version (p=0.032). Furthermore, there was no correlation between musical experience or musical ability and the number of melodies recognised (p>0.1).Since a reduction of overtones did not worsen the melody recognition, overtone reduction should be considered in future music processing programs for the CI. This could reduce the energy consumption of the CI.

The Perceptual and Emotional Consequences of Articulation in Music

Two experiments investigated perceptual and emotional consequences of note articulation in music by examining the degree to which participants perceived notes to be separated from each other in a musical phrase. Seven-note piano melodies were synthesized with staccato notes (short decay) or legato notes (gradual/sustained decay). Experiment 1 (n = 64) addressed the impact of articulation on perceived melodic cohesion and perceived emotion expressed through melodies. Participants rated melodic cohesion and perceived emotions conveyed by 32 legato and 32 staccato melodies. Legato melodies were rated more cohesive than staccato melodies and perceived as emotionally calmer and sadder than staccato melodies. Staccato melodies were perceived as having greater tension and energy. Experiment 2 (n = 60) addressed whether articulation is associated with humor and fear in music, and whether the impact of articulation depends on major vs. minor mode. For both modes, legato melodies were scarier than staccato melodies, whereas staccato melodies were more amusing and surprising. The effect of articulation on perceived happiness and sadness was dependent on mode: staccato enhanced perceived happiness for minor melodies; legato enhanced perceived sadness for minor melodies. Findings are discussed in relation to theories of music processing, with implications for music composition, performance, and pedagogy.

The Music of Silence: Part II: Music Listening Induces Imagery Responses.

During music listening, humans routinely acquire the regularities of the acoustic sequences and use them to anticipate and interpret the ongoing melody. Specifically, in line with this predictive framework, it is thought that brain responses during such listening reflect a comparison between the bottom-up sensory responses and top-down prediction signals generated by an internal model that embodies the music exposure and expectations of the listener. To attain a clear view of these predictive responses, previous work has eliminated the sensory inputs by inserting artificial silences (or sound omissions) that leave behind only the corresponding predictions of the thwarted expectations. Here, we demonstrate a new alternate approach in which we decode the predictive electroencephalography (EEG) responses to the silent intervals that are naturally interspersed within the music. We did this as participants (experiment 1, 20 participants, 10 female; experiment 2, 21 participants, 6 female) listened or imagined Bach piano melodies. Prediction signals were quantified and assessed via a computational model of the melodic structure of the music and were shown to exhibit the same response characteristics when measured during listening or imagining. These include an inverted polarity for both silence and imagined responses relative to listening, as well as response magnitude modulations that precisely reflect the expectations of notes and silences in both listening and imagery conditions. These findings therefore provide a unifying view that links results from many previous paradigms, including omission reactions and the expectation modulation of sensory responses, all in the context of naturalistic music listening.SIGNIFICANCE STATEMENT Music perception depends on our ability to learn and detect melodic structures. It has been suggested that our brain does so by actively predicting upcoming music notes, a process inducing instantaneous neural responses as the music confronts these expectations. Here, we studied this prediction process using EEGs recorded while participants listen to and imagine Bach melodies. Specifically, we examined neural signals during the ubiquitous musical pauses (or silent intervals) in a music stream and analyzed them in contrast to the imagery responses. We find that imagined predictive responses are routinely co-opted during ongoing music listening. These conclusions are revealed by a new paradigm using listening and imagery of naturalistic melodies.

The effects of dual-task interference in predicting turn-ends in speech and music

Determining when a partner’s spoken or musical turn will end requires well-honed predictive abilities. Evidence suggests that our motor systems are activated during perception of both speech and music, and it has been argued that motor simulation is used to predict turn-ends across domains. Here we used a dual-task interference paradigm to investigate whether motor simulation of our partner’s action underlies our ability to make accurate turn-end predictions in speech and in music. Furthermore, we explored how specific this simulation is to the action being predicted. We conducted two experiments, one investigating speech turn-ends, and one investigating music turn-ends. In each, 34 proficient pianists predicted turn-endings while (1) passively listening, (2) producing an effector-specific motor activity (mouth/hand movement), or (3) producing a task- and effector-specific motor activity (mouthing words/fingering a piano melody). In the speech experiment, any movement during speech perception disrupted predictions of spoken turn-ends, whether the movement was task-specific or not. In the music experiment, only task-specific movement (i.e., fingering a piano melody) disrupted predictions of musical turn-ends. These findings support the use of motor simulation to make turn-end predictions in both speech and music but suggest that the specificity of this simulation may differ between domains.

Robotic Musicianship Based on Least Squares and Sequence Generative Adversarial Networks

Robotic musicianship research aims to the configuration of robots can analyze, reason, and generate music autonomous. The goal of this research is to achieve the inspiring and meaningful musical interactions between humans and artificially creative robots. This research presents a new model of automatic music generation which is based on least squares and generative adversarial networks (GANs). This research specifies classical piano as the music source and uses the sensors as the context-awareness technology to sense and receive the input audio in human-robot interaction. Therefore, this research applies sequence generation adversarial network (SeqGAN) techniques that are better able to address discrete issues in generating samples of classical piano melodies and datasets. Modifying the SeqGAN approach, this research presented the Least Squares SeqGAN (LS-SeqGAN) method to create melody units on different chords and generates a set of music pieces as testing dataset. In this research, we implement the original method and use the least squares method to stabilize the training of GANs. The performance evaluation shows that proposed LS-SeqGAN method can fulfill the need both of music quality and creativity. It offers a robust infrastructure for the human-robotic interaction that can be used to promote the related robotic applications.

The singing voice is special: Persistence of superior memory for vocal melodies despite vocal-motor distractions

Vocal melodies sung without lyrics (la la) are remembered better than instrumental melodies. What causes the advantage? One possibility is that vocal music elicits subvocal imitation, which could promote enhanced motor representations of a melody. If this motor interpretation is correct, distracting the motor system during encoding should reduce the memory advantage for vocal over piano melodies. In Experiment 1, participants carried out movements of the mouth (i.e., chew gum) or hand (i.e., squeeze a beanbag) while listening to 24 unfamiliar folk melodies (half vocal, half piano). In a subsequent memory test, they rated the same melodies and 24 timbre-matched foils from ‘1–Definitely New’ to ‘7–Definitely Old’. There was a memory advantage for vocal over piano melodies with no effect of group and no interaction. In Experiment 2, participants carried out motor activities during encoding more closely related to singing, either silently articulating (la la) or vocalizing without articulating (humming continuously). Once again, there was a significant advantage for vocal melodies with no effect or interaction of group. In Experiment 3, participants audibly whispered (la la) repeatedly during encoding. Again, the voice advantage was present and did not differ appreciably from prior research with no motor task during encoding. However, we observed that the spontaneous phase-locking of whisper rate and musical beat tended to predict enhanced memory for vocal melodies. Altogether the results challenge the notion that subvocal rehearsal of the melody drives enhanced memory for vocal melodies. Instead, the voice may enhance engagement.

Piano Triptych “Shadows” by F. Schmitt as a Reflection of the French National Tradition

Abstract The author of the article examines the cultural processes in France, which resulted in the interpenetration of related spheres of creative activity. The role of salons has been noted in fruitful artistic ties to appear, particularly, the friendship of F. Schmitt and M. Ravel. Some parallels between their piano opuses “Shadows” and “Gaspard of the Night” have been found; one of the similarities is the use of poetic text in the form of an epigraph preceding the musical text. The author has made an analysis of the triptych “Shadows” by F. Schmitt in regards to the influence of the poetic images of Lautreamont and W. Whitman on the musical expression means chosen by the composer, and how deeply he materializes the poetic idea. The preceding dedication pieces are designated as a guideline in unraveling the composer’s creative intentions. The properties of the piano texture, melody, harmony, form, dynamics, metro rhythm, and technical equipment of the pieces of the cycle have been summarized. The study also reveals the individual features of the author's style of F. Schmitt, at the head of which lies a multifaceted polyphonized texture, where the vertical structures make one feel the movement of the multilayer whole, the horizontal structures are perceived as a part of harmonious development, and the continuity of their dialogue creates the process linearity and constructive integrity.

HIIT the Road Jack: An Exploratory Study on the Effects of an Acute Bout of Cardiovascular High-Intensity Interval Training on Piano Learning.

Pairing high-intensity interval training (HIIT) with motor skill acquisition may improve learning of some implicit motor sequences (albeit with some variability), but it is unclear if HIIT enhances explicit learning of motor sequences. We asked whether a single bout of HIIT after non-musicians learned to play a piano melody promoted better retention of the melody than low-intensity interval training (LIIT). Further, we investigated whether HIIT facilitated transfer of learning to a new melody. We generated individualized exercise protocols by having participants (n = 25) with little musical training undergo a graded maximal exercise test (GXT) to determine their cardiorespiratory fitness (VO2peak) and maximum power output (Wmax). In a subsequent session, participants practiced a piano melody (skill acquisition) and were randomly assigned to a single bout of HIIT or LIIT. Retention of the piano melody was tested 1 hour, 1 day, and 1 week after skill acquisition. We also evaluated transfer to learning a new melody 1 week after acquisition. Pitch and rhythm accuracy were analyzed with linear mixed-effects modeling. HIIT did not enhance sequence-specific retention of pitch or rhythmic elements of the piano melody, but there was modest evidence that HIIT facilitated transfer to learning a new melody. We tentatively conclude that HIIT enhances explicit, task-general motor consolidation.

Is consonance attractive to budgerigars? No evidence from a place preference study

Consonant tone combinations occur naturally in the overtone series of harmonic sounds. These include sounds that many non-human animals produce to communicate. As such, non-human animals may be attracted to consonant intervals, interpreting them, e.g., as a feature of important social stimuli. There is preliminary evidence of attraction to consonance in various bird species in the wild, but few experimental studies with birds. We tested budgerigars (Melopsittacus undulatus) for attraction to consonant over dissonant intervals in two experiments. In Experiment 1, we tested humans and budgerigars using a place preference paradigm in which individuals could explore an environment with multiple sound sources. Both species were tested with consonant and dissonant versions of a previously studied piano melody, and we recorded time spent with each stimulus as a measure of attraction. Human females spent more time with consonant than dissonant stimuli in this experiment, but human males spent equal time with both consonant and dissonant stimuli. Neither male nor female budgerigars spent more time with either stimulus type. In Experiment 2, we tested budgerigars with more ecologically relevant stimuli comprised of sampled budgerigar vocalizations arranged into consonant or dissonant chords. These stimuli, however, also failed to produce any evidence of preference in budgerigar responses. We discuss these results in the context of ongoing research on the study of consonance as a potential general feature of auditory perception in animals with harmonic vocalizations, with respect to similarities and differences between human and budgerigar vocal behaviour, and future methodological directions.

Moving in time: Simulating how neural circuits enable rhythmic enactment of planned sequences

Many complex actions are mentally pre-composed as plans that specify orderings of simpler actions. To be executed accurately, planned orderings must become active in working memory, and then enacted one-by-one until the sequence is complete. Examples include writing, typing, and speaking. In cases where the planned complex action is musical in nature (e.g. a choreographed dance or a piano melody), it appears to be possible to deploy two learned sequences at the same time, one composed from actions and a second composed from the time intervals between actions. Despite this added complexity, humans readily learn and perform rhythm-based action sequences. Notably, people can learn action sequences and rhythmic sequences separately, and then combine them with little trouble (Ullén & Bengtsson 2003). Related functional MRI data suggest that there are distinct neural regions responsible for the two different sequence types (Bengtsson et al. 2004). Although research on musical rhythm is extensive, few computational models exist to extend and inform our understanding of its neural bases. To that end, this article introduces the TAMSIN (Timing And Motor System Integration Network) model, a systems-level neural network model capable of performing arbitrary item sequences in accord with any rhythmic pattern that can be represented as a sequence of integer multiples of a base interval. In TAMSIN, two Competitive Queuing (CQ) modules operate in parallel. One represents and controls item order (the ORD module) and the second represents and controls the sequence of inter-onset-intervals (IOIs) that define a rhythmic pattern (RHY module). Further circuitry helps these modules coordinate their signal processing to enable performative output consistent with a desired beat and tempo.

Metacognition in Auditory Distraction: How Expectations about Distractibility Influence the Irrelevant Sound Effect.

Task-irrelevant, to-be-ignored sound disrupts serial short-term memory for visually presented items compared to a quiet control condition. We tested whether disruption by changing state irrelevant sound is modulated by expectations about the degree to which distractors would disrupt serial recall performance. The participants’ expectations were manipulated by providing the (bogus) information that the irrelevant sound would be either easy or difficult to ignore. In Experiment 1, piano melodies were used as auditory distractors. Participants who expected the degree of disruption to be low made more errors in serial recall than participants who expected the degree of disruption to be high, independent of whether distractors were present or not. Although expectation had no effect on the magnitude of disruption, participants in the easy-to-ignore group reported after the experiment that they were less disrupted by the irrelevant sound than participants in the difficult-to-ignore group. In Experiment 2, spoken texts were used as auditory distractors. Expectations about the degree of disruption did not affect serial recall performance. Moreover, the subjective and objective distraction by irrelevant speech was similar in the easy-to-ignore group and in the difficult-to-ignore group. Thus, while metacognitive beliefs about whether the auditory distractors would be easy or difficult to ignore can have an effect on task engagement and subjective distractibility ratings, they do not seem to have an effect on the actual degree to which the auditory distractors disrupt serial recall performance.

Mu rhythm suppression demonstrates action representation in pianists during passive listening of piano melodies.

Musicians undergo extensive training which enhances established neural links between auditory and motor areas of the brain. Long-term training develops, strengthens and enables flexibility in these connections allowing proficiency in performance. Previous research has indicated that passive listening of trained music results in the recruitment of premotor areas. It has been argued that this sound-action representation may rely on activity in mirror neuron systems and that these systems are heavily dependent on actual sensorimotor experience. Action observation studies using electroencephalography have associated changes in mu rhythm activity with the mirror neuron system in the visuomotor domain. We aimed to investigate similar effects in the audiomotor domain. We utilised a mu suppression method in our action-listening study to detect involuntary motor coactivation when pianists passively listened to piano melodies. Wavelet analysis revealed sensorimotor mu rhythm suppression while pianists listened passively to piano melodies. Thus, we show that this spectral analysis method can also be used to demonstrate that auditory stimuli can activate the human mirror neuron system when sounds are linked to actions. Mu suppression could be a useful index for further research on action representation and training-induced plasticity.

Music-to-Color Associations of Single-Line Piano Melodies in Non-synesthetes.

Prior research has shown that non-synesthetes' color associations to classical orchestral music are strongly mediated by emotion. The present study examines similar cross-modal music-to-color associations for much better controlled musical stimuli: 64 single-line piano melodies that were generated from four basic melodies by Mozart, whose global musical parameters were manipulated in tempo(slow/fast), note-density (sparse/dense), mode (major/minor) and pitch-height (low/high). Participants first chose the three colors (from 37) that they judged to be most consistent with (and, later, the three that were most inconsistent with) the music they were hearing. They later rated each melody and each color for the strength of its association along four emotional dimensions: happy/sad, agitated/calm, angry/not-angry and strong/weak. The cross-modal choices showed that faster music in the major mode was associated with lighter, more saturated, yellower (warmer) colors than slower music in the minor mode. These results replicate and extend those of Palmer et al. (2013, Proc. Natl Acad. Sci. 110, 8836-8841) with more precisely controlled musical stimuli. Further results replicated strong evidence for emotional mediation of these cross-modal associations, in that the emotional ratings of the melodies were very highly correlated with the emotional associations of the colors chosen as going best/worst with the melodies (r = 0.92, 0.85, 0.82 and 0.70 for happy/sad, strong/weak,angry/not-angry and agitated/calm, respectively). The results are discussed in terms of common emotional associations forming a cross-modal bridge between highly disparate sensory inputs.

Pupils dilate for vocal or familiar music.

Previous research reveals that vocal melodies are remembered better than instrumental renditions. Here we explored the possibility that the voice, as a highly salient stimulus, elicits greater arousal than nonvocal stimuli, resulting in greater pupil dilation for vocal than for instrumental melodies. We also explored the possibility that pupil dilation indexes memory for melodies. We tracked pupil dilation during a single exposure to 24 unfamiliar folk melodies (half sung to la la, half piano) and during a subsequent recognition test in which the previously heard melodies were intermixed with 24 novel melodies (half sung, half piano) from the same corpus. Pupil dilation was greater for vocal melodies than for piano melodies in the exposure phase and in the test phase. It was also greater for previously heard melodies than for novel melodies. Our findings provide the first evidence that pupillometry can be used to measure recognition of stimuli that unfold over several seconds. They also provide the first evidence of enhanced arousal to vocal melodies during encoding and retrieval, thereby supporting the more general notion of the voice as a privileged signal. (PsycINFO Database Record

Pianists exhibit enhanced memory for vocal melodies but not piano melodies.

Nonmusicians remember vocal melodies (i.e., sung to la la) better than instrumental melodies. If greater exposure to the voice contributes to those effects, then long-term experience with instrumental timbres should elicit instrument-specific advantages. Here we evaluate this hypothesis by comparing pianists with other musicians and nonmusicians. We also evaluate the possibility that absolute pitch (AP), which involves exceptional memory for isolated pitches, influences melodic memory. Participants heard 24 melodies played in four timbres (voice, piano, banjo, marimba) and were subsequently required to distinguish the melodies heard previously from 24 novel melodies presented in the same timbres. Musicians performed better than nonmusicians, but both groups showed a comparable memory advantage for vocal melodies. Moreover, pianists performed no better on melodies played on piano than on other instruments, and AP musicians performed no differently than non-AP musicians. The findings confirm the robust nature of the voice advantage and rule out explanations based on familiarity, practice, and motor representations.

Enhanced processing of vocal melodies in childhood.

Music cognition is typically studied with instrumental stimuli. Adults remember melodies better, however, when they are presented in a biologically significant timbre (i.e., the human voice) than in various instrumental timbres (Weiss, Trehub, & Schellenberg, 2012). We examined the impact of vocal timbre on children's processing of melodies. In Study 1, 9- to 11-year-olds listened to 16 unfamiliar folk melodies (4 each of voice, piano, banjo, or marimba). They subsequently listened to the same melodies and 16 timbre-matched foils, and judged whether each melody was old or new. Vocal melodies were recognized better than instrumental melodies, which did not differ from one another, and the vocal advantage was consistent across age. In Study 2, 5- to 6-year-olds and 7- to 8-year-olds were tested with a simplified design that included only vocal and piano melodies. Both age groups successfully differentiated old from new melodies, but memory was more accurate for the older group. The older children recognized vocal melodies better than piano melodies, whereas the younger children tended to label vocal melodies as old whether they were old or new. The results provide the first evidence of differential processing of vocal and instrumental melodies in childhood.

Syncopation creates the sensation of groove in synthesized music examples.

In order to better understand the musical properties which elicit an increased sensation of wanting to move when listening to music—groove—we investigate the effect of adding syncopation to simple piano melodies, under the hypothesis that syncopation is correlated to groove. Across two experiments we examine listeners' experience of groove to synthesized musical stimuli covering a range of syncopation levels and densities of musical events, according to formal rules implemented by a computer algorithm that shifts musical events from strong to weak metrical positions. Results indicate that moderate levels of syncopation lead to significantly higher groove ratings than melodies without any syncopation or with maximum possible syncopation. A comparison between the various transformations and the way they were rated shows that there is no simple relation between syncopation magnitude and groove.

Continuous loudness response to acoustic intensity dynamics in melodies: Effects of melodic contour, tempo, and tonality

The aim of this work was to investigate perceived loudness change in response to melodies that increase (up-ramp) or decrease (down-ramp) in acoustic intensity, and the interaction with other musical factors such as melodic contour, tempo, and tonality (tonal/atonal). A within-subjects design manipulated direction of linear intensity change (up-ramp, down-ramp), melodic contour (ascending, descending), tempo, and tonality, using single ramp trials and paired ramp trials, where single up-ramps and down-ramps were assembled to create continuous up-ramp/down-ramp or down-ramp/up-ramp pairs. Twenty-nine (Exp 1) and thirty-six (Exp 2) participants rated loudness continuously in response to trials with monophonic 13-note piano melodies lasting either 6.4s or 12s. Linear correlation coefficients >.89 between loudness and time show that time-series loudness responses to dynamic up-ramp and down-ramp melodies are essentially linear across all melodies. Therefore, ‘indirect’ loudness change derived from the difference in loudness at the beginning and end points of the continuous response was calculated. Down-ramps were perceived to change significantly more in loudness than up-ramps in both tonalities and at a relatively slow tempo. Loudness change was also greater for down-ramps presented with a congruent descending melodic contour, relative to an incongruent pairing (down-ramp and ascending melodic contour). No differential effect of intensity ramp/melodic contour congruency was observed for up-ramps. In paired ramp trials assessing the possible impact of ramp context, loudness change in response to up-ramps was significantly greater when preceded by down-ramps, than when not preceded by another ramp. Ramp context did not affect down-ramp perception. The contribution to the fields of music perception and psychoacoustics are discussed in the context of real-time perception of music, principles of music composition, and performance of musical dynamics.