Pitch Prediction Research Articles

Different plasticity patterns of schematic and dynamic expectations in musical pitch prediction

Different plasticity patterns of schematic and dynamic expectations in musical pitch prediction

Ship pitch prediction method based on LSTMC and multi-head attention

When ships sail in the sea, ship motion is unstable and stochasticity due to the ever-changing weather and marine conditions. Aiming to address the issue of low prediction accuracy in traditional prediction methods, short term predictions of ship pitch at 1 step, 3 steps, 6 steps, and 12 steps in advance are conducted, and the impact of input-output ratios on the prediction model is studied. The seven types of data of the marine environment and ship motion are taken as model inputs, with pitch as the model output. When predicting 1 step in advance, a prediction model based on LSTM and multi-head attention (LMA) is proposed. This model firstly uses LSTM layer to extract the temporal feature of the input sequence, and then uses the multi-head attention layer to assign larger weights to the key information to enhance prediction accuracy. When predicting 3 steps, 6 steps, and 12 steps in advance, as the input-output ratio increases, the length of the input sequence keeps growing. The model needs to sufficiently capture the local spatial characteristic and long-range dependency between multi-variable inputs, a deep prediction model based on Conv2d and LMA model is proposed. The model firstly uses two-dimensional convolution layer to extract local spatial feature of long sequences, and then through the attention layer, assign larger weights to temporal information and spatial information that are conducive to continuous prediction in order to enhance prediction accuracy. Research on the input-output ratio indicates that, as the number of predicted steps increases, the appropriate ratio continuously decreases, but the length of the input sequence keeps increasing. Finally, four evaluation metrics are used to evaluate the proposed algorithms on real ship data, and the prediction stability and accuracy of the LMA model and the LCMA model are also verified.

Multi-task melody extraction using feature optimization and CRNN-CRF

At present, mainstream melody extraction mostly uses deep learning methods, but there are still problems: such as incomplete network architecture, lack of research on the importance of input features for melody extraction, etc. Based on the previous issues, to predict the melody more accurately, we firstly use phase correction after short-time Fourier transform (STFT) feature extraction method according to audio features. Secondly, we use a weight-based and multi-task network feature optimization mechanism to assign weights to the input features of the main and auxiliary networks, to fully capture the important parts in the time sequence of the melodic. Thirdly, to make use of the context information of features and label sequences, we further propose a multi-task pitch extraction network based on Convolutional Recurrent Neural Network-Conditional Random Field (CRNN-CRF) to decode the optimal label sequences. Besides, we use the joint loss function of the previous two tasks to train and optimize the model to improve the accuracy of pitch prediction. After testing and evaluating the various datasets before visual comparison, the experimental results have demonstrated the superiority of our proposed method.

Differentiable Short-Term Models for Efficient Online Learning and Prediction in Monophonic Music

As pieces of music are usually highly self-similar, online-learning short-term models are well-suited for musical sequence prediction tasks. Due to their simplicity and interpretability, Markov chains (MCs) are often used for such online learning, with Prediction by Partial Matching (PPM) being a more sophisticated variant of simple MCs. PPM, also used in the well-known IDyOM model, constitutes a variable-order MC that relies on exact matches between observed <em>n</em>-grams and weights more recent events higher than those further in the past. We argue that these assumptions are limiting and propose the Differentiable Short-Term Model (DSTM) that is not limited to exact matches of <em>n</em>-grams and can also learn the relative importance of events. During (offline-)training, the DSTM learns representations of <em>n</em>-grams that are useful for constructing fast weights (that resemble an MC transition matrix) in online learning of intra-opus pitch prediction. We propose two variants: the Discrete Code Short-Term Model and the Continuous Code Short-Term Model. We compare the models to different baselines on the <em>“The Session”</em> dataset and find, among other things, that the Continuous Code Short-Term Model has a better performance than Prediction by Partial Matching, as it adapts faster to changes in the data distribution. We perform an extensive evaluation of the models, and we discuss some analogies of DSTMs with linear transformers. The source code for model training and the experiments is available at <a href="https://github.com/muthissar/diffstm" target="_blank">https://github.com/muthissar/diffstm</a>.

Deterministic and probabilistic ship pitch prediction using a multi-predictor integration model based on hybrid data preprocessing, reinforcement learning and improved QRNN

The deterministic and probabilistic prediction of ship motion is important for safe navigation and stable real-time operational control of ships at sea. However, the volatility and randomness of ship motion, the non-adaptive nature of single predictors and the poor coverage of quantile regression pose serious challenges to uncertainty prediction, making research in this field limited. In this paper, a multi-predictor integration model based on hybrid data preprocessing, reinforcement learning and improved quantile regression neural network (QRNN) is proposed to explore the deterministic and probabilistic prediction of ship pitch motion. To validate the performance of the proposed multi-predictor integrated prediction model, an experimental study is conducted with three sets of actual ship longitudinal motions during sea trials in the South China Sea. The experimental results indicate that the root mean square errors (RMSEs) of the proposed model of deterministic prediction are 0.0254°, 0.0359°, and 0.0188°, respectively. Taking series #2 as an example, the prediction interval coverage probabilities (PICPs) of the proposed model of probability predictions at 90%, 95%, and 99% confidence levels (CLs) are 0.9400, 0.9800, and 1.0000, respectively. This study signifies that the proposed model can provide trusted deterministic predictions and can effectively quantify the uncertainty of ship pitch motion, which has the potential to provide practical support for ship early warning systems.

Ship Pitch Prediction Based on Bi-ConvLSTM-CA Model

When a ship is sailing at sea, its pitch angle will be affected by ship motions such as turning angle, relative wind speed, relative wind direction, velocity in surge and velocity in sway of the ship. Due to the randomness of ship motion attitude and the difficulty of capturing the motion rules, traditional machine learning models, statistical learning models and single deep learning models cannot accurately capture the correlation information between multiple variables, which results in poor prediction accuracy. To solve this problem, the bidirectional convolutional long short-term memory neural network (Bi-ConvLSTM) and channel attention (CA) for ship pitch prediction are used to build a Bi-ConvLSTM-CA model in this paper. The Bi-ConvLSTM-CA prediction model can simultaneously extract the time information and spatial information of the ship motion data, and use the channel attention mechanism to process the output of different time steps to obtain the corresponding weight of each channel. Using the weights to do dot product with the output of Bi-ConvLSTM, the resulting attention mechanism output is processed to produce predicted value by the fully connected layer. Compared with other models, the RMSE index of Bi-ConvLSTM-CA model decreased by at least 28.20%; the MAPE index decreased by at least 29.39%; the MAE index decreased by at least 22.68%. The experimental results of real ship data show that the proposed Bi-ConvLSTM-CA model has a significant reduction in mean absolute percentage error (MAPE), mean square error (MSE) and mean absolute error (MAE) compared with other advanced models, which verifies the effectiveness of the Bi-ConvLSTM-CA model in predicting ship pitch angle.

Brain perturbation to reveal causal contributions of executive networks to the mechanisms of auditory attention

Listeners face the challenge of understanding speech every day. A key cognitive strategy for parsing speech in noisy environments relies on auditory selective attention, which recruits a network of multiple cortical regions. Our recent studies identified at least two different neural mechanisms of auditory attention. Space-based attention engages alpha oscillations in the parietal cortex, which are thought to cause lateral suppression of streams in the ignored portion in space. In contrast, talker identity-based attention modulates beta oscillations in the temporal lobe, suggesting that a neural pitch prediction mechanism in the middle temporal gyrus (MTG) focuses attention to talker. Based on these findings, here we use a brain perturbation method to establish causal links between such oscillatory activity and control of different forms of auditory attention. We hypothesize that perturbing alpha oscillations in parietal cortex will impair spatial attention while perturbing beta oscillations in the MTG will impair speaker identity-based attention. Our ongoing study utilizes repetitive transcranial magnetic stimulation (rTMS) to induce such perturbations in the pre-determined regions of interests non-invasively and safely. This presentation will provide proof-of-concept data to validate the feasibility of this approach.

Speech Information Hiding Method Based on Random Position Selection and Matrix Coding

For the low-rate speech encoding problem, an information hidden algorithm based on the G.723.1 coding standard is proposed. In the pitch prediction coding process, by controlling the search range of the closed-loop pitch period (adaptive codebook), combined with the Random Position Selection (RPS) method and the Matrix Coding Method (MCM), the secret information is embedded, which is implemented in the speech coding process. The adoption of the RPS method reduces the correlation between the carrier code-words, and the adoption of the MCM method reduces the rate of change of the carrier. The experimental results show that the average PESQ (Perceptual Evaluation of Speech Quality) deterioration rate under the algorithm is 1.63%, and the concealment is good.

A novel approach for flutter prediction of pitch–plunge airfoils using an efficient one-shot method

In this work, the one-shot method previously developed for the solution of limit cycle oscillation problems is extended to predict flutter boundaries of aeroelastic systems. In essence, the one-shot method determines the aeroelastic response of wings and airfoils in a tightly-coupled fashion where both aerodynamic and structural dynamic problems are solved simultaneously using harmonic balance. This approach is superior to the frequency-based techniques previously reported in the literature such that it eliminates the need to sweep over a range of frequencies to determine flutter conditions. For each Mach number of interest, the values of flutter frequency and flutter velocity are determined as part of a single aeroelastic run. A method for identifying appropriate initial conditions is also presented. It is shown that the flutter onset point for given flow conditions can be accurately identified by prescribing a very small pitch amplitude treating flutter prediction as a response problem instead of the classical stability problem. Using this technique, three two-degree-of-freedom aeroelastic models, including a flat plate, the NACA 64A010 airfoil and the supercritical NLR 7301 airfoil, are studied under different flow conditions ranging from low-speed, inviscid flow to transonic, viscous, turbulent flow. The results are verified against reference results from the literature. In addition, two other established flutter methods are implemented in this work for verification purposes, and the efficiency and robustness of the one-shot method are investigated.

Beta oscillatory power modulation reflects the predictability of pitch change

Humans process highly dynamic auditory information in real time, and regularities in stimuli such as speech and music can aid such processing by allowing sensory predictions for upcoming events. Auditory sequences contain information about both the identity of sounds (what) and their timing (when they occur). Temporal prediction in isochronous sequences is reflected in neural oscillatory power modulation in the beta band (∼20 Hz). Specifically, power decreases (desynchronization) after tone onset and then increases (resynchronization) to reach a maximum around the expected time of the next tone. The current study investigates whether the predictability of the pitch of a tone (what) is also reflected in beta power modulation. We presented two isochronous auditory oddball sequences, each with 20% of tones at a deviant pitch. In one sequence the deviant tones occurred regularly every fifth tone (predictably), but in the other sequence they occurred pseudorandomly (unpredictably). We recorded the electroencephalogram (EEG) while participants listened passively to these sequences. The results showed that auditory beta power desynchronization was larger prior to a predictable than an unpredictable pitch change. A single-trial correlation analysis using linear mixed-effect (LME) models further showed that the deeper the pre-deviant beta desynchronization depth, the smaller the event-related P3a amplitude following the deviant, and this effect only occurred when the pitch change was predictable. Given that P3a is associated with attentional response to prediction error, larger beta desynchronization depth indicates better prediction of an upcoming deviant pitch. Thus, these findings suggest that beta oscillations reflect predictions for what in additional to when during dynamic auditory information processing.

In situ vocal fold properties and pitch prediction by dynamic actuation of the songbird syrinx

The biomechanics of sound production forms an integral part of the neuromechanical control loop of avian vocal motor control. However, we critically lack quantification of basic biomechanical parameters describing the vocal organ, the syrinx, such as material properties of syringeal elements, forces and torques exerted on, and motion of the syringeal skeleton during song. Here, we present a novel marker-based 3D stereoscopic imaging technique to reconstruct 3D motion of servo-controlled actuation of syringeal muscle insertions sites in vitro and focus on two muscles controlling sound pitch. We furthermore combine kinematic analysis with force measurements to quantify elastic properties of sound producing medial labia (ML). The elastic modulus of the zebra finch ML is 18 kPa at 5% strain, which is comparable to elastic moduli of mammalian vocal folds. Additionally ML lengthening due to musculus syringealis ventralis (VS) shortening is intrinsically constraint at maximally 12% strain. Using these values we predict sound pitch to range from 350–800 Hz by VS modulation, corresponding well to previous observations. The presented methodology allows for quantification of syringeal skeleton motion and forces, acoustic effects of muscle recruitment, and calibration of computational birdsong models, enabling experimental access to the entire neuromechanical control loop of vocal motor control.

Improving HMM speech synthesis of interrogative sentences by pitch track transformations

Modeling interrogative sentence prosody is a challenging task due to the significant variation of questions. Prosody is produced by intonation, intensity and duration features. Intonation clearly identifies the type of question in most European languages. If only limited training data is available from certain sentence types, synthetic intonation often lacks accuracy, richness and detail due to averaging, inherent in statistical approaches. In this paper, we discuss two rule-based solutions to improve intonation of interrogative sentences. The first approach utilizes a pitch prediction algorithm where a rule-set forms the pitch pattern. The output pattern is combined with an HMM generated F0 contour and the resulting pattern is used for speech synthesis. Our second solution uses key points to define a scaling function. The position of the key points is described by a rule-set, and the value at intermediate points is calculated in training time in a data-driven way in order to maintain the characteristics of the speakers’ voice. The proposed two hybrid rule-based-HMM systems were evaluated by speech experts and by perceptual tests. Our evaluation shows that both approaches could significantly improve the prosodic representation of questions in our framework without deteriorating the perceived naturalness of synthetic speech. The pitch contours generated by the systems were compared and evaluated according to how well they could reproduce the unique characteristics of different Hungarian interrogative sentence types. The analysis shows that both of the methods could successfully model the required patterns. In a separate perception test the interrogative prosody identification rate of the solutions was also measured. The results demonstrate that applying our proposed methods the identification rate of questions could be improved significantly. Although our work focuses on our mother tongue -Hungarian- the methodology can be extended to other languages.

Dynamic life prediction of pitch and yaw bearings for wind turbine

The design guideline of the National Renewable Energy Laboratory (NREL) is a standard for designing and verifying pitch and yaw bearings of wind turbines. The usability of the guideline was evaluated by comparing with analytical results obtained by a commercial software. Results from calculation equations of the NREL guideline and from a commercial software were compared in terms of the main design evaluation items of pitch and yaw bearings, such as dynamic life, static load factor, and case-core interface models. Calculation and analysis were conducted using specifications and loading conditions for the yaw bearing of a 2.5 MW wind turbine. Results showed that the values computed using the NREL guideline were more conservative than those derived using the commercial software. Although the guideline showed some differences from the analytical results obtained using the software, they can be used in the initial design of pitch and yaw bearings because of the advantage of the simplicity and the characteristics of conservative output.

Studying Musical and Linguistic Prediction in Comparable Ways: The Melodic Cloze Probability Method.

Prediction or expectancy is thought to play an important role in both music and language processing. However, prediction is currently studied independently in the two domains, limiting research on relations between predictive mechanisms in music and language. One limitation is a difference in how expectancy is quantified. In language, expectancy is typically measured using the cloze probability task, in which listeners are asked to complete a sentence fragment with the first word that comes to mind. In contrast, previous production-based studies of melodic expectancy have asked participants to sing continuations following only one to two notes. We have developed a melodic cloze probability task in which listeners are presented with the beginning of a novel tonal melody (5–9 notes) and are asked to sing the note they expect to come next. Half of the melodies had an underlying harmonic structure designed to constrain expectations for the next note, based on an implied authentic cadence (AC) within the melody. Each such ‘authentic cadence’ melody was matched to a ‘non-cadential’ (NC) melody matched in terms of length, rhythm and melodic contour, but differing in implied harmonic structure. Participants showed much greater consistency in the notes sung following AC vs. NC melodies on average. However, significant variation in degree of consistency was observed within both AC and NC melodies. Analysis of individual melodies suggests that pitch prediction in tonal melodies depends on the interplay of local factors just prior to the target note (e.g., local pitch interval patterns) and larger-scale structural relationships (e.g., melodic patterns and implied harmonic structure). We illustrate how the melodic cloze method can be used to test a computational model of melodic expectation. Future uses for the method include exploring the interplay of different factors shaping melodic expectation, and designing experiments that compare the cognitive mechanisms of prediction in music and language.

Realtime voice activity and pitch modulation for laryngectomy transducers using head and facial gestures

Individuals who have undergone laryngectomy often rely on handheld transducers (i.e., the electrolarynx) to excite the vocal tract and produce speech. Widely used electrolarynx designs are limited, in that they require manual control of voice activity and pitch modulation. It would be advantageous to have an interface that requires less training, perhaps using the remaining, intact speech production system as a scaffold. Strong evidence exists that aspects of head motion and facial gestures are highly correlated with gestures of voicing and pitch. Therefore, the goal of project MANATEE is to develop an electrolarynx control interface which takes advantage of those correlations. The focus of the current study is to determine the feasibility of using head and facial features to accurately and efficiently modulate the pitch of speaker's electrolarynx in real time on a mobile platform using the built-in video camera. A prototype interface, capable of running on desktop machines and compatible Android devices, is implemented using OpenCV for video feature extraction and statistical prediction of the electrolarynx control signal. Initial performance evaluation is promising, showing pitch prediction accuracies at double the chance-level baseline, and prediction delays well below the perceptually-relevant, ~50 ms threshold.

On the enhancement of coupling potential flow models to RANS solvers for the prediction of propeller effective wakes

The calculation of the effective wake within the CFD context is usually made by combining a potential-flow method for modeling the propeller forces with a RANS equation solver for simulating the viscous flow around the hull and possible appendages. The different assumptions and/or simplifications made in the potential flow model relative to the viscous flow solver may result in significant errors in the prediction of the effective wake particularly for high loadings. This is especially troublesome for ships with full forms where large differences are expected between the nominal and effective wake, and for special propulsion applications such as contra-rotating units. Such errors are responsible within the hydrodynamic design problem for an unadjusted prediction of the propeller pitch, and within the hydrodynamic analysis problem for a deficient prediction of self-propulsion point. This paper presents an approach based on correction factors which converts propeller-induced velocities approximately estimated via potential flow theory into viscous induced velocities on the basis of a viscous flow RANS analysis. The correction factors are calculated for one reference advance number and work accurately in a neighboring region where the propeller loading may change about ±50 %. This procedure allows controlling one of the errors present in the calculation of effective wakes, namely the error derived from coupling a potential-flow method for the representation of the propeller with a RANS solver. Consequently, it permits calculating the effective wake more precisely. The approach is illustrated for a simple case in which the potential flow model representing the propeller is an actuator disk.

Development of a mathematical model for simultaneous heave, pitch and roll motions of planing vessel in regular waves

The market for planing boats has shown steady growth in the recent years and this has caused an increase in the need for designing tools. The need is even more vital for the prediction of seakeeping and maneuverability of planing boats. Although the numerical methods are proved to be useful for the displacement vessels, they seem to be relatively impractical dealing with the complexities of the hydrodynamic behavior of planing boats. In this paper, a simple method for prediction of Heave, Pitch and Roll motions of the planing vessels in regular waves is presented. In the formulation of the present mathematical model, a 2-dimensional technique using momentum theory is applied. Different aspects of the added mass and 2D+t theories are also implemented for the section-based simulations of the planing vessel. The developed method is analyzed through a set of validation tests while the experimental results of the literature are used in order to validate the obtained results. The results are then evaluated in different degrees of freedom and favorable agreement has been achieved against the experimental data.

Improved prediction error filters for adaptive feedback cancellation in hearing aids

Acoustic feedback is a well-known problem in hearing aids, caused by the undesired acoustic coupling between the hearing aid loudspeaker and microphone. Acoustic feedback produces annoying howling sounds and limits the maximum achievable hearing aid amplification. This paper is focused on adaptive feedback cancellation (AFC) where the goal is to adaptively model the acoustic feedback path and estimate the feedback signal, which is then subtracted from the microphone signal. The main problem in identifying the acoustic feedback path model is the correlation between the near-end signal and the loudspeaker signal caused by the closed signal loop, in particular when the near-end signal is spectrally colored as is the case for a speech signal. This paper adopts a prediction-error method (PEM)-based approach to AFC, which is based on the use of decorrelating prediction error filters (PEFs). We propose a number of improved PEF designs that are inspired by harmonic sinusoidal modeling and pitch prediction of speech signals. The resulting PEM-based AFC algorithms are evaluated in terms of the maximum stable gain (MSG), filter misadjustment, and computational complexity. Simulation results for a hearing aid scenario indicate an improvement up to 5–7dB in MSG and up to 6–8dB in terms of filter misadjustment.

Transient noise reduction in speech signal with a modified long-term predictor

This article proposes an efficient median filter based algorithm to remove transient noise in a speech signal. The proposed algorithm adopts a modified long-term predictor (LTP) as the pre-processor of the noise reduction process to reduce speech distortion caused by the nonlinear nature of the median filter. This article shows that the LTP analysis does not modify to the characteristic of transient noise during the speech modeling process. Oppositely, if a short-term linear prediction (STP) filter is employed as a pre-processor, the enhanced output includes residual noise because the STP analysis and synthesis process keeps and restores transient noise components. To minimize residual noise and speech distortion after the transient noise reduction, a modified LTP method is proposed which estimates the characteristic of speech more accurately. By ignoring transient noise presence regions in the pitch lag detection step, the modified LTP successfully avoids being affected by transient noise. A backward pitch prediction algorithm is also adopted to reduce speech distortion in the onset regions. Experimental results verify that the proposed system efficiently eliminates transient noise while preserving desired speech signal.

User-Specific Learning for Recognizing a Singer's Intended Pitch

We consider the problem of automatic vocal melody transcription: translating an audio recording of a sung melody into a musical score. While previous work has focused on finding the closest notes to the singer's tracked pitch, we instead seek to recover the melody the singer intended to sing. Often, the melody a singer intended to sing differs from what they actually sang; our hypothesis is that this occurs in a singer-specific way. For example, a given singer may often be flat in certain parts of her range, or another may have difficulty with certain intervals. We thus pursue methods for singer-specific training which use learning to combine different methods for pitch prediction. In our experiments with human subjects, we show that via a short training procedure we can learn a singer-specific pitch predictor and significantly improve transcription of intended pitch over other methods. For an average user, our method gives a 20 to 30 percent reduction in pitch classification errors with respect to a baseline method which is comparable to commercial voice transcription tools. For some users, we achieve even more dramatic reductions. Our best results come from a combination of singer-specific-learning with non-singer-specific feature selection. We also discuss the implications of our work for training more general control signals. We make our experimental data available to allow others to replicate or extend our results.