Pitch Segments Research Articles

Automatic Team Assignment and Jersey Number Recognition in Football Videos

Football is one of the most-watched sports, but analyzing players’ performance is currently difficult and labor intensive. Performance analysis is done manually, which means that someone must watch video recordings and then log each player’s performance. This includes the number of passes and shots taken by each player, the location of the action, and whether or not the play had a successful outcome. Due to the time-consuming nature of manual analyses, interest in automatic analysis tools is high despite the many interdependent phases involved, such as pitch segmentation, player and ball detection, assigning players to their teams, identifying individual players, activity recognition, etc. This paper proposes a system for developing an automatic video analysis tool for sports. The proposed system is the first to integrate multiple phases, such as segmenting the field, detecting the players and the ball, assigning players to their teams, and identifying players’ jersey numbers. In team assignment, this research employed unsupervised learning based on convolutional autoencoders (CAEs) to learn discriminative latent representations and minimize the latent embedding distance between the players on the same team while simultaneously maximizing the distance between those on opposing teams. This paper also created a highly accurate approach for the real-time detection of the ball. Furthermore, it also addressed the lack of jersey number datasets by creating a new dataset with more than 6,500 images for numbers ranging from 0 to 99. Since achieving a high performance in deep learning requires a large training set, and the collected dataset was not enough, this research utilized transfer learning (TL) to first pretrain the jersey number detection model on another large dataset and then fine-tune it on the target dataset to increase the accuracy. To test the proposed system, this paper presents a comprehensive evaluation of its individual stages as well as of the system as a whole. All codes, datasets, and experiments are available on GitHub ().

A Robust and Low Computational Cost Pitch Estimation Method.

Pitch estimation is widely used in speech and audio signal processing. However, the current methods of modeling harmonic structure used for pitch estimation cannot always match the harmonic distribution of actual signals. Due to the structure of vocal tract, the acoustic nature of musical equipment, and the spectrum leakage issue, speech and audio signals’ harmonic frequencies often slightly deviate from the integer multiple of the pitch. This paper starts with the summation of residual harmonics (SRH) method and makes two main modifications. First, the spectral peak position constraint of strict integer multiple is modified to allow slight deviation, which benefits capturing harmonics. Second, a main pitch segment extension scheme with low computational cost feature is proposed to utilize the smooth prior of pitch more efficiently. Besides, the pitch segment extension scheme is also integrated into the SRH method’s voiced/unvoiced decision to reduce short-term errors. Accuracy comparison experiments with ten pitch estimation methods show that the proposed method has better overall accuracy and robustness. Time cost experiments show that the time cost of the proposed method reduces to around 1/8 of the state-of-the-art fast NLS method on the experimental computer.

Approach for CM/DM Current Extraction and Crosstalk Analysis of Twisted-Wire Pairs with Random Non-uniform Twisting

In this paper, a twisted-wire pairs (TWP) with random non-uniform twisting is established. It is divided into a complete pitch segment and a noncomplete pitch segment by the ratio between the pitch and the length. The randomness of the actual TWP cable is accurately simulated by the following methods: 1) random combination of complete pitch segments; 2) random combination of non-complete pitch segments; 3) random combination between 1) and 2). Based on the TWP model, an equivalent multi-conductor transmission lines (MTLs) model can be obtained. The neural network algorithm is introduced to describe the complex relationship between the arbitrary position of the TWP and the per-unit-length (p.u.l) parameter matrix. In addition, the crosstalk and the common-mode (CM) and differential-mode (DM) noise under field-to-wire coupling are predicted. The numerical results show that crosstalk and CM/DM noise in TWP cable are susceptible to the twisted pitch at high frequencies. Compared with full-wave simulation, the accuracy of the proposed method is proved.

Design and stiffness analysis of a pitch-varying folded flexure hinge (PFFH)

Based on the theory and characteristics of the coil spring, we proposed a pitch-varying folded flexure hinge (PFFH), which is formed by a series of folded flexure units of variable pitches. With the increase of deformation, different pitch segments are contacted in order to achieve the purpose of variable stiffness. After giving an example design of the hinge structure, a theoretical calculation model of its equivalent spring constant is then derived accordingly. The hinge's stiffness characteristics are discussed and analyzed through both theoretical calculation and finite element analysis (FEA). The results showed that the stiffness of the hinge is varying, and the theoretical calculations results are well-matched with the results of FEA. In addition, the influence of different pitches on the stress and the trend of its stress changes are also analyzed.

Speech Personality Recognition Based on Annotation Classification Using Log-Likelihood Distance and Extraction of Essential Audio Features

Speech personality recognition relies on training models that require an excessive number of features and are, in most cases, designed specifically for certain databases. As a result, when tested on different datasets, overfitted classifier models are not always reliable because their accuracy changes with changes in the domain of speakers. Moreover, personality annotations are often subjective, which creates variability in raters perception during labeling. These problems inhibit the effectiveness of speech personality recognition applications. To reduce the unexplained variance caused by unknown differences in raters perception, a structure that uses Balanced Iterative Reducing and Clustering using Hierarchies (BIRCH) algorithm is proposed. Furthermore, a feature extraction method is proposed to filter out undesirable adulterations be it noise, silence, or uncertain pitch segments, while extracting essential audio features, i.e., signal power roll-off, pitch, and pause rate. Experiments on the standard SSPNet dataset records a relative 4% increase in overall accuracy when log-likelihood based annotations are used. Moreover, improved consistency in accuracy is observed when this method is tested on male and female subsets.

The ability to glimpse dynamic pitch in noise by younger and older listeners.

While dynamic pitch is helpful for speech perception in temporally-modulated noise, the ability to benefit from this cue varies substantially among older listeners. To examine the perceptual factors that contribute to this variability, this study aimed to characterize individuals' ability to perceive dynamic pitch in temporally-modulated noise using dynamic pitch segments extracted from real speech and embedded in temporally modulated noise. Data from younger and older listeners showed stronger pitch contours were more easily perceived than weaker pitch contours. The metric significantly predicted speech-in-noise ability in older listeners. Potential implications of this work are discussed.

RVAD: An unsupervised segment-based robust voice activity detection method

This paper presents an unsupervised segment-based method for robust voice activity detection (rVAD). The method consists of two passes of denoising followed by a voice activity detection (VAD) stage. In the first pass, high-energy segments in a speech signal are detected by using a posteriori signal-to-noise ratio (SNR) weighted energy difference and if no pitch is detected within a segment, the segment is considered as a high-energy noise segment and set to zero. In the second pass, the speech signal is denoised by a speech enhancement method, for which several methods are explored. Next, neighbouring frames with pitch are grouped together to form pitch segments, and based on speech statistics, the pitch segments are further extended from both ends in order to include both voiced and unvoiced sounds and likely non-speech parts as well. In the end, a posteriori SNR weighted energy difference is applied to the extended pitch segments of the denoised speech signal for detecting voice activity. We evaluate the VAD performance of the proposed method using two databases, RATS and Aurora-2, which contain a large variety of noise conditions. The rVAD method is further evaluated, in terms of speaker verification performance, on the RedDots 2016 challenge database and its noise-corrupted versions. Experiment results show that rVAD is compared favourably with a number of existing methods. In addition, we present a modified version of rVAD where computationally intensive pitch extraction is replaced by computationally efficient spectral flatness calculation. The modified version significantly reduces the computational complexity at the cost of moderately inferior VAD performance, which is an advantage when processing a large amount of data and running on low resource devices. The source code of rVAD is made publicly available.

Pitch segmentation of speech signals based on short-time energy waveform

In general, speech is constituted of quasi-repetitive patterns called pitches representing the speech fundamental period and tonal information of the voice. Extraction of pitch information that is crucial for many speech processing techniques, usually faces a noise problem and interference caused by high-order harmonic components. This paper introduces a novel, noise-robust method for determining speech fundamental frequency and pitch segmentation, based on a short-time energy waveform (SEW), defined as a moving average squared signal. When applying a moving average filter with a window size closed to the fundamental period, nearly repetitive patterns, with fewer ripples, synchronizing with actual pitches can clearly be observed in the SEW. The DC component in the SEW is removed using morphological top-hat and bottom-hat transforms. The fundamental frequency is determined as the frequency corresponding to the largest peak of the power spectrum of the DC-removed SEW. Finally, a time-domain window search is then performed to locate local extrema associated with pitches. Compared to traditional pitch detection techniques, the proposed technique yields pitch segmentation results with a higher rate of accuracy and greater noise robustness.

Recognition of Mould Colony on Unhulled Paddy Based on Computer Vision using Conventional Machine-learning and Deep Learning Techniques

To investigate the potential of conventional and deep learning techniques to recognize the species and distribution of mould in unhulled paddy, samples were inoculated and cultivated with five species of mould, and sample images were captured. The mould recognition methods were built using support vector machine (SVM), back-propagation neural network (BPNN), convolutional neural network (CNN), and deep belief network (DBN) models. An accuracy rate of 100% was achieved by using the DBN model to identify the mould species in the sample images based on selected colour-histogram parameters, followed by the SVM and BPNN models. A pitch segmentation recognition method combined with different classification models was developed to recognize the mould colony areas in the image. The accuracy rates of the SVM and CNN models for pitch classification were approximately 90% and were higher than those of the BPNN and DBN models. The CNN and DBN models showed quicker calculation speeds for recognizing all of the pitches segmented from a single sample image. Finally, an efficient uniform CNN pitch classification model for all five types of sample images was built. This work compares multiple classification models and provides feasible recognition methods for mouldy unhulled paddy recognition.

Improving the Flexibility of Dynamic Prosody Modification Using Instants of Significant Excitation

Modification of suprasegmental features such as pitch and duration of original speech by fixed scaling factors is referred to as static prosody modification. In dynamic prosody modification, the prosodic scaling factors (time-varying modification factors) are defined for all the pitch cycles present in the original speech. The present work is focused on improving the naturalness of the prosody modified speech by reducing the generation of piecewise constant segments in the modified pitch contour. The prosody modification is performed by anchoring around the accurate instants of significant excitation estimated from the original speech. The division of longer pitch intervals into many equal intervals over long speech segments introduces step-like discontinuities in the form of piecewise constant segments in the modified pitch contours. The effectiveness of proposed dynamic modification method is initially confirmed from the smooth modified pitch contour plot obtained for finer static prosody scaling factors, waveforms, spectrogram plots and comparison subjective evaluations. Also, the average $$F_0$$F0 jitter computed from the pitch segments of each glottal activity region in the modified speech is proposed as an objective measure for the prosody modification. The naturalness of the prosody modified speech using the proposed method is objectively and subjectively compared with that of the existing zero frequency filtered signal-based dynamic prosody modification. Also, the proposed algorithm effectively preserves the dynamics of the prosodic patterns in singing voices where in the $$F_0$$F0 parameters rapidly and continuously fluctuate within a higher $$F_0$$F0 range.

Development of a Segmented Linear Variable Flux Reluctance Motor with DC-Field Coil

This paper proposes a new type of linear motor based on the concept of variable flux reluctance motor (VFRM). By adopting segmented primary stator, the segmented linear VFRM (SLVFRM) can eliminate asymmetry between phases due to end-effect. Meanwhile, multi-phase SLVFRM of any number of phases can be obtained by adjusting number of segments directly. The stator/mover tooth pitch combination and segments arrangement rules are also illustrated. An optimized 3-phase SLVFRM is analyzed by finite element analysis, with focus on characteristics such as cogging force, back-EMF and winding inductances. Further, both brushless AC (BLAC) and brushless DC (BLDC) drive for SLVFRM are investigated. Besides much smaller force ripple, BLDC drive can obtain higher average force at same RMS current due to trapezoidal back-EMF. The force characteristicwith variousAC and DC currents and current angles are also investigated in the paper, and it shows that SLVFRM has negligible reluctance force while keeping the same AC and DC currents is optimal for maximum efficiency operation under a fixed copper loss.

TV-필터와 최적 음표조합을 이용한 개선된 가변템포 음악채보방법

This paper proposes three methods for improving the accuracy of auto-music transcription considering with time-varying tempo from monophonic sound. The first one that uses TV(Total Variation) filter for smoothing the pitch data reduces the fragmentation in the pitch segmentation result. Also, the measure finding method that combines three different ways based on pitch and energy of sound data, respectively as well as based on rules produces more stable result. In addition the temporal result of note-length encoding is corrected in optimal way that the resulted encoding minimizes the sum of quantization error in a measure while the sum of note-lengths is equal to the number of beats. In the experiment with 16 children songs, we obtained the improved result in which measure finding was complete, the accuracy of encoding for note-length and pitch was about 91.3 and 86.7, respectively.

Potential for reducing the numbers of SiPM readout surfaces of laser-processed X’tal cube PET detectors

We are developing a three-dimensional (3D) position-sensitive detector with isotropic spatial resolution, the X’tal cube. Originally, our design consisted of a crystal block for which all six surfaces were covered with arrays of multi-pixel photon counters (MPPCs). In this paper, we examined the feasibility of reducing the number of surfaces on which a MPPC array must be connected with the aim of reducing the complexity of the system. We evaluated two kinds of laser-processed X’tal cubes of 3 mm and 2 mm pitch segments while varying the numbers of the 4 × 4 MPPC arrays down to two surfaces. The sub-surface laser engraving technique was used to fabricate 3D grids into a monolithic crystal block. The 3D flood histograms were obtained by the Anger-type calculation. Two figures of merit, peak-to-valley ratios and distance-to-width ratios, were used to evaluate crystal identification performance. Clear separation was obtained even in the 2-surface configuration for the 3 mm X’tal cube, and the average peak-to-valley ratios and the distance-to-width ratios were 6.7 and 2.6, respectively. Meanwhile, in the 2 mm X’tal cube, the 6-surface configuration could separate all crystals and even the 2-surface case could also, but the flood histograms were relatively shrunk in the 2-surface case, especially on planes parallel to the sensitive surfaces. However, the minimum peak-to-valley ratio did not fall below 3.9. We concluded that reducing the numbers of MPPC readout surfaces was feasible for both the 3 mm and the 2 mm X’tal cubes.

Tagging Webcast Text in Baseball Videos by Video Segmentation and Text Alignment

Sports video annotation, an active research area in the field of multimedia content understanding, is an essential process in applications, such as summarization, highlight extraction, event detection, and retrieval. This paper considers the issue in relation to the annotation of baseball videos. Conventional baseball video annotation frameworks are based primarily on video content analysis, such as scoreboard recognition and machine learning techniques, which require a substantial amount of human input to collect and organize training data. The performance of such frameworks might become unstable if they encounter audiovisual patterns not included in the training data. To address the issue, we propose a novel framework for baseball video annotation that aligns high-level webcast text with low-level video content. Several cues, which are derived from the video content and webcast text, are utilized for alignment by leveraging hierarchical agglomerative clustering and genetic algorithm optimization. In addition, we develop an unsupervised method to learn the pitch segment properties of baseball videos by Markov random walk, and thereby reduce the need for human intervention substantially. Our experiments demonstrate that the proposed framework yields a robust result against a variety of video content and enhances the automaticity in baseball video annotation.

Pitch encoding in speech and nonspeech contexts in the human auditory brainstem

Frequency-following responses were recorded from Chinese and English participants at the level of the brainstem in response to four Mandarin tonal contours presented in a speech and nonspeech context. Pitch strength analysis of these preattentive brainstem responses showed that the Chinese group exhibited stronger pitch representation than the English group regardless of context. Moreover, the Chinese group exhibited relatively more robust pitch representation of rapidly changing pitch segments. These findings support the view that at early preattentive stages of subcortical processing, neural mechanisms underlying pitch representation are shaped by particular features of the auditory stream rather than speech per se. These findings have implications for optimizing signal-processing strategies for cochlear implant design for speakers of tonal languages.

Method and apparatus for generating modified speech from pitch-synchronous segmented speech waveforms

A system that synchronously segments a speech waveform using pitch period and a center of the pitch waveform. The pitch waveform center is determined by finding a local minimum of a centroid histogram waveform of the low-pass filtered speech waveform for one pitch period. The speech waveform can then be represented by one or more of such pitch waveforms or segments during speech compression, reconstruction or synthesis. The pitch waveform can be modified by frequency enhancement/filtering, waveform stretching/shrinking in speech synthesis or speech disguise. The utterance rate can also be controlled to speed up or slow down the speech.

Transglottic Air Flow Related to Selected Parameters of the Voice

This study was concerned with the relationship of air flow to systematic changes of vocal pitch, vocal intensity, and voice register. Nine young adult women with no previous vocal training were selected as subjects. Their respective pitch ranges were determined for the modal and upper registers and these ranges were divided into 10% segments. Vocal intensity ranges at each of these pitch levels were established and also divided into 10% segments. Air-flow measures were obtained from respirometric recordings made as each subject sustained the vowel /a/ for 4 sec at each intensity level for each pitch segment in both registers Results of these measurements showed that a progressive increase in air-flow accompanied increases in vocal intensity in both the modal and upper registers. A progressive increase in air flow also was found as vocal pitch was increased in the modal register. No such relationship was found in rite upper register.