Conventional Spectral Features Research Articles

MEMD-HHT based Emotion Detection from EEG using 3D CNN.

In this study, the Multivariate Empirical Mode Decomposition (MEMD) is applied to multichannel EEG to obtain scale-aligned intrinsic mode functions (IMFs) as input features for emotion detection. The IMFs capture local signal variation related to emotion changes. Among the extracted IMFs, the high oscillatory ones are found to be significant for the intended task. The Marginal Hilbert spectrum (MHS) is computed from the selected IMFs. A 3D convolutional neural network (CNN) is adopted to perform emotion detection with spatial-temporal-spectral feature representations that are constructed by stacking the multi-channel MHS over consecutive signal segments. The proposed approach is evaluated on the publicly available DEAP database. On binary classification of valence and arousal level (high versus low), the attained accuracies are 89.25% and 86.23% respectively, which significantly outperform previously reported systems with 2D CNN and/or conventional temporal and spectral features.

Stacked spectral feature space patch: An advanced spectral representation for precise crop classification based on convolutional neural network

Spectral and spatial features in remotely sensed data play an irreplaceable role in classifying crop types for precision agriculture. Despite the thriving establishment of the handcrafted features, designing or selecting such features valid for specific crop types requires prior knowledge and thus remains an open challenge. Convolutional neural networks (CNNs) can effectively overcome this issue with their advanced ability to generate high-level features automatically but are still inadequate in mining spectral features compared to mining spatial features. This study proposed an enhanced spectral feature called Stacked Spectral Feature Space Patch (SSFSP) for CNN-based crop classification. SSFSP is a stack of two-dimensional (2D) gridded spectral feature images that record various crop types’ spatial and intensity distribution characteristics in a 2D feature space consisting of two spectral bands. SSFSP can be input into 2D-CNNs to support the simultaneous mining of spectral and spatial features, as the spectral features are successfully converted to 2D images that can be processed by CNN. We tested the performance of SSFSP by using it as the input to seven CNN models and one multilayer perceptron model for crop type classification compared to using conventional spectral features as input. Using high spatial resolution hyperspectral datasets at three sites, the comparative study demonstrated that SSFSP outperforms conventional spectral features regarding classification accuracy, robustness, and training efficiency. The theoretical analysis summarizes three reasons for its excellent performance. First, SSFSP mines the spectral interrelationship with feature generality, which reduces the required number of training samples. Second, the intra-class variance can be largely reduced by grid partitioning. Third, SSFSP is a highly sparse feature, which reduces the dependence on the CNN model structure and enables early and fast convergence in model training. In conclusion, SSFSP has great potential for practical crop classification in precision agriculture.

Line-scan Raman scattering image and multivariate analysis for rapid and noninvasive detection of restructured beef.

The mean spectral (MS) features were extracted from Raman scattering images (RSI) of beef samples over the region of interest covering the spectral range of 789-1710cm-1 and the spatial offset range of 0-5 mm (for two sides of the incident laser). The RSI monitored the main change in the protein, amide bands, lipids, and amino acid residues. The classification model performance based on MS features compared the conventional Raman spectral features and confirmed the usefulness of RSI. Finally, the results showed that RSI technology is a reliable tool for rapid and noninvasive detection of restructured beef.

All-dielectric photonic crystal with unconventional higher-order topology

Photonic crystals (PhCs) have been demonstrated as a versatile platform for the study of topological phenomena. The recent discovery of higher-order topological insulators introduces new aspects of topological PhCs that are yet to be explored. Here, we propose an all-dielectric PhC with an unconventional higher-order band topology. Besides the conventional spectral features of gapped edge states and in-gap corner states, topological band theory predicts that the corner boundary of the higher-order topological insulator hosts a 2/3 fractional charge. We demonstrate that in the PhC such a fractional charge can be verified from the local density-of-states of photons, through the concept of local spectral charge as an analog of the local electric charge due to the band filling anomaly in electronic systems. Furthermore, we show that by introducing a disclination in the proposed PhC, localized states and a 2/3 fractional spectral charge emerge around the disclination core. The emergence of the fractional spectral charges and topological boundary modes here, however, is distinct from the known cases; particularly by the 2/3 fractional spectral charges and the unique topological indices. The predicted effects can be readily observed in the state-of-the-art experiments and may lead to potential applications in integrated and quantum photonics.

Improved speech emotion recognition with Mel frequency magnitude coefficient

Automatic speech emotion recognition using machine learning is a demanding research topic in the field of affective computing. Identifying the speech features for speech emotion recognition is a challenging issue as the feature needs to emphasize the information about emotion from the speech. Spectral features play a vital role in emotion recognition from speech signals. In this paper, two modifications are made in the extraction of Mel frequency cepstral coefficient, they are, using magnitude spectrum instead of energy spectrum and exclusion of discrete cosine transform and extract Mel Frequency Magnitude Coefficient. Mel frequency magnitude coefficient is the log of magnitude spectrum on a non-linear Mel scale frequency. Mel frequency magnitude coefficient and three conventional spectral features, Mel frequency cepstral coefficient, log frequency power coefficient and linear prediction cepstral coefficient are tested on Berlin, Ravdess, Savee, EMOVO, eNTERFACE and Urdu databases with multiclass support vector machine as the classifier. Mel frequency magnitude coefficient as a stand alone feature recognizes emotion with an accuracy of 81.50% for Berlin, 64.31% for Ravdess, 75.63% for Savee, 73.30% for EMOVO, 56.41% for eNTERFACE and 95.25% for Urdu databases. Mel frequency magnitude coefficient is found to be the better spectral feature for the identification of emotion from speech compared to the conventional features.

Predicting task performance from biomarkers of mental fatigue in global brain activity

Objective. Detection and early prediction of mental fatigue (i.e. shifts in vigilance), could be used to adapt neuromodulation strategies to effectively treat patients suffering from brain injury and other indications with prominent chronic mental fatigue. Approach. In this study, we analyzed electrocorticography (ECoG) signals chronically recorded from two healthy non-human primates (NHP) as they performed a sustained attention task over extended periods of time. We employed a set of spectrotemporal and connectivity biomarkers of the ECoG signals to identify periods of mental fatigue and a gradient boosting classifier to predict performance, up to several seconds prior to the behavioral response. Main results. Wavelet entropy and the instantaneous amplitude and frequency were among the best single features across sessions in both NHPs. The classification performance using higher order spectral-temporal (HOST) features was significantly higher than that of conventional spectral power features in both NHPs. Across the 99 sessions analyzed, average F1 scores of 77.5% ± 8.2% and 91.2% ± 3.6%, and accuracy of 79.5% ± 8.9% and 87.6% ± 3.9% for the classifier were obtained for each animal, respectively. Significance. Our results here demonstrate the feasibility of predicting performance and detecting periods of mental fatigue by analyzing ECoG signals, and that this general approach, in principle, could be used for closed-loop control of neuromodulation strategies.

Combined Non-parametric and Parametric Classification Method Depending on Normality of PDF of Training Samples

Classification method with combined nonparametric and parametric classifications which depends on the normality of Probability Density Function of training samples is proposed. The proposed classification method is also based on spatial information for high spatial resolution of satellite based optical sensor images is proposed. Also, a classification method which takes into account not only spectral but also spatial features for LANDSAT-4 and 5 Thematic Mapper (TM) data is proposed. Treatment of the spatial-spectral variability existing within a region is more important for such high spatial resolution of satellite imagery data. Standard deviations in small cells, such as 2x2, 3x3 and 4x4 pixels, were used as measures to represent the spatial-spectral variabilities. This information can be used together with conventional spectral features in a unified way, for the traditional classifier such as the pixelwise Maximum Likelihood Decision Rule (MLHDR). The classification performance of new clear cuts and alpine meadows which are very close in spectral space characteristics and difficult to distinguish them by conventional methods are focused. Through experiments, it is found that there is a substantial improvement in overall classification accuracy for TM forestry data. The Probability of Correct Classification (PCC) for the new clear cuts and the alpine meadows classes rose by 7% to 97% correct. The confusion between alpine meadows and new clear cuts was reduced from 9% to 3%.

Exploring EEG microstates for affective computing: decoding valence and arousal experiences during video watching.

Investigating the electroencephalography (EEG) correlates of human emotional experiences has attracted increasing interest in the field of affective computing. Substantial progress has been made during the past decades, mainly by using EEG features extracted from localized brain activities. The present study explored a brain network-based feature defined by EEG microstates for a possible representation of emotional experiences. A publicly available and widely used benchmarking EEG dataset called DEAP was used, in which 32 participants watched 40 one-minute music videos with their 32channel EEG recorded. Four quasi-stable prototypical microstates were obtained, and their temporal parameters were extracted as features. In random forest regression, the microstate features showed better performances for decoding valence (model fitting mean squared error (MSE) = 3.85±0.28 and 4.07 ± 0.30, respectively, p = 0.022) and comparable performances for decoding arousal (MSE = 3.30±0.30 and 3.41 ±0.31, respectively, p = 0.169), as compared to conventional spectral power features. As microstate features describe neural activities from a global spatiotemporal dynamical perspective, our findings demonstrate a possible new mechanism for understanding human emotion and provide a promising type of EEG feature for affective computing.

Long-Term Spectral Statistics for Voice Presentation Attack Detection

Automatic speaker verification systems can be spoofed through recorded, synthetic, or voice converted speech of target speakers. To make these systems practically viable, the detection of such attacks, referred to as presentation attacks, is of paramount interest. In that direction, this paper investigates two aspects: 1) a novel approach to detect presentation attacks where, unlike conventional approaches, no speech signal modeling related assumptions are made, rather the attacks are detected by computing first-order and second-order spectral statistics and feeding them to a classifier, and 2) generalization of the presentation attack detection systems across databases. Our investigations on ASVspoof 2015 challenge database and AVspoof database show that, when compared to the approaches based on conventional short-term spectral features, the proposed approach with a linear discriminative classifier yields a better system, irrespective of whether the spoofed signal is replayed to the microphone or is directly injected into the system software process. Cross-database investigations show that neither the short-term spectral processing-based approaches nor the proposed approach yield systems which are able to generalize across databases or methods of attack. Thus, revealing the difficulty of the problem and the need for further resources and research.

Evaluation of wavelet spectral features in pathological detection and discrimination of yellow rust and powdery mildew in winter wheat with hyperspectral reflectance data

Hyperspectral absorption features are important indicators of characterizing plant biophysical variables for the automatic diagnosis of crop diseases. Continuous wavelet analysis has proven to be an advanced hyperspectral analysis technique for extracting absorption features; however, specific wavelet features (WFs) and their relationship with pathological characteristics induced by different infestations have rarely been summarized. The aim of this research is to determine the most sensitive WFs for identifying specific pathological lesions from yellow rust and powdery mildew in winter wheat, based on 314 hyperspectral samples measured in field experiments in China in 2002, 2003, 2005, and 2012. The resultant WFs could be used as proxies to capture the major spectral absorption features caused by infestation of yellow rust or powdery mildew. Multivariate regression analysis based on these WFs outperformed conventional spectral features in disease detection; meanwhile, a Fisher discrimination model exhibited considerable potential for generating separable clusters for each infestation. Optimal classification returned an overall accuracy of 91.9% with a Kappa of 0.89. This paper also emphasizes the WFs and their relationship with pathological characteristics in order to provide a foundation for the further application of this approach in monitoring winter wheat diseases at the regional scale.

Articulatory and excitation source features for speech recognition in read, extempore and conversation modes

In our previous works, we have explored articulatory and excitation source features to improve the performance of phone recognition systems (PRSs) using read speech corpora. In this work, we have extended the use of articulatory and excitation source features for developing PRSs of extempore and conversation modes of speech, in addition to the read speech. It is well known that the overall performance of speech recognition system heavily depends on accuracy of phone recognition. Therefore, the objective of this paper is to enhance the accuracy of phone recognition systems using articulatory and excitation source features in addition to conventional spectral features. The articulatory features (AFs) are derived from the spectral features using feedforward neural networks (FFNNs). We have considered five AF groups, namely: manner, place, roundness, frontness and height. Five different AF-based tandem PRSs are developed using the combination of Mel frequency cepstral coefficients (MFCCs) and AFs derived from FFNNs. Hybrid PRSs are developed by combining the evidences from AF-based tandem PRSs using weighted combination approach. The excitation source information is derived by processing the linear prediction residual of the speech signal. The vocal tract information is captured using MFCCs. The combination of vocal tract and excitation source features is used for developing PRSs. The PRSs are developed using hidden Markov models. Bengali speech database is used for developing PRSs of read, extempore and conversation modes of speech. The results are analyzed and the performance is compared across different modes of speech. From the results, it is observed that the use of either articulatory or excitation source features along-with to MFCCs will improve the performance of PRSs in all three modes of speech. The improvement in the performance using AFs is much higher compared to the improvement obtained using excitation source features.

Evaluation of semivariogram features for object-based image classification

Inclusion of textures in image classification has been shown beneficial. This paper studies an efficient use of semivariogram features for object-based high-resolution image classification. First, an input image is divided into segments, for each of which a semivariogram is then calculated. Second, candidate features are extracted as a number of key locations of the semivariogram functions. Then we use an improved Relief algorithm and the principal component analysis to select independent and significant features. Then the selected prominent semivariogram features and the conventional spectral features are combined to constitute a feature vector for a support vector machine classifier. The effect of such selected semivariogram features is compared with those of the gray-level co-occurrence matrix (GLCM) features and window-based semivariogram texture features (STFs). Tests with aerial and satellite images show that such selected semivariogram features are of a more beneficial supplement to spectral features. The described method in this paper yields a higher classification accuracy than the combination of spectral and GLCM features or STFs.

Feature-location analyses for identification of urban tree species from very high resolution remote sensing data

The interference from background noises and the weak spectral separability between species have negative impacts on the identification of urban tree species from remote sensing images. The density of neighbouring members (members mean both pixels and patches) similar to the centre pixel in some image features may offer an opportunity to improve the separability. This paper focuses on the density-involved feature-location analyses (refer to as F-L analyses) developed from the framework of integrated analysis of feature and space. We expressed the density of feature-carried members in two elaborated models: using the density dimension and adding the density descriptors to a feature space to conduct the F-L analyses during a procedure of classifying urban tree species. Experimental results indicate that the two models cannot only increase the number of available independent components for constructing an input vector therefore making the feature space richer, but also provide the reference of spatial dependence among the feature-carried members thus finally making the identification less difficult. The method with the density-involved F-L analyses obviously outperforms that with only conventional spectral features analyses in the classification. The average overall accuracy (OA) derived from the former is 23% higher than that from the latter.

Parcel-Based Change Detection in Land-Use Maps by Adopting the Holistic Feature

Updating the existing land-use maps with remote sensing imagery has become a common method to produce the latest land-use database. An important step is to extract the changed information. In this paper, we propose a novel method to extract the changed parcels in land-use maps by using the holistic feature called “Spatial Envelope,” which encodes each parcel without segmenting it into homogeneous objects or small regions. The holistic feature is based on the energy spectrum of the windowed Fourier transform (WFT) of each land-use parcel, which is ideal for scene categorization. Unlike the pixel-based change detection using the difference image (DI) leading to speckled results or object-based method which requires a complicated process to segment the land-use parcel into homogeneous land-cover objects, our parcel-based change detection treats each land-use parcel as an entirety by calculating the holistic feature for the former and latter parcels. Then, the distance between the corresponding former and latter parcels is compared against a threshold to select the changed parcels. Experiments have demonstrated that our procedure can extract the changed parcels with the overall accuracy of more than 92%. The performance of our procedure is reliable not only on the high-resolution (HR) images of the same sensor, but also on the images acquired by different sensors with the same or approximate spatial resolution. Comparative experiments have also proved that the holistic feature is better than conventional spectral and textural features in parcel-based change detection.

Comparison between wavelet spectral features and conventional spectral features in detecting yellow rust for winter wheat

Detection of yellow rust is of great importance in disease control and reducing the use of fungicide. Spectral analysis is an important method for disease detection in terms of remote sensing. In this study, an emerging spectral analysis method known as continuous wavelet analysis (CWA) was examined and compared with several conventional spectral features for the detection of yellow rust disease at a leaf level. The leaf spectral measurements were made by a spectroradiometer at both Zodaks 37 and 70 stages with a large sample size. The results showed that the wavelet features were able to capture the major spectral signatures of yellow rust, and exhibited considerable potential for disease detection at both growth stages. Both the accuracies of the univariate and multivariate models suggested that wavelet features outperformed conventional spectral features in quantifying disease severity at leaf level. Optimal accuracies returned a coefficient of determination (R2) of 0.81 and a root mean square error (RMSE) of 0.110 for pooled data at both stages. Furthermore, wavelet features showed a stronger response to the yellow rust at Zodaks 70 stage than at Zodaks 37 stage, indicating reliable estimation of disease severity can be made until the Zodaks 70 stage.

Automated detection of perinatal hypoxia using time–frequency-based heart rate variability features

Perinatal hypoxia is a cause of cerebral injury in foetuses and neonates. Detection of foetal hypoxia during labour based on the pattern recognition of heart rate signals suffers from high observer variability and low specificity. We describe a new automated hypoxia detection method using time-frequency analysis of heart rate variability (HRV) signals. This approach uses features extracted from the instantaneous frequency and instantaneous amplitude of HRV signal components as well as features based on matrix decomposition of the signals' time-frequency distributions using singular value decomposition and non-negative matrix factorization. The classification between hypoxia and non-hypoxia data is performed using a support vector machine classifier. The proposed method is tested on a dataset obtained from a newborn piglet model with a controlled hypoxic insult. The chosen HRV features show strong performance compared to conventional spectral features and other existing methods of hypoxia detection with a sensitivity 93.3%, specificity 98.3% and accuracy 95.8%. The high predictive value of this approach to detecting hypoxia is a substantial step towards developing a more accurate and reliable hypoxia detection method for use in human foetal monitoring.

Pitch synchronous and glottal closure based speech analysis for language recognition

This paper explores pitch synchronous and glottal closure (GC) based spectral features for analyzing the language specific information present in speech. For determining pitch cycles (for pitch synchronous analysis) and GC regions, instants of significant excitation (ISE) are used. The ISE correspond to the instants of glottal closure (epochs) in the case of voiced speech, and some random excitations like onset of burst in the case of nonvoiced speech. For analyzing the language specific information in the proposed features, Indian language speech database (IITKGP-MLILSC) is used. Gaussian mixture models are used to capture the language specific information from the proposed features. Proposed pitch synchronous and glottal closure spectral features are evaluated using language recognition studies. The evaluation results indicate that language recognition performance is better with pitch synchronous and GC based spectral features compared to conventional spectral features derived through block processing. GC based spectral features are found to be more robust against degradations due to background noise. Performance of proposed features is also analyzed on standard Oregon Graduate Institute Multi-Language Telephone-based Speech (OGI-MLTS) database.

Spectroscopic Leaf Level Detection of Powdery Mildew for Winter Wheat Using Continuous Wavelet Analysis

Powdery mildew (Blumeria graminis) is one of the most destructive crop diseases infecting winter wheat plants, and has devastated millions of hectares of farmlands in China. The objective of this study is to detect the disease damage of powdery mildew on leaf level by means of the hyperspectral measurements, particularly using the continuous wavelet analysis. In May 2010, the reflectance spectra and the biochemical properties were measured for 114 leaf samples with various disease severity degrees. A hyperspectral imaging system was also employed for obtaining detailed hyperspectral information of the normal and the pustule areas within one diseased leaf. Based on these spectra data, a continuous wavelet analysis (CWA) was carried out in conjunction with a correlation analysis, which generated a so-called correlation scalogram that summarizes the correlations between disease severity and the wavelet power at different wavelengths and decomposition scales. By using a thresholding approach, seven wavelet features were isolated for developing models in determining disease severity. In addition, 22 conventional spectral features (SFs) were also tested and compared with wavelet features for their efficiency in estimating disease severity. The multivariate linear regression (MLR) analysis and the partial least square regression (PLSR) analysis were adopted as training methods in model development. The spectral characteristics of the powdery mildew on leaf level were found to be closely related with the spectral characteristics of the pustule area and the content of chlorophyll. The wavelet features performed better than the conventional SFs in capturing this spectral change. Moreover, the regression model composed by seven wavelet features outperformed (R2=0.77, relative root mean square error RRMSE=0.28) the model composed by 14 optimal conventional SFs (R2=0.69, RRMSE=0.32) in estimating the disease severity. The PLSR method yielded a higher accuracy than the MLR method. A combination of CWA and PLSR was found to be promising in providing relatively accurate estimates of disease severity of powdery mildew on leaf level.

Discriminative feature extraction for speech recognition using continuous output codes

Feature transformation techniques have been widely investigated to reduce feature redundancy and to introduce additional discriminative information with the aim to improve the performance of automatic speech recognition (ASR). In this paper, we propose a novel method to obtain discriminative feature transformation based on output coding technique for speech recognition. The output coding transformation projects the speech features from their original space to a new one where each dimension of the features captures information to distinguish different phones. Using polynomial expansion, the short-time spectral features are first expanded to a high-dimensional space where the generalized linear discriminant sequence kernel is applied on the sequences of input feature vectors. Then, the output coding transformation formulated via a set of linear SVMs projects the sequences of high dimensional vectors into a tractable low-dimensional feature space where the resultant features are well-separated continuous output codes for the subsequent multi-class classification problem. Our experimental results on the TIMIT corpus show that the proposed features achieve 10.5% ASR error rate reduction over the conventional spectral features.

Transcribing Mandarin Broadcast Speech Using Multi-Layer Perceptron Acoustic Features

Recently, several multi-layer perceptron (MLP)-based front-ends have been developed and used for Mandarin speech recognition, often showing significant complementary properties to conventional spectral features. Although widely used in multiple Mandarin systems, no systematic comparison of all the different approaches as well as their scalability has been proposed. The novelty of this correspondence is mainly experimental. In this work, all the MLP front-ends recently developed at multiple sites are described and compared in a systematic manner on a 100 hours setup. The study covers the two main directions along which the MLP features have evolved: the use of different input representations to the MLP and the use of more complex MLP architectures beyond the three-layer perceptron. The results are analyzed in terms of confusion matrices and the paper discusses a number of novel findings that the comparison reveals. Furthermore, the two best front-ends used in the GALE 2008 evaluation, referred as MLP1 and MLP2, are studied in a more complex LVCSR system in order to investigate their scalability in terms of the amount of training data (from 100 hours to 1600 hours) and the parametric system complexity (maximum likelihood versus discriminative training, speaker adaptative training, lattice level combination). Results on 5 hours of evaluation data from the GALE project reveal that the MLP features consistently produce improvements in the range of 15%-23% relative at the different steps of a multipass system when compared to mel-frequency cepstral coefficient (MFCC) and PLP features, suggesting that the improvements scale with the amount of data and with the complexity of the system. The integration of those features into the GALE 2008 evaluation system provide very competitive performances compared to other Mandarin systems.