Time-frequency Components Research Articles

Neural correlates of multimodal metaphor comprehension: Evidence from event-related potentials and time-frequency decompositions

The present study examined the event-related potential (ERP) and time-frequency components correlates with the comprehension process of multimodal metaphors that were represented by the combination of “a vehicle picture+a written word of an animal”. Electroencephalogram data were recorded when participants decided whether the metaphor using an animal word for the vehicle rendered by a picture was appropriate or not. There were two conditions: appropriateness (e.g., sport utility vehicles+tiger) vs. inappropriateness (e.g., sport utility vehicles+cat). The ERP results showed that inappropriate metaphor elicited larger N300 (280–360ms) and N400 (380–460ms) amplitude than appropriate one, which were different from previous exclusively verbal metaphor studies that rarely observed the N300 effect. A P600 (550–750ms) was also observed and larger in appropriate metaphor condition. Besides, the time-frequency principal component analysis revealed that two independent theta activities indexed the separable processes (retrieval of semantic features and semantic integration) underlying the N300 and N400. Delta band was also induced within a later time window and best characterized the integration process underlying P600. These results indicate the specific cognitive mechanism of multimodal metaphor comprehension that is different from verbal metaphor processing, mirrored by several separable processes indexed by ERP components and time-frequency components. The present study extends the metaphor research by uncovering the functional roles of delta and theta as well as their unique contributions to the ERP components during multimodal metaphor comprehension.

Comparison of photopic negative response measurements in the time and time–frequency domains

To compare measurements of the full-field photopic negative response (PhNR), as well as intra-subject variation in the PhNR, using time and time-frequency domain analyses. Full-field ERGs were recorded from 20 normally sighted subjects (aged 24-65years) elicited by a long-wavelength pulse (3cdsm-2) presented against a short-wavelength adapting field (12.5cdm-2). Three to 10 waveforms were obtained from each subject, and each waveform was analyzed using standard time domain analyses of the PhNR, as well as a discrete wavelet transform (DWT) to extract time-frequency components that correspond to the PhNR. Three different measures of the PhNR were derived and compared: (1) amplitude at the PhNR trough; (2) amplitude at 72ms following stimulus onset; (3) energy in the 11Hz, 60-120ms DWT frequency bin that corresponds to the PhNR. In addition, the effect of normalizing the PhNR by the b-wave was evaluated for each of the measures. Coefficients of variation (CVs) were computed for each definition to evaluate intra-subject variation. PhNR amplitudes measured at the trough and at 72ms were significantly correlated (r=0.88, p<0.001). Additionally, PhNR energy derived by DWT was significantly correlated with the amplitude measured at the trough (r=0.64, p=0.002) and at 72ms (r=0.60, p=0.005). Mean (±SD) intra-subject CVs were 26% (15%), 49% (26%), and 30% (15%), for measures at the trough, 72ms, and DWT, respectively. Normalization by the b-wave amplitude (i.e., PhNR/b) had minimal effect on the intra-subject CVs, whereas normalization by the sum of the b-wave and PhNR amplitudes (i.e., PhNR/[b+PhNR]) substantially reduced the CVs for all three measures (mean CVs were less than 17% for all conditions). Although each PhNR definition has advantages and disadvantages, all three metrics provide similar estimates of the PhNR. Intra-subject CVs, however, were relatively high for measurements made at 72ms, indicating that definitions based on a fixed time point may introduce variability. The substantial decrease in intra-subject variation after normalization by the sum of the PhNR and b-wave amplitudes may be advantageous under some conditions.

Shear wave elastography using Wigner-Ville distribution: a simulated multilayer media study.

Shear Wave Elastography (SWE) is a quantitative ultrasound-based imaging modality for distinguishing normal and abnormal tissue types by estimating the local viscoelastic properties of the tissue. These properties have been estimated in many studies by propagating ultrasound shear wave within the tissue and estimating parameters such as speed of wave. Vast majority of the proposed techniques are based on the cross-correlation of consecutive ultrasound images. In this study, we propose a new method of wave detection based on time-frequency (TF) analysis of the ultrasound signal. The proposed method is a modified version of the Wigner-Ville Distribution (WVD) technique. The TF components of the wave are detected in a propagating ultrasound wave within a simulated multilayer tissue and the local properties are estimated based on the detected waves. Image processing techniques such as Alternative Sequential Filters (ASF) and Circular Hough Transform (CHT) have been utilized to improve the estimation of TF components. This method has been applied to a simulated data from Wave3000™ software (CyberLogic Inc., New York, NY). This data simulates the propagation of an acoustic radiation force impulse within a two-layer tissue with slightly different viscoelastic properties between the layers. By analyzing the local TF components of the wave, we estimate the longitudinal and shear elasticities and viscosities of the media. This work shows that our proposed method is capable of distinguishing between different layers of a tissue.

Combining EEG and eye movement recording in free viewing: Pitfalls and possibilities

Co-registration of EEG and eye movement has promise for investigating perceptual processes in free viewing conditions, provided certain methodological challenges can be addressed. Most of these arise from the self-paced character of eye movements in free viewing conditions. Successive eye movements occur within short time intervals. Their evoked activity is likely to distort the EEG signal during fixation. Due to the non-uniform distribution of fixation durations, these distortions are systematic, survive across-trials averaging, and can become a source of confounding. We illustrate this problem with effects of sequential eye movements on the evoked potentials and time-frequency components of EEG and propose a solution based on matching of eye movement characteristics between experimental conditions. The proposal leads to a discussion of which eye movement characteristics are to be matched, depending on the EEG activity of interest. We also compare segmentation of EEG into saccade-related epochs relative to saccade and fixation onsets and discuss the problem of baseline selection and its solution. Further recommendations are given for implementing EEG-eye movement co-registration in free viewing conditions. By resolving some of the methodological problems involved, we aim to facilitate the transition from the traditional stimulus-response paradigm to the study of visual perception in more naturalistic conditions.

Forecasting effective drought index using a wavelet extreme learning machine (W-ELM) model

A drought forecasting model is a practical tool for drought-risk management. Drought models are used to forecast drought indices (DIs) that quantify drought by its onset, termination, and subsequent properties such as the severity, duration, and peak intensity in order to monitor and evaluate the impacts of future drought. In this study, a wavelet-based drought model using the extreme learning machine (W-ELM) algorithm where the input data are first screened through the wavelet pre-processing technique for better accuracy is developed to forecast the monthly effective DI (EDI). The EDI is an intensive index that considers water accumulation with a weighting function applied to rainfall data with the passage of time in order to analyze the drought-risk. Determined by the autocorrelation function (ACF) and partial ACFs, the lagged EDI signals for the current and past months are used as significant inputs for 1 month lead-time EDI forecasting. For drought model development, 97 years of data for three hydrological stations (Bathurst Agricultural, Wilsons Promontory and Merredin in Australia) are partitioned in approximately 90:5:5 ratios for training, cross-validation and test purposes, respectively. The discrete wavelet transformation (DWT) is applied to the predictor datasets to decompose inputs into their time–frequency components that capture important information on periodicities. DWT sub-series are used to develop new EDI sub-series as inputs for the W-ELM model. The forecasting capability of W-ELM is benchmarked with ELM, artificial neural network (ANN), least squares support vector regression (LSSVR) and their wavelet-equivalent (W-ANN, W-LSSVR) models. Statistical metrics based on agreement between the forecasted and observed EDI, including the coefficient of determination, Willmott’s index, Nash–Sutcliffe coefficient, percentage peak deviation, root-mean-square error, mean absolute error, and model execution time are used to assess the effectiveness of the models. The results demonstrate enhanced forecast skill of the drought models that use wavelet pre-processing of the predictor dataset. Based on statistical measures, W-ELM outperformed traditional ELM, LSSVR, ANN and their wavelet-equivalent counterparts (W-ANN, W-LSSVR). It is found that the W-ELM model is computationally efficient as shown by a faster running time with the majority of forecasting errors in lower frequency bands. The results demonstrate the usefulness of W-ELM over W-ANN and W-LSSVR models and the benefits of wavelet transformation of input data to improve the performance of drought forecasting models.

Dual-Microphone Noise Reduction in Car Environments With Determinant Analysis of Input Correlation Matrix

In this paper, a novel noise reduction technique is proposed to improve the speech interface performance in car environments. The proposed noise reduction method with dual microphones is primarily based on the determinant analysis of the input correlation matrix. Through the analysis, a robust feature for speech activity detection and signal-to-noise ratio (SNR) estimation is derived. Using the feature, the SNR of each time-frequency component is estimated, and an enhanced speech signal is obtained through Wiener filtering. To evaluate the proposed noise reduction technique, we constructed a database in a real car environment and comparatively analyzed the performances of noise reduction methods. The results show that meaningful SNR and perceptual speech quality improvements with less signal distortion are achieved compared with the other competing methods.

Super-resolution radar

In this paper, we study the identification of a time-varying linear system from its response to a known input signal. More specifically, we consider systems whose response to the input signal is given by a weighted superposition of delayed and Doppler-shifted versions of the input. This problem arises in a multitude of applications such as wireless communications and radar imaging. Due to practical constraints, the input signal has finite bandwidth $B$ , and the received signal is observed over a finite time interval of length $T$ only. This gives rise to a delay and Doppler resolution of $1/B$ and $1/T$ . We show that this resolution limit can be overcome, i.e., we can exactly recover the continuous delay–Doppler pairs and the corresponding attenuation factors, by solving a convex optimization problem. This result holds provided that the distance between the delay–Doppler pairs is at least $2.37/B$ in time or $2.37/T$ in frequency. Furthermore, this result allows the total number of delay–Doppler pairs to be linear up to a log-factor in $BT$ , the dimensionality of the response of the system, and thereby the limit for identifiability. Stated differently, we show that we can estimate the time–frequency components of a signal that is $S$ -sparse in the continuous dictionary of time–frequency shifts of a random window function, from a number of measurements that is linear up to a log-factor in $S$ .

Decoding of object categories from brain signals using cross frequency coupling methods

Abstract The procedure by which information about conceptual category of visual objects is encoded in brain signals, is the topic of many studies in the field of visual system. A phenomenon that is recently mentioned as one of information encoding strategies in the brain is cross frequency coupling (CFC), means the interactions between different frequency bands in physiological signals such as EEG. This interaction can occur between various components of such signals including amplitude or power, phase and frequency. Different types of CFC have been reported in various cognitive tasks and within or between different brain regions. However the role of CFC in encoding of information about the category of visual objects is greatly unknown and here we attempted to investigate it. So in this paper, we used machine learning algorithms to find out whether CFC contains such information. To this end, we recorded EEG from 10 participants while they were observing stimuli from 12 different visual object categories. Then amplitude–amplitude coupling (AAC), phase–amplitude coupling (PAC) and phase–phase coupling (PPC) within each electrode were calculated for the recorded signals in order to use them as input to SVM classifier. The results show that phase-phase coupling can provide more information about the category of objects compared to other types of CFC, by classification performance of 92.33% against 70.28% and 60.52% for phase–amplitude and amplitude–amplitude coupling, respectively. In addition, this feature was more informative when occurred between alpha and gamma frequency bands. The performance of classification by means of CFC features was then compared with the result of classification by wavelet transform on the same data. We observed that PAC can improve the performance of categorical-based classification relative to wavelet coefficients (with the performance of 70.73%). So we can conclude that cross frequency coupling encompass useful information about semantic category of stimuli which is not available in time-frequency components of the signal obtained by wavelet transform.

Phonocardiogram signal compression using sound repetition and vector quantization

BackgroundA phonocardiogram (PCG) signal can be recorded for long-term heart monitoring. A huge amount of data is produced if the time of a recording is as long as days or weeks. It is necessary to compress the PCG signal to reduce storage space in a record and play system. In another situation, the PCG signal is transmitted to a remote health care center for automatic analysis in telemedicine. Compression of the PCG signal in that situation is necessary as a means for reducing the amount of data to be transmitted. Since heart beats are of a cyclical nature, compression can make use of the similarities in adjacent cycles by eliminating repetitive elements as redundant. This study proposes a new compression method that takes advantage of these repetitions. MethodsData compression proceeds in two stages, a training stage followed by the compression as such. In the training stage, a section of the PCG signal is selected and its sounds and murmurs (if any) decomposed into time–frequency components. Basic components are extracted from these by clustering and collected to form a dictionary that allows the generative reconstruction and retrieval of any heart sound or murmur. In the compression stage, the heart sounds and murmurs are reconstructed from the basic components stored in the dictionary. Compression is made possible because only the times of occurrence and the dictionary indices of the basic components need to be stored, which greatly reduces the number of bits required to represent heart sounds and murmurs. The residual that cannot be reconstructed in this manner appears as a random sequence and is further compressed by vector quantization. What we propose are quick search parameters for this vector quantization. ResultsFor normal PCG signals the compression ratio ranges from 20 to 149, for signals with median murmurs it ranges from 14 to 35, and for those with heavy murmurs, from 8 to 20, subject to a degree of distortion of ~5% (in percent root-mean-square difference) and a sampling frequency of 4kHz. DiscussionWe discuss the selection of the training signal and the contribution of vector quantization. Performance comparisons between the method proposed in this study and existing methods are conducted by computer simulations. ConclusionsWhen recording and compressing cyclical sounds, any repetitive components can be removed as redundant. The redundancies in the residual can be reduced by vector quantization. The method proposed in this study achieves a better performance than existing methods.

Functional decomposition of the human ERG based on the discrete wavelet transform.

The morphology of the electroretinogram (ERG) can be altered as a result of normal and pathological processes of the retina. However, given that the ERG is almost solely assessed in terms of its amplitude and timing, defining the shape of the ERG waveform so that subtle, physiologically driven, morphological changes can be systematically and reproducibly detected remains a challenging problem. We examined if the discrete wavelet transform (DWT) could meet this challenge. Normal human photopic ERGs evoked to a broad range of luminance intensities (to yield waveforms of various shapes, amplitudes, and timings) were analyzed using DWT descriptors of the ERG. Luminance-response curves that were generated using the various DWT descriptors revealed distinct (p < 0.05) luminance-dependence patterns, indicating that the stimulus luminance differently modulates the various time-frequency components of the ERG and thus its morphology. The latter represents the first attempt to study the luminance-dependence of ERG descriptors obtained with the DWT. Analyses of ERGs obtained from patients affected with ON or OFF retinal pathway anomalies were also presented. We show here for the first time that distinct time-frequency descriptors can be specifically associated to the function of the ON and OFF cone pathway. Therefore, in this study, the DWT revealed reproducible, physiologically meaningful and diagnostically relevant descriptors of the ERG over a wide range of signal amplitudes and morphologies. The DWT analysis thus represents a valuable addition to the electrophysiologist's armamentarium that will improve the quantification and interpretation of normal and pathological ERG responses.

Analysis of multi-scale chaotic characteristics of wind power based on Hilbert–Huang transform and Hurst analysis

The causes of uncertainty in wind farm power generation are not yet fully understood. A method for the scale division of wind power based on the Hilbert–Huang transform (HHT) and Hurst analysis is proposed in this paper, which allows the various multi-scale chaotic characteristics of wind power to be investigated to reveal further information about the dynamic behavior of wind power. First, the time–frequency characteristics of wind power are analyzed using the HHT, and then Hurst analysis is applied to analyze the stochastic/persistent characteristics of the different time–frequency components. Second, based on their fractal structures, the components are superposed and reconstructed into three series, which are defined as the Micro-, Meso- and Macro-scale subsequences. Finally, indices related to the statistical and behavioral characteristics of the subsequences are calculated and used to analyze their nonlinear dynamic behavior. The data collected from a wind farm of Hebei Province, China, are selected for case studies. The simulation results reveal that (1) although the time–frequency components can be decomposed, the different fractal structures of the signal are also derived from the original series; (2) the three scale subsequences all present chaotic characteristics and each of them exhibits its own unique properties. The Micro-scale subsequence shows strong randomness and contributes the least to the overall fluctuations; the Macro-scale subsequence is the steadiest and exhibits the most significant tendency; the Meso-scale subsequence which possesses the greatest variance contribution rate and the maximum largest Lyapunov exponent, is the dominant factor driving the fluctuation and dynamic behavior of wind power; (3) the short-term predictions of these three subsequences based on extreme learning machine (ELM) and least-squares support vector machine (LSSVM) models have validated the above analysis results, which show that the number of steps of look-ahead predictability have pursued an ordinal trend in term of the Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) and the prediction error contribution rate of the Meso-scale subsequence is the maximum. Furthermore, the short-term wind power forecasting of 6-step-ahead based on the multi-scale analysis is performed by EMD-LSSVM+ELM and the normalized Mean Absolute Error (nMAE) and normalized Root Mean Square Error (nRMSE) have been decreased by 49.45% and 44.30% compared with those of LSSVM, and 37.96% and 27.12% compared with those of EMD-LSSVM, respectively.

Time-frequency patterns of somatosensory evoked potentials in predicting the location of spinal cord injury

Somatosensory evoked potentials (SEPs) were found to exhibit different time-frequency patterns after acute spinal cord injury (SCI) at different levels, which implies that changes of these patterns may be associated with the location of SCI. Based on this finding, we propose the hypothesis that there are information regarding the location of SCI contained in the time-frequency patterns of SEPs. Purpose of the present study is to verify this hypothesis by comparing the time-frequency patterns of SEPs after acute and chronic SCI at the same level. The study examined the distribution patterns of the time-frequency components (TFCs) of SEPs before and after acute and chronic injury at C5 level in the spinal cord. Experimental results of SEP recordings from 24 adult rats show that there are common areas in the time-frequency distributions of SEPs. The TFCs from both the acute injury group and the chronic injury group are located in these areas with no TFCs from the normal group. Findings suggest that these areas are likely to possess information concerning the site of neurological deficits in spinal cord while independent of the modality of injury. This study provides basis for identification of stable time-frequency patterns of SEPs after different types and locations of SCI, which will guide the development of SEP-based SCI location detection.

Scalp high frequency oscillations (HFOs) in absence epilepsy: An independent component analysis (ICA) based approach

BackgroundHigh frequency oscillations (HFOs) have provided a new insight in understanding ictogenesis and seizure localization. In absence seizures, HFOs were predominantly localized in the medial prefrontal cortex (MPFC) in MEG studies. MethodsWe studied HFOs (80–250Hz) in scalp EEGs in patients with absence epilepsy, and evaluated their frequency bandwidth and spatial-temporal distribution. EEG of 9 patients with absence epilepsy (CAE:JAE–8:1; M:F=7:2; age: 8.1±2.1years; age at onset: 6.6±1.3 years) were evaluated with scalp EEG (sampling rate: 2048Hz). Finite impulse response filters on the unipolar or longitudinal bipolar montages were band-passed between 80 and 250Hz using Cartool® and EEGLAB®. Sensitivity and paper speed were modified accordingly to study the HFOs. Thousand four hundred and thirty eight artifact free ‘spike-wave’ epileptiform discharges were analyzed. Sleep ictal SW discharges, defined as runs of 3Hz GSWDs lasting ≥4s in stages 1 and 2 sleep, were analyzed by independent component analysis and component time-frequency and channel time-frequency maps using cyclical tapering wavelet transform. A total of 926 HFOs were identified of 1438 GSWDS. ResultsThe HFOs were associated with inter-ictal generalized spike-wave discharges (IiGSWDs-241/454), ictal GSWDs (IcGSWD-634/884), sporadic SWDs (sSWDs-51/100). The percentage of HFOs was higher in IcGSWD when compared to both IiGSWDs and sporadic SWDs together (χ2=52.864, d.o.f=1, p<0.0001). The mean frequency of HFOs was 96.4±10.4Hz. A channel wise analysis showed the maximum HFO band width in the right fronto-central region (F4=28Hz, C4=24Hz). ConclusionNarrow bandwidth interictal and ictal HFOs can be identified in scalp EEG of patients with absence epilepsy. Further characterization of the various properties of pHFOs will be helpful in opening up a domain of clinical evaluation and interpretation of the various epileptic disorders. To improve the insights into the onset and spread of absence seizures, and to study the network properties, one could analyze the HFOs in high density EEG arrays with multimodal integration using MEG or fMRI.

Effect of "Touch Rugby" Training on the Cardiovascular Autonomic Control In Sedentary Subjects.

This study aimed to explore the effects of "touch-rugby" training on the cardiovascular autonomic control in sedentary subjects. 22 adults (30-64 years old) were included in this study. Before (pre-test) and after (post-test) the period of training, cardio-respiratory recordings were achieved at rest and during a graded maximal exercise on a treadmill. The Smoothed-Pseudo-Wigner-Ville Distribution provided instantaneous time frequency components of RR intervals and systolic blood pressure variability in low- and high-frequency bands. The baroreflex sensitivity was assessed in low-frequency and high-frequency bands. Between pre-test and post-test, resting heart rate (74±10 vs. 69±12 beats.min(-1), p<0.05) and systolic blood pressure (118±19 vs. 103±22 mm Hg, p<0.01) decreased. Root mean square of successive differences (34.6±30.1 vs. 47.6±34.8 ms, p<0.001), high-frequency RR variability (590±288 vs. 1262±767 ms², p<0.001) increased whereas low-frequency/high-frequency ratio decreased (3.5±3.4 vs. 1.5±0.9, p<0.05). The high-frequency baroreflex sensitivity increased (13.4±10.1 vs. 26.0±20.9 ms.mmHg(-1), p<0.05). Playing touch rugby with one session weekly over 3 months modified the cardiovascular autonomic control of sedentary subjects. A decrease in the sympathetic tone combined with both an increase in the vagal tone and a decrease in systolic blood pressure at rest were observed. Therefore, such training appears to be beneficial to cardiac health.

Time–frequency audio feature extraction based on tensor representation of sparse coding

A time–frequency audio feature extraction scheme is proposed, in which features are decomposed from a frequency–time–scale tensor. The tensor, derived from a weight vector and a Gabor dictionary in sparse coding, represents the frequency, time centre and scale of transient time–frequency components with different dimensions. The distinguishing Gabor atoms are represented by individual tensor elements, and their associated coding weights are represented by tensor element values. The experimental results of sound effects classification showed performance improvement against that of sparse coding features.

Holter ECG-Based Apnea Hypopnea Index to Screen Obstructive Sleep Apnea: A New Proposal and Evaluation of Feasibility

Obstructive sleep apnea syndrome (OSAS) is a common sleep disorder. It has been reported that approximately 40% of patients with moderate or severe OSAS die within the first eight years of disease. In hospitals, OSAS is inspected using polysomnography, which uses a number of sensors. Because of the cumbersome nature of this polysomnography, an initial OSAS screening is usually conducted. In recent years, OSAS screening techniques using Holter electrocardiogram (ECG) have been reported. However, the techniques so far reported cannot perform an OSAS severity assessment. The present study presents a new method to distinguish the obstructive sleep apnea (OSA) and non-OSA epochs at one-second intervals based on the Apnea Hypopnea Index assessment, defined as the duration of continuous apnea. In the proposed method, the time-frequency components of the heart rate variability and three ECG-derived respiration signals calculated by the complex Morlet wavelet transformation are adopted as features. A support vector machine is employed for classification. The proposed method is evaluated using three eight-hour ECG recordings containing OSA episodes from three subjects. As a result, the sensitivity and specificity of classification are found to reach approximately 90%, a level suitable for OSAS screening in clinical settings.

Susceptibility of linear and nonlinear otoacoustic emission components to low-dose styrene exposure

Objective: To investigate potential susceptibility of active cochlear mechanisms to low-level styrene exposure by comparing TEOAEs in workers and controls. Design: Two advanced analysis techniques were applied to detect sub-clinical changes in linear and nonlinear cochlear mechanisms of OAE generation: the wavelet transform to decompose TEOAEs into time-frequency components and extract signal-to-noise ratio and latency of each component, and the bispectrum to detect and extract nonlinear TEOAE contributions as quadratic frequency couplings (QFCs). Study sample: Two cohorts of workers were examined: subjects exposed exclusively to styrene (N = 9), and subjects exposed to styrene and noise (N = 6). The control group was perfectly matched by age and sex to the exposed group. Results: Exposed subjects showed significantly lowered SNR in TEOAE components at mid-to-high frequencies (above 1.6 kHz) and a shift of QFC distribution towards lower frequencies than controls. No systematic differences were observed in latency. Conclusion: Low-level styrene exposure may have induced a modification of cochlear functionality as concerns linear and nonlinear OAE generation mechanisms. The lack of change in latency seems to suggest that the OAE components, where generation region and latency are tightly coupled, may not have been affected by styrene and noise exposure levels considered here.

Cortical EEG components that reflect inverse effectiveness during visuotactile integration processing

Multisensory integration is required to interpret information from multiple senses and then produce the appropriate behavioral output. Inverse effectiveness, a phenomenon in which weaker stimuli induce greater activation of multisensory neurons than do stronger stimuli, is an essential component of multisensory integration. The superior colliculus is especially abundant in multisensory neurons with properties of inverse effectiveness. However, multisensory neurons are distributed throughout the brain, suggesting that a wide range of brain regions are involved in generating multisensory behavior in addition to the superior colliculus. In this study, we aimed to use scalp EEG to elucidate the cortical responses that respond to multisensory stimuli according to the principle of inverse effectiveness. By modulating the intensity of the tactile aspect of a simultaneous visuotactile stimulus, we explored the time–frequency component of scalp EEG waveforms produced during multi-sensory stimulation. Using this method, we determined the relative values and temporal dynamics of the increment of power spectrum (event-related spectrum perturbation) and phase coherence across trials (e.g., inter-trial phase coherency) produced by multisensory stimulation compared to unisensory stimulation. In the somatosensory and anterior cingulate cortices, we observed significant differences in the inter-trial phase coherence of the theta band oscillation 200–400ms post-stimulus, in response to visuotactile (multisensory) and tactile (unisensory) stimulation, while no differences were found in later time windows and other cortical areas. These results suggest that inverse effectiveness is an important aspect of multi-sensory processing in the somato-sensory and frontal cerebral cortices.

Time-Frequency Analysis as Probabilistic Inference

This paper proposes a new view of time-frequency analysis framed in terms of probabilistic inference. Natural signals are assumed to be formed by the superposition of distinct time-frequency components, with the analytic goal being to infer these components by application of Bayes' rule. The framework serves to unify various existing models for natural time-series; it relates to both the Wiener and Kalman filters, and with suitable assumptions yields inferential interpretations of the short-time Fourier transform, spectrogram, filter bank, and wavelet representations. Value is gained by placing time-frequency analysis on the same probabilistic basis as is often employed in applications such as denoising, source separation, or recognition. Uncertainty in the time-frequency representation can be propagated correctly to application-specific stages, improving the handing of noise and missing data. Probabilistic learning allows modules to be co-adapted; thus, the time-frequency representation can be adapted to both the demands of the application and the time-varying statistics of the signal at hand. Similarly, the application module can be adapted to fine properties of the signal propagated by the initial time-frequency processing. We demonstrate these benefits by combining probabilistic time-frequency representations with non-negative matrix factorization, finding benefits in audio denoising and inpainting tasks, albeit with higher computational cost than incurred by the standard approach.

Application of sparse time-frequency decomposition to seismic data

The Gabor and S transforms are frequently used in time-frequency decomposition methods. Constrained by the uncertainty principle, both transforms produce low-resolution time-frequency decomposition results in the time and frequency domains. To improve the resolution of the time-frequency decomposition results, we use the instantaneous frequency distribution function (IFDF) to express the seismic signal. When the instantaneous frequencies of the nonstationary signal satisfy the requirements of the uncertainty principle, the support of IFDF is just the support of the amplitude ridges in the signal obtained using the short-time Fourier transform. Based on this feature, we propose a new iteration algorithm to achieve the sparse time-frequency decomposition of the signal. The iteration algorithm uses the support of the amplitude ridges of the residual signal obtained with the short-time Fourier transform to update the time-frequency components of the signal. The summation of the updated time-frequency components in each iteration is the result of the sparse time-frequency decomposition. Numerical examples show that the proposed method improves the resolution of the time-frequency decomposition results and the accuracy of the analysis of the nonstationary signal. We also use the proposed method to attenuate the ground roll of field seismic data with good results.