Speech Brain-computer Interface Research Articles

Speech motor cortex enables BCI cursor control and click.

Decoding neural activity from ventral (speech) motor cortex is known to enable high-performance speech brain-computer interface (BCI) control. It was previously unknown whether this brain area could also enable computer control via neural cursor and click, as is typically associated with dorsal (arm and hand) motor cortex. We recruited a clinical trial participant with ALS and implanted intracortical microelectrode arrays in ventral precentral gyrus (vPCG), which the participant used to operate a speech BCI in a prior study. We developed a cursor BCI driven by the participant's vPCG neural activity, and evaluated performance on a series of target selection tasks. The reported vPCG cursor BCI enabled rapidly-calibrating (40 seconds), accurate (2.90 bits per second) cursor control and click. The participant also used the BCI to control his own personal computer independently. These results suggest that placing electrodes in vPCG to optimize for speech decoding may also be a viable strategy for building a multi-modal BCI which enables both speech-based communication and computer control via cursor and click.

Continuous and discrete decoding of overt speech with scalp electroencephalography (EEG).

Neurological disorders affecting speech production adversely impact quality of
life for over 7 million individuals in the US. Traditional speech interfaces like eyetracking
devices and P300 spellers are slow and unnatural for these patients. An
alternative solution, speech Brain-Computer Interfaces (BCIs), directly decodes speech
characteristics, offering a more natural communication mechanism. This research
explores the feasibility of decoding speech features using non-invasive EEG. Nine
neurologically intact participants were equipped with a 63-channel EEG system
with additional sensors to eliminate eye artifacts. Participants read aloud sentences
displayed on a screen selected for phonetic similarity to the English language. Deep
learning models, including Convolutional Neural Networks and Recurrent Neural
Networks with and without attention modules, were optimized with a focus on
minimizing trainable parameters and utilizing small input window sizes for real-time
application. These models were employed for discrete and continuous speech decoding
tasks, achieving statistically significant participant-independent decoding performance
for discrete classes and continuous characteristics of the produced audio signal. A
frequency sub-band analysis highlighted the significance of certain frequency bands
(delta, theta, and gamma) for decoding performance, and a perturbation analysis
was used to identify crucial channels. Assessed channel selection methods did not
significantly improve performance, suggesting a distributed representation of speech
information encoded in the EEG signals. Leave-One-Out training demonstrated
the feasibility of utilizing common speech neural correlates, reducing data collection
requirements from individual participants.

Iterative alignment discovery of speech-associated neural activity.

Objective. Brain-computer interfaces (BCIs) have the potential to preserve or restore speech in patients with neurological disorders that weaken the muscles involved in speech production. However, successful training of low-latency speech synthesis and recognition models requires alignment of neural activity with intended phonetic or acoustic output with high temporal precision. This is particularly challenging in patients who cannot produce audible speech, as ground truth with which to pinpoint neural activity synchronized with speech is not available.Approach. In this study, we present a new iterative algorithm for neural voice activity detection (nVAD) called iterative alignment discovery dynamic time warping (IAD-DTW) that integrates DTW into the loss function of a deep neural network (DNN). The algorithm is designed to discover the alignment between a patient's electrocorticographic (ECoG) neural responses and their attempts to speak during collection of data for training BCI decoders for speech synthesis and recognition.Main results. To demonstrate the effectiveness of the algorithm, we tested its accuracy in predicting the onset and duration of acoustic signals produced by able-bodied patients with intact speech undergoing short-term diagnostic ECoG recordings for epilepsy surgery. We simulated a lack of ground truth by randomly perturbing the temporal correspondence between neural activity and an initial single estimate for all speech onsets and durations. We examined the model's ability to overcome these perturbations to estimate ground truth. IAD-DTW showed no notable degradation (<1% absolute decrease in accuracy) in performance in these simulations, even in the case of maximal misalignments between speech and silence.Significance. IAD-DTW is computationally inexpensive and can be easily integrated into existing DNN-based nVAD approaches, as it pertains only to the final loss computation. This approach makes it possible to train speech BCI algorithms using ECoG data from patients who are unable to produce audible speech, including those with Locked-In Syndrome.

Stability of ECoG high gamma signals during speech and implications for a speech BCI system in an individual with ALS: a year-long longitudinal study

Objective. Speech brain–computer interfaces (BCIs) have the potential to augment communication in individuals with impaired speech due to muscle weakness, for example in amyotrophic lateral sclerosis (ALS) and other neurological disorders. However, to achieve long-term, reliable use of a speech BCI, it is essential for speech-related neural signal changes to be stable over long periods of time. Here we study, for the first time, the stability of speech-related electrocorticographic (ECoG) signals recorded from a chronically implanted ECoG BCI over a 12 month period. Approach. ECoG signals were recorded by an ECoG array implanted over the ventral sensorimotor cortex in a clinical trial participant with ALS. Because ECoG-based speech decoding has most often relied on broadband high gamma (HG) signal changes relative to baseline (non-speech) conditions, we studied longitudinal changes of HG band power at baseline and during speech, and we compared these with residual high frequency noise levels at baseline. Stability was further assessed by longitudinal measurements of signal-to-noise ratio, activation ratio, and peak speech-related HG response magnitude (HG response peaks). Lastly, we analyzed the stability of the event-related HG power changes (HG responses) for individual syllables at each electrode. Main Results. We found that speech-related ECoG signal responses were stable over a range of syllables activating different articulators for the first year after implantation. Significance. Together, our results indicate that ECoG can be a stable recording modality for long-term speech BCI systems for those living with severe paralysis. Clinical Trial Information. ClinicalTrials.gov, registration number NCT03567213.

Evoking artificial speech perception through invasive brain stimulation for brain-computer interfaces: current challenges and future perspectives.

Encoding artificial perceptions through brain stimulation, especially that of higher cognitive functions such as speech perception, is one of the most formidable challenges in brain-computer interfaces (BCI). Brain stimulation has been used for functional mapping in clinical practices for the last 70 years to treat various disorders affecting the nervous system, including epilepsy, Parkinson's disease, essential tremors, and dystonia. Recently, direct electrical stimulation has been used to evoke various forms of perception in humans, ranging from sensorimotor, auditory, and visual to speech cognition. Successfully evoking and fine-tuning artificial perceptions could revolutionize communication for individuals with speech disorders and significantly enhance the capabilities of brain-computer interface technologies. However, despite the extensive literature on encoding various perceptions and the rising popularity of speech BCIs, inducing artificial speech perception is still largely unexplored, and its potential has yet to be determined. In this paper, we examine the various stimulation techniques used to evoke complex percepts and the target brain areas for the input of speech-like information. Finally, we discuss strategies to address the challenges of speech encoding and discuss the prospects of these approaches.

Text and image generation from intracranial electroencephalography using an embedding space for text and images

Objective. Invasive brain–computer interfaces (BCIs) are promising communication devices for severely paralyzed patients. Recent advances in intracranial electroencephalography (iEEG) coupled with natural language processing have enhanced communication speed and accuracy. It should be noted that such a speech BCI uses signals from the motor cortex. However, BCIs based on motor cortical activities may experience signal deterioration in users with motor cortical degenerative diseases such as amyotrophic lateral sclerosis. An alternative approach to using iEEG of the motor cortex is necessary to support patients with such conditions. Approach. In this study, a multimodal embedding of text and images was used to decode visual semantic information from iEEG signals of the visual cortex to generate text and images. We used contrastive language-image pretraining (CLIP) embedding to represent images presented to 17 patients implanted with electrodes in the occipital and temporal cortices. A CLIP image vector was inferred from the high-γ power of the iEEG signals recorded while viewing the images. Main results. Text was generated by CLIPCAP from the inferred CLIP vector with better-than-chance accuracy. Then, an image was created from the generated text using StableDiffusion with significant accuracy. Significance. The text and images generated from iEEG through the CLIP embedding vector can be used for improved communication.

Considerations for implanting speech brain computer interfaces based on functional magnetic resonance imaging

Objective. Brain-computer interfaces (BCIs) have the potential to reinstate lost communication faculties. Results from speech decoding studies indicate that a usable speech BCI based on activity in the sensorimotor cortex (SMC) can be achieved using subdurally implanted electrodes. However, the optimal characteristics for a successful speech implant are largely unknown. We address this topic in a high field blood oxygenation level dependent functional magnetic resonance imaging (fMRI) study, by assessing the decodability of spoken words as a function of hemisphere, gyrus, sulcal depth, and position along the ventral/dorsal-axis. Approach. Twelve subjects conducted a 7T fMRI experiment in which they pronounced 6 different pseudo-words over 6 runs. We divided the SMC by hemisphere, gyrus, sulcal depth, and position along the ventral/dorsal axis. Classification was performed on in these SMC areas using multiclass support vector machine (SVM). Main results. Significant classification was possible from the SMC, but no preference for the left or right hemisphere, nor for the precentral or postcentral gyrus for optimal word classification was detected. Classification while using information from the cortical surface was slightly better than when using information from deep in the central sulcus and was highest within the ventral 50% of SMC. Confusion matrices where highly similar across the entire SMC. An SVM-searchlight analysis revealed significant classification in the superior temporal gyrus and left planum temporale in addition to the SMC. Significance. The current results support a unilateral implant using surface electrodes, covering the ventral 50% of the SMC. The added value of depth electrodes is unclear. We did not observe evidence for variations in the qualitative nature of information across SMC. The current results need to be confirmed in paralyzed patients performing attempted speech.

Evaluation of an English language phoneme-based imagined speech brain computer interface with low-cost electroencephalography.

Paralyzed and physically impaired patients face communication difficulties, even when they are mentally coherent and aware. Electroencephalographic (EEG) brain-computer interfaces (BCIs) offer a potential communication method for these people without invasive surgery or physical device controls. Although virtual keyboard protocols are well documented in EEG BCI paradigms, these implementations are visually taxing and fatiguing. All English words combine 44 unique phonemes, each corresponding to a unique EEG pattern. In this study, a complete phoneme-based imagined speech EEG BCI was developed and tested on 16 subjects. Using open-source hardware and software, machine learning models, such as k-nearest neighbor (KNN), reliably achieved a mean accuracy of 97 ± 0.001%, a mean F1 of 0.55 ± 0.01, and a mean AUC-ROC of 0.68 ± 0.002 in a modified one-versus-rest configuration, resulting in an information transfer rate of 304.15 bits per minute. In line with prior literature, the distinguishing feature between phonemes was the gamma power on channels F3 and F7. However, adjustments to feature selection, trial window length, and classifier algorithms may improve performance. In summary, these are iterative changes to a viable method directly deployable in current, commercially available systems and software. The development of an intuitive phoneme-based EEG BCI with open-source hardware and software demonstrates the potential ease with which the technology could be deployed in real-world applications.

Stable Decoding from a Speech BCI Enables Control for an Individual with ALS without Recalibration for 3 Months.

Brain-computer interfaces (BCIs) can be used to control assistive devices by patients with neurological disorders like amyotrophic lateral sclerosis (ALS) that limit speech and movement. For assistive control, it is desirable for BCI systems to be accurate and reliable, preferably with minimal setup time. In this study, a participant with severe dysarthria due to ALS operates computer applications with six intuitive speech commands via a chronic electrocorticographic (ECoG) implant over the ventral sensorimotor cortex. Speech commands are accurately detected and decoded (median accuracy: 90.59%) throughout a 3-month study period without model retraining or recalibration. Use of the BCI does not require exogenous timing cues, enabling the participant to issue self-paced commands at will. These results demonstrate that a chronically implanted ECoG-based speech BCI can reliably control assistive devices over long time periods with only initial model training and calibration, supporting the feasibility of unassisted home use.

Recommendations for promoting user agency in the design of speech neuroprostheses.

Brain-computer interfaces (BCI) that directly decode speech from brain activity aim to restore communication in people with paralysis who cannot speak. Despite recent advances, neural inference of speech remains imperfect, limiting the ability for speech BCIs to enable experiences such as fluent conversation that promote agency - that is, the ability for users to author and transmit messages enacting their intentions. Here, we make recommendations for promoting agency based on existing and emerging strategies in neural engineering. The focus is on achieving fast, accurate, and reliable performance while ensuring volitional control over when a decoder is engaged, what exactly is decoded, and how messages are expressed. Additionally, alongside neuroscientific progress within controlled experimental settings, we argue that a parallel line of research must consider how to translate experimental successes into real-world environments. While such research will ultimately require input from prospective users, here we identify and describe design choices inspired by human-factors work conducted in existing fields of assistive technology, which address practical issues likely to emerge in future real-world speech BCI applications.

A high-performance speech neuroprosthesis

Speech brain–computer interfaces (BCIs) have the potential to restore rapid communication to people with paralysis by decoding neural activity evoked by attempted speech into text1,2 or sound3,4. Early demonstrations, although promising, have not yet achieved accuracies sufficiently high for communication of unconstrained sentences from a large vocabulary1–7. Here we demonstrate a speech-to-text BCI that records spiking activity from intracortical microelectrode arrays. Enabled by these high-resolution recordings, our study participant—who can no longer speak intelligibly owing to amyotrophic lateral sclerosis—achieved a 9.1% word error rate on a 50-word vocabulary (2.7 times fewer errors than the previous state-of-the-art speech BCI2) and a 23.8% word error rate on a 125,000-word vocabulary (the first successful demonstration, to our knowledge, of large-vocabulary decoding). Our participant’s attempted speech was decoded at 62 words per minute, which is 3.4 times as fast as the previous record8 and begins to approach the speed of natural conversation (160 words per minute9). Finally, we highlight two aspects of the neural code for speech that are encouraging for speech BCIs: spatially intermixed tuning to speech articulators that makes accurate decoding possible from only a small region of cortex, and a detailed articulatory representation of phonemes that persists years after paralysis. These results show a feasible path forward for restoring rapid communication to people with paralysis who can no longer speak.

Online Ternary Classification of Covert Speech by Leveraging the Passive Perception of Speech.

Brain-computer interfaces (BCIs) provide communicative alternatives to those without functional speech. Covert speech (CS)-based BCIs enable communication simply by thinking of words and thus have intuitive appeal. However, an elusive barrier to their clinical translation is the collection of voluminous examples of high-quality CS signals, as iteratively rehearsing words for long durations is mentally fatiguing. Research on CS and speech perception (SP) identifies common spatiotemporal patterns in their respective electroencephalographic (EEG) signals, pointing towards shared encoding mechanisms. The goal of this study was to investigate whether a model that leverages the signal similarities between SP and CS can differentiate speech-related EEG signals online. Ten participants completed a dyadic protocol where in each trial, they listened to a randomly selected word and then subsequently mentally rehearsed the word. In the offline sessions, eight words were presented to participants. For the subsequent online sessions, the two most distinct words (most separable in terms of their EEG signals) were chosen to form a ternary classification problem (two words and rest). The model comprised a functional mapping derived from SP and CS signals of the same speech token (features are extracted via a Riemannian approach). An average ternary online accuracy of 75.3% (60% chance level) was achieved across participants, with individual accuracies as high as 93%. Moreover, we observed that the signal-to-noise ratio (SNR) of CS signals was enhanced by perception-covert modeling according to the level of high-frequency ([Formula: see text]-band) correspondence between CS and SP. These findings may lead to less burdensome data collection for training speech BCIs, which could eventually enhance the rate at which the vocabulary can grow.

The Effects of Different Brain Regions on fNIRS-based Task-state Detection in Speech Imagery.

Brain-computer interface (BCI) based on speech imagery can decode users' verbal intent and help people with motor disabilities communicate naturally. Functional near-infrared spectroscopy (fNIRS) is a commonly used brain signal acquisition method. Asynchronous BCI can response to control commands at any time, which provides great convenience for users. Task state detection, defined as identifying whether user starts or continues covertly articulating, plays an important role in speech imagery BCIs. To better distinguish task state from idle state during speech imagery, this work used fNIRS signals from different brain regions to study the effects of different brain regions on task state detection accuracy. The imagined tonal syllables included four lexical tones and four vowels in Mandarin Chinese. The brain regions that were measured included Broca's area, Wernicke's area, Superior temporal cortex and Motor cortex. Task state detection accuracies of imagining tonal monosyllables with four different tones were analyzed. The average accuracy of four speech imagery tasks based on the whole brain was 0.67 and it was close to 0.69, which was the average accuracy based on Broca's area. The accuracies of Broca's area and the whole brain were significantly higher than those of other brain regions. The findings of this work demonstrated that using a few channels of Broca's area could result in a similar task state detection accuracy to that using all the channels of the brain. Moreover, it was discovered that speech imagery with tone 2/3 tasks yielded higher task state detection accuracy than speech imagery with other tones.

Decoding and synthesizing tonal language speech from brain activity.

Recent studies have shown that the feasibility of speech brain-computer interfaces (BCIs) as a clinically valid treatment in helping nontonal language patients with communication disorders restore their speech ability. However, tonal language speech BCI is challenging because additional precise control of laryngeal movements to produce lexical tones is required. Thus, the model should emphasize the features from the tonal-related cortex. Here, we designed a modularized multistream neural network that directly synthesizes tonal language speech from intracranial recordings. The network decoded lexical tones and base syllables independently via parallel streams of neural network modules inspired by neuroscience findings. The speech was synthesized by combining tonal syllable labels with nondiscriminant speech neural activity. Compared to commonly used baseline models, our proposed models achieved higher performance with modest training data and computational costs. These findings raise a potential strategy for approaching tonal language speech restoration.

Decoding Articulation Motor Imagery using Early Connectivity Information in the Motor Cortex: A Functional Near-infrared Spectroscopy Study.

Brain computer interface (BCI) based on speech imagery can help people with motor disorders communicate their thoughts to the outside world in a natural way. Due to being portable, non-invasive, and safe, functional near-infrared spectroscopy (fNIRS) is preferred for developing BCIs. Previous BCIs based on fNIRS mainly relied on activation information, which ignored the functional connectivity between neural areas. In this study, a 4-class speech imagery BCI based on fNIRS is presented to decode simplified articulation motor imagery (only the movements of jaw and lip were retained) of different vowels. Synchronization information in the motor cortex was extracted as features. In multiclass (four classes) settings, the mean subject-dependent classification accuracies approximated or exceeded 40% in the 0-2.5 s and 0-10 s time windows, respectively. In binary class settings (the average classification accuracies of all pairwise comparisons between two vowels), the mean subject-dependent classification accuracies exceeded 70% in the 0-2.5 s and 0-10 s time windows. These results demonstrate that connectivity features can effectively differentiate different vowels even if the time window size was reduced from 10 s to 2.5 s and the decoding performance in both the time windows was almost the same. This finding suggests that speech imagery BCI based on fNIRS can be further optimized in terms of feature extraction and command generation time reduction. In addition, simplified articulation motor imagery of vowels can be distinguished, and therefore, the potential contribution of articulation motor imagery information extracted from the motor cortex should be emphasized in speech imagery BCI based on fNIRS to improve decoding performance.

Dataset of Speech Production in intracranial Electroencephalography

Speech production is an intricate process involving a large number of muscles and cognitive processes. The neural processes underlying speech production are not completely understood. As speech is a uniquely human ability, it can not be investigated in animal models. High-fidelity human data can only be obtained in clinical settings and is therefore not easily available to all researchers. Here, we provide a dataset of 10 participants reading out individual words while we measured intracranial EEG from a total of 1103 electrodes. The data, with its high temporal resolution and coverage of a large variety of cortical and sub-cortical brain regions, can help in understanding the speech production process better. Simultaneously, the data can be used to test speech decoding and synthesis approaches from neural data to develop speech Brain-Computer Interfaces and speech neuroprostheses.

Multiclass Classification of Imagined Speech Vowels and Words of Electroencephalography Signals Using Deep Learning

The paper’s emphasis is on the imagined speech decoding of electroencephalography (EEG) neural signals of individuals in accordance with the expansion of the brain-computer interface to encompass individuals with speech problems encountering communication challenges. Decoding an individual’s imagined speech from nonstationary and nonlinear EEG neural signals is a complex task. Related research work in the field of imagined speech has revealed that imagined speech decoding performance and accuracy require attention to further improve. The evolution of deep learning technology increases the likelihood of decoding imagined speech from EEG signals with enhanced performance. We proposed a novel supervised deep learning model that combined the temporal convolutional networks and the convolutional neural networks with the intent of retrieving information from the EEG signals. The experiment was carried out using an open-access dataset of fifteen subjects’ imagined speech multichannel signals of vowels and words. The raw multichannel EEG signals of multiple subjects were processed using discrete wavelet transformation technique. The model was trained and evaluated using the preprocessed signals, and the model hyperparameters were adjusted to achieve higher accuracy in the classification of imagined speech. The experiment results demonstrated that the multiclass imagined speech classification of the proposed model exhibited a higher overall accuracy of 0.9649 and a classification error rate of 0.0350. The results of the study indicate that individuals with speech difficulties might well be able to leverage a noninvasive EEG-based imagined speech brain-computer interface system as one of the long-term alternative artificial verbal communication mediums.

Mapping Acoustics to Articulatory Gestures in Dutch: Relating Speech Gestures, Acoustics and Neural Data.

Completely locked-in patients suffer from paralysis affecting every muscle in their body, reducing their communication means to brain-computer interfaces (BCIs). State-of-the-art BCIs have a slow spelling rate, which inevitably places a burden on patients' quality of life. Novel techniques address this problem by following a bio-mimetic approach, which consists of decoding sensory-motor cortex (SMC) activity that underlies the movements of the vocal tract's articulators. As recording articulatory data in combination with neural recordings is often unfeasible, the goal of this study was to develop an acoustic-to-articulatory inversion (AAI) model, i.e. an algorithm that generates articulatory data (speech gestures) from acoustics. A fully convolutional neural network was trained to solve the AAI mapping, and was tested on an unseen acoustic set, recorded simultaneously with neural data. Representational similarity analysis was then used to assess the relationship between predicted gestures and neural responses. The network's predictions and targets were significantly correlated. Moreover, SMC neural activity was correlated to the vocal tract gestural dynamics. The present AAI model has the potential to further our understanding of the relationship between neural, gestural and acoustic signals and lay the foundations for the development of a bio-mimetic speech BCI. Clinical Relevance- This study investigates the relationship between articulatory gestures during speech and the underlying neural activity. The topic is central for development of brain-computer interfaces for severely paralysed individuals.

Neural decoding of imagined speech from EEG signals using the fusion of graph signal processing and graph learning techniques

Imagined Speech (IS) is the imagination of speech without using the tongue or muscles. In recent studies, IS tasks are increasingly investigated for the Brain-Computer Interface (BCI) applications. Electroencephalography (EEG) signals, which record brain activity, can be used to analyze BCI-based tasks utilizing Machine Learning (ML) methods. The current paper considers decoding IS brain waves using the fusion of classical signal processing, Graph Signal Processing (GSP), and Graph Learning (GL) based features. The proposed fusion method, named GraphIS (short for a Graph-based Imagined Speech BCI decoder), is applied to the four-class classification (three classes of the imagined words, in addition to the rest state) on EEG recordings of fifteen subjects. Results show that GSP and GL-based features can highly improve the performance of classification outcomes compared to using only classical signal processing features and over the state-of-the-art Common Spatial Pattern (CSP) feature extractor by considering the spatial information of the signals as well as interactions between channels in regions of interest. The proposed GraphIS method leads to a mean accuracy of 50.10% in the studied four-class IS classification task, compared to using only one feature set with an accuracy of 47.86% and 46.10%, and also the state-of-the-art CSP with an accuracy of 47.10%. Additionally, using an EEG connectivity map of the electrode signals obtained from GL methods, we also found a strong connection in the right frontal region as well as in the left frontal regions during IS, which had not been focused on in the previous IS papers.

Idle-state detection in motor imagery of articulation using early information: A functional Near-infrared spectroscopy study

A speech imagery-based brain computer interface (BCI) provides an alternative way for people to interact with the outside world intuitively. Most speech imagery BCIs based on functional near-infrared spectroscopy (fNIRS) are disadvantageous in applications outside the laboratory due to being incapable of detecting asynchronous (self-paced) actions. This work aimed to develop a two-class asynchronous BCI that detects idle state in the context of motor imagery of articulation (MIA). In this study, 19 healthy subjects were asked to rehearse the Chinese vowels/a/and/u/covertly and to accommodate a rest state. A feature selection strategy was designed to combine time domain, spatial domain, and functional connectivity features. All differentiating information was extracted from fNIRS signals in a 0–2.5 s time window. Among single-modality features, the centrality features of brain networks performed better than any other features and yielded a subject-dependent classification accuracy rate of 75.1%. The subject-dependent classification accuracy rate of the combined features at 78.9% was significantly better than that of any single feature. These results demonstrate that it is feasible to detect idle states from active MIA states in a reduced time window size. The proposed combination of information propagation pattern, spatial pattern, and activation pattern was meaningful for extracting discriminative information, hence improving classification accuracy. The high classification accuracy and fast information transfer rate of the presented BCI show promising practicality for real world applications.