Statistical Learning Paradigm Research Articles

Perceptual Expectations Are Reflected by Early Alpha Power Reduction.

The predictability of a stimulus can be characterized by its transitional probability. Perceptual expectations derived from the transitional probability of the stimulus were found to modulate the early alpha oscillations in the sensory regions of the brain when neural responses to expected versus unexpected stimuli were compared. The objective of our study was to find out the extent to which this low-frequency oscillation reflects stimulus predictability. We aimed to detect the alpha-power difference with smaller differences in transitional probabilities by comparing expected stimuli with neutral ones. We studied the effect of expectation on perception by applying an unsupervised visual statistical learning paradigm with expected and neutral stimuli embedded in an image sequence while recording EEG. Time-frequency analysis showed that expected stimuli elicit lower alpha power in the window of 8-12 Hz and 0-400 msec after stimulus presentation, appearing in the centroparietal region. Comparing previous findings of expectancy-based alpha-band modulation with our results suggests that early alpha oscillation shows an inverse relationship with stimulus predictability. Although current data are insufficient to determine the origin of the alpha power reduction, this could be a potential sign of expectation suppression in cortical oscillatory activity.

Statistical learning of phonotactics by children can be affected by another statistical learning task

AbstractChildren typically produce high-frequency phonotactic sequences, such as the /st/ in “toaster,” more accurately than the lower frequency /mk/ in “tomcat.” This high-frequency advantage can be simulated experimentally with a statistical learning paradigm, and when 4-year-old children are familiarized with many examples of a sequence like /mk/, they generally produce it more accurately than if they are exposed to just a few examples. Here, we sought to expand our understanding of the high-frequency advantage, but surprisingly, we instead uncovered an exception. Twenty-nine children between 4 and 5 years of age completed a phonotactic statistical learning experiment, but they also completed a separate experiment focused on statistical learning of prosodic contours. The order of the experiments was randomized, with the phonotactic statistical learning experiment occurring first for half of the children. For the children who completed the phonotactic learning experiment first, the results were consistent with previous research and a high-frequency advantage. However, children who completed the phonotactic learning experiment second produced low-frequency sequences more accurately than high-frequency sequences. There is little precedent for the latter effect, but studies of multistream statistical learning may provide some context for unpacking and extending the result.

A Comparative Analysis of Slope Failure Prediction Using a Statistical and Machine Learning Approach on Displacement Data: Introducing a Tailored Performance Metric

Slope failures pose significant threats to human safety and vital infrastructure. The urgent need for the accurate prediction of these geotechnical events is driven by two main goals: advancing our understanding of the underlying geophysical mechanisms and establishing efficient evacuation protocols. Although traditional physics-based models offer in-depth insights, their reliance on numerous assumptions and parameters limits their practical usability. In our study, we constructed an experimental artificial slope and monitored it until failure, generating an in-depth displacement dataset. Leveraging this dataset, we developed and compared prediction models rooted in both statistical and machine learning paradigms. Furthermore, to bridge the gap between generic evaluation metrics and the specific needs of slope failure prediction, we introduced a bespoke performance. Our results indicate that while the statistical approach did not effectively provide early warnings, the machine learning models, when assessed with our bespoke performance metric, showed significant promise as reliable early warning systems. These findings hold potential to fortify disaster prevention measures and prioritize human safety.

Structure shapes the representation of a novel category.

Concepts contain rich structures that support flexible semantic cognition. These structures can be characterized by patterns of feature covariation: Certain features tend to cluster in the same items (e.g., feathers, wings, can fly). Existing computational models demonstrate how this kind of structure can be leveraged to slowly learn the distinctions between categories, on developmental timescales. However, it is not clear whether and how we leverage feature structure to quickly learn a novel category. We thus investigated how the internal structure of a new category is first extracted from experience, with the prediction that feature-based structure would have a rapid and broad influence on the learned category representation. Across three experiments, novel categories were designed with patterns of feature associations determined by carefully constructed graph structures, with Modular graphs-exhibiting strong clusters of feature covariation-compared against Random and Lattice graphs. In Experiment 1, a feature inference task using verbal stimuli revealed that Modular structure broadly facilitated category learning. Experiment 2 replicated this effect in visual categories. In Experiment 3, a statistical learning paradigm revealed that this Modular benefit relates to high-level structure rather than pairwise feature associations and persists even when category structure is incidental to the task. A neural network model was readily able to account for these effects, suggesting that correlational feature structure may be encoded within rapidly learned, distributed category representations. These findings constrain theories of category representation and link theories of category learning with structure learning more broadly. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Brain-correlates of processing local dependencies within a statistical learning paradigm

Statistical learning refers to the implicit mechanism of extracting regularities in our environment. Numerous studies have investigated the neural basis of statistical learning. However, how the brain responds to violations of auditory regularities based on prior (implicit) learning requires further investigation. Here, we used functional magnetic resonance imaging (fMRI) to investigate the neural correlates of processing events that are irregular based on learned local dependencies. A stream of consecutive sound triplets was presented. Unbeknown to the subjects, triplets were either (a) standard, namely triplets ending with a high probability sound or, (b) statistical deviants, namely triplets ending with a low probability sound. Participants (n = 33) underwent a learning phase outside the scanner followed by an fMRI session. Processing of statistical deviants activated a set of regions encompassing the superior temporal gyrus bilaterally, the right deep frontal operculum including lateral orbitofrontal cortex, and the right premotor cortex. Our results demonstrate that the violation of local dependencies within a statistical learning paradigm does not only engage sensory processes, but is instead reminiscent of the activation pattern during the processing of local syntactic structures in music and language, reflecting the online adaptations required for predictive coding in the context of statistical learning.

Interaction within and between cortical networks subserving multisensory learning and its reorganization due to musical expertise

Recent advancements in the field of network science allow us to quantify inter-network information exchange and model the interaction within and between task-defined states of large-scale networks. Here, we modeled the inter- and intra- network interactions related to multisensory statistical learning. To this aim, we implemented a multifeatured statistical learning paradigm and measured evoked magnetoencephalographic responses to estimate task-defined state of functional connectivity based on cortical phase interaction. Each network state represented the whole-brain network processing modality-specific (auditory, visual and audiovisual) statistical learning irregularities embedded within a multisensory stimulation stream. The way by which domain-specific expertise re-organizes the interaction between the networks was investigated by a comparison of musicians and non-musicians. Between the modality-specific network states, the estimated connectivity quantified the characteristics of a supramodal mechanism supporting the identification of statistical irregularities that are compartmentalized and applied in the identification of uni-modal irregularities embedded within multisensory stimuli. Expertise-related re-organization was expressed by an increase of intra- and a decrease of inter-network connectivity, showing increased compartmentalization.

The paradigm of statistical inference and the paradigm of statistical learning

The starting point for the presentation of the similarities and differences between the principles of conducting statistical research according to the rules of both statistical inference and statistical learning is the paradigm theory, formulated by Thomas Kuhn. In the first section of this paper, the essential features of the statistical inference paradigm are characterised, with particular attention devoted to its limitations in contemporary statistical research. Subsequently, the article presents the challenges faced by this research jointly with the expanding opportunities for their effective reduction. The essence of learning from data is discussed and the principles of statistical learning are defined. Moreover, significant features of the statistical learning paradigm are formulated in the context of the differences between the statistical inference paradigm and the statistical learning paradigm. It is emphasised that the statistical learning paradigm, as the more universal one of the two discussed, broadens the possibilities of conducting statistical research, especially in socio-economic sciences.

K-relevance vectors: Considering relevancy beside nearness

This study combines two different learning paradigms, k-nearest neighbor (k-NN) rule, as memory-based learning paradigm and relevance vector machines (RVM), as statistical learning paradigm. The purpose is to improve the performance of k-NN rule through selection of important features with sparse Bayesian learning method. This combination is performed in kernel space and is called k-relevance vector (k-RV). The proposed model significantly prunes irrelevant features. Our combination of k-NN and RVM presents a new concept of similarity measurement for k-NN rule, we call it k-relevancy which aims to consider “relevancy” in the feature space beside “nearness” in the input space. We also introduce a new parameter, responsible for early stopping of iterations in RVM that is able to improve the classification accuracy. Intensive experiments are conducted on several classification datasets from University of California Irvine (UCI) repository and two real datasets from computer vision domain. The performance of k-RV is highly competitive compared to a few state-of-the-arts in terms of classification accuracy.

Dynamics of nonlinguistic statistical learning: From neural entrainment to the emergence of explicit knowledge

Humans are highly attuned to patterns in the environment. This ability to detect environmental patterns, referred to as statistical learning, plays a key role in many diverse aspects of cognition. However, the spatiotemporal neural mechanisms underlying implicit statistical learning, and how these mechanisms may relate or give rise to explicit learning, remain poorly understood. In the present study, we investigated these different aspects of statistical learning by using an auditory nonlinguistic statistical learning paradigm combined with magnetoencephalography. Twenty-four healthy volunteers were exposed to structured and random tone sequences, and statistical learning was quantified by neural entrainment. Already early during exposure, participants showed strong entrainment to the embedded tone patterns. A significant increase in entrainment over exposure was detected only in the structured condition, reflecting the trajectory of learning. While source reconstruction revealed a wide range of brain areas involved in this process, entrainment in areas around the left pre-central gyrus as well as right temporo-frontal areas significantly predicted behavioral performance. Sensor level results confirmed this relationship between neural entrainment and subsequent explicit knowledge. These results give insights into the dynamic relation between neural entrainment and explicit learning of triplet structures, suggesting that these two aspects are systematically related yet dissociable. Neural entrainment reflects robust, implicit learning of underlying patterns, whereas the emergence of explicit knowledge, likely built on the implicit encoding of structure, varies across individuals and may depend on factors such as sufficient exposure time and attention.

A distributional perspective on the gavagai problem in early word learning

Word learning entails the mapping of an auditory word-form to its appropriate grammatical category (e.g., noun, verb, adjective), but before that mapping can occur, the naïve learner must infer which of the myriad of possible referents of that word was intended by the speaker. This creates a computational explosion of referential ambiguity referred to as the gavagai problem. In a set of corpus analyses of parent-directed speech to young infants, we describe the distributional information available to early word learners, with a focus on nouns and adjectives that refer to whole objects and object properties. And in two experiments on word-learning in adults spanning seven different distributional conditions, we document how variations in the ratio of novel labels for objects and properties affect the robustness of word learning. Our results suggest that the language input to 6- to 20-month-olds is robustly populated with high-frequency object words and high-frequency property words, but their co-occurrence is sparse. Although this distributional information slightly favors object words over property words, a more plausible account of the whole-object bias in early word learning is the inability to encode the details of an object/event during rapid naming. Our results from adults, presented with novel labels for multi-referent objects in a cross-situational statistical learning paradigm, also reveal this whole-object bias as well as the absence of property-label generalization to novel objects, even when the distribution of labels is shifted almost exclusively to property words. These results are discussed in terms of the relative ease of mapping auditory word-forms to whole objects vs. object properties, thereby limiting the combinatorics of the gavagai problem, especially in infants with immature encoding and memory representation abilities.

A study design for statistical learning technique to predict radiological progression with an application of idiopathic pulmonary fibrosis using chest CT images

BackgroundIdiopathic pulmonary fibrosis (IPF) is a fatal interstitial lung disease characterized by an unpredictable decline in lung function. Predicting IPF progression from the early changes in lung function tests have known to be a challenge due to acute exacerbation. Although it is unpredictable, the neighboring regions of fibrotic reticulation increase during IPF's progression. With this clinical information, quantitative characteristics of high-resolution computed tomography (HRCT) and a statistical learning paradigm, the aim is to build a model to predict IPF progression. DesignA paired set of anonymized 193 HRCT images from IPF subjects with 6–12 month intervals were collected retrospectively. The study was conducted in two parts: (1) Part A collects the ground truth in small regions of interest (ROIs) with labels of “expected to progress” or “expected to be stable” at baseline HRCT and develop a statistical learning model to classify voxels in the ROIs. (2) Part B uses the voxel-level classifier from Part A to produce whole-lung level scores of a single-scan total probability's (STP) baseline. MethodsUsing annotated ROIs from 71 subjects' HRCT scans in Part A, we applied Quantum Particle Swarm Optimization–Random Forest (QPSO-RF) to build the classifier. Then, 122 subjects' HRCT scans were used to test the prediction. Using Spearman rank correlations and survival analyses, we ascertained STP associations with 6–12 month changes in quantitative lung fibrosis and forced vital capacity. ConclusionThis study can serve as a reference for collecting ground truth, and developing statistical learning techniques to predict progression in medical imaging.

Statistically defined visual chunks engage object-based attention

Although objects are the fundamental units of our representation interpreting the environment around us, it is still not clear how we handle and organize the incoming sensory information to form object representations. By utilizing previously well-documented advantages of within-object over across-object information processing, here we test whether learning involuntarily consistent visual statistical properties of stimuli that are free of any traditional segmentation cues might be sufficient to create object-like behavioral effects. Using a visual statistical learning paradigm and measuring efficiency of 3-AFC search and object-based attention, we find that statistically defined and implicitly learned visual chunks bias observers’ behavior in subsequent search tasks the same way as objects defined by visual boundaries do. These results suggest that learning consistent statistical contingencies based on the sensory input contributes to the emergence of object representations.

The role of attention, inhibition and statistical learning in Chinese character recognition by novices

• We investigated statistical learning in logographic language, in novices. • In addition, we explored the role of attention and inhibition. • This study included a novel online implementation of experimental procedures. • Congruent with our hypotheses, novices displayed statistical learning. • Our results do not support a clear role of attention and inhibition. Statistical learning is a universal learning mechanism and has been extensively studied in alphabetic languages, yet less explored in logographic language. The present study aimed to address this research gap and to assess the role of attention and inhibition in statistical learning. By applying a browser-based experiment tool, statistical learning was measured by an adapted statistical learning paradigm. Attention and inhibition were measured by the Go/No-Go task. 38 non-Chinese participants performed in the study. The results showed that accuracy and response speed was higher for characters with high occurrence probability relative to lower occurrence probability, but this was restricted to whole characters. Our results did not support a significant role of sustained attention and inhibition in relation to task performance. Our findings suggest that statistical learning is a general mechanism underlying Chinese character recognition.

Successful Implicit Vocabulary Intervention for Three Cantonese-Speaking Toddlers: A Replicated Single-Case Design.

Purpose We report on a replicated single-case design study that measured the feasibility of an expressive vocabulary intervention for three Cantonese-speaking toddlers with small expressive lexicons relative to their age. The aim was to assess the cross-cultural and cross-linguistic feasibility of an intervention method developed for English-speaking children. Method A nonconcurrent multiple-baseline design was used with four baseline data points and 16 intervention sessions per participant. The intervention design incorporated implicit learning principles, high treatment dosage, and control of the phonological neighborhood density of the stimuli. The children (24-39 months) attended 7-9 weeks of twice weekly input-based treatment in which no explicit verbal production was required from the child. Each target word was provided as input a minimum of 64 times in at least two intervention sessions. Treatment feasibility was measured by comparison of how many of the target and control words the child produced across the intervention period, and parent-reported expressive vocabulary checklists were completed for comparison of pre- and postintervention child spoken vocabulary size. An omnibus effect size for the treatment effect of the number of target and control words produced across time was calculated using Kendall's Tau. Results There was a significant treatment effect for target words learned in intervention relative to baselines, and all children produced significantly more target than control words across the intervention period. The effect of phonological neighborhood density on expressive word production could not be evaluated because two of the three children learned all target words. Conclusion The results provide cross-cultural evidence of the feasibility of a model of intervention that incorporated a high-dosage, cross-situational statistical learning paradigm to teach spoken word production to children with small expressive lexicons.

Different mechanisms underlie implicit visual statistical learning in honey bees and humans

The ability of developing complex internal representations of the environment is considered a crucial antecedent to the emergence of humans' higher cognitive functions. Yet it is an open question whether there is any fundamental difference in how humans and other good visual learner species naturally encode aspects of novel visual scenes. Using the same modified visual statistical learning paradigm and multielement stimuli, we investigated how human adults and honey bees (Apis mellifera) encode spontaneously, without dedicated training, various statistical properties of novel visual scenes. We found that, similarly to humans, honey bees automatically develop a complex internal representation of their visual environment that evolves with accumulation of new evidence even without a targeted reinforcement. In particular, with more experience, they shift from being sensitive to statistics of only elemental features of the scenes to relying on co-occurrence frequencies of elements while losing their sensitivity to elemental frequencies, but they never encode automatically the predictivity of elements. In contrast, humans involuntarily develop an internal representation that includes single-element and co-occurrence statistics, as well as information about the predictivity between elements. Importantly, capturing human visual learning results requires a probabilistic chunk-learning model, whereas a simple fragment-based memory-trace model that counts occurrence summary statistics is sufficient to replicate honey bees' learning behavior. Thus, humans' sophisticated encoding of sensory stimuli that provides intrinsic sensitivity to predictive information might be one of the fundamental prerequisites of developing higher cognitive abilities.

The Neuroscience of Implicit Learning

AbstractOver the past decades, research employing artificial grammar, sequence learning, and statistical learning paradigms has flourished, not least because these methods appear to offer a window, albeit with a restricted view, on implicit learning processes underlying natural language learning. But these paradigms usually provide relatively little exposure, use meaningless stimuli, and do not even necessarily target natural language structures. So the question arises whether they engage the same brain regions as natural language. The aim of this review is to use data from brain imaging, brain stimulation, and the effects of brain damage to identify the main brain regions that show sensitivity to structural regularities in implicit learning paradigms and to consider their relationship to natural language processing and learning.

Cross-situational statistical learning in younger and older adults

ABSTRACT Research investigating statistical learning, the process of tracking regularities in the environment, in older adults has been limited; with existing studies suggesting there are age-related declines. We aim to further understand older adults’ statistical learning abilities using a cross-situational statistical learning paradigm in which learners map novel words to novel objects. In Experiment 1, we manipulated task difficulty and found an overall age deficit but no interaction between age and difficulty. In Experiment 2, after extended practice with a first set of object-word mappings, learners could remap a subset of previously learned words to novel objects. Based on hyper-binding, older adults might be more willing to remap previously learned words to novel objects. However, despite overall poorer learning, older adults were actually less likely to remap. Even though older adults may have an associative memory deficit, learned associations are not more weakly bound for older relative to younger adults.

Perceptual Expectations Modulate Low-Frequency Activity: A Statistical Learning Magnetoencephalography Study.

Perceptual expectations can change how a visual stimulus is perceived. Recent studies have shown mixed results in terms of whether expectations modulate sensory representations. Here, we used a statistical learning paradigm to study the temporal characteristics of perceptual expectations. We presented participants with pairs of object images organized in a predictive manner and then recorded their brain activity with magnetoencephalography while they viewed expected and unexpected image pairs on the subsequent day. We observed stronger alpha-band (7-14 Hz) activity in response to unexpected compared with expected object images. Specifically, the alpha-band modulation occurred as early as the onset of the stimuli and was most pronounced in left occipito-temporal cortex. Given that the differential response to expected versus unexpected stimuli occurred in sensory regions early in time, our results suggest that expectations modulate perceptual decision-making by changing the sensory response elicited by the stimuli.

Thumbs up, thumbs down: non-verbal human-robot interaction through real-time EMG classification via inductive and supervised transductive transfer learning

In this study, we present a transfer learning method for gesture classification via an inductive and supervised transductive approach with an electromyographic dataset gathered via the Myo armband. A ternary gesture classification problem is presented by states of ’thumbs up’, ’thumbs down’, and ’relax’ in order to communicate in the affirmative or negative in a non-verbal fashion to a machine. Of the nine statistical learning paradigms benchmarked over 10-fold cross validation (with three methods of feature selection), an ensemble of Random Forest and Support Vector Machine through voting achieves the best score of 91.74% with a rule-based feature selection method. When new subjects are considered, this machine learning approach fails to generalise new data, and thus the processes of Inductive and Supervised Transductive Transfer Learning are introduced with a short calibration exercise (15 s). Failure of generalisation shows that 5 s of data per-class is the strongest for classification (versus one through seven seconds) with only an accuracy of 55%, but when a short 5 s per class calibration task is introduced via the suggested transfer method, a Random Forest can then classify unseen data from the calibrated subject at an accuracy of around 97%, outperforming the 83% accuracy boasted by the proprietary Myo system. Finally, a preliminary application is presented through social interaction with a humanoid Pepper robot, where the use of our approach and a most-common-class metaclassifier achieves 100% accuracy for all trials of a ‘20 Questions’ game.

Cross-Situational Statistical Learning of New Words Despite Bilateral Hippocampal Damage and Severe Amnesia.

Word learning requires learners to bind together arbitrarily-related phonological, visual, and conceptual information. Prior work suggests that this binding can be robustly achieved via incidental cross-situational statistical exposure to words and referents. When cross-situational statistical learning (CSSL) is tested in the laboratory, there is no information on any given trial to identify the referent of a novel word. However, by tracking which objects co-occur with each word across trials, learners may acquire mappings through statistical association. While CSSL behavior is well-characterized, its brain correlates are not. The arbitrary nature of CSSL mappings suggests hippocampal involvement, but the incremental, statistical nature of the learning raises the possibility of neocortical or procedural learning systems. Prior studies have shown that neurological patients with hippocampal pathology have word-learning impairments, but this has not been tested in a statistical learning paradigm. Here, we used a neuropsychological approach to test whether patients with bilateral hippocampal pathology (N = 3) could learn new words in a CSSL paradigm. In the task, patients and healthy comparison participants completed a CSSL word-learning task in which they acquired eight word/object mappings. During each trial of the CSSL task, participants saw two objects on a computer display, heard one novel word, and selected the most likely referent. Across trials, words were 100% likely to co-occur with their referent, but only 14.3% likely with non-referents. Two of three amnesic patients learned the associations between objects and word forms, although performance was impaired relative to healthy comparison participants. Our findings show that the hippocampus is not strictly necessary for CSSL for words, although it may facilitate such learning. This is consistent with a hybrid account of CSSL supported by implicit and explicit memory systems, and may have translational applications for remediation of (word-) learning deficits in neurological populations with hippocampal pathology.