Predictive Processing Research Articles

Invariant representations in abstract concept grounding - the physical world in grounded cognition.

Grounded cognition states that mental representations of concepts consist of experiential aspects. For example, the concept "cup" consists of the sensorimotor experiences from interactions with cups. Typical modalities in which concepts are grounded are: The sensorimotor system (including interoception), emotion, action, language, and social aspects. Here, we argue that this list should be expanded to include physical invariants (unchanging features of physical motion; e.g., gravity, momentum, friction). Research on physical reasoning consistently demonstrates that physical invariants are represented as fundamentally as other grounding substrates, and therefore should qualify. We assess several theories of concept representation (simulation, conceptual metaphor, conceptual spaces, predictive processing) and their positions on physical invariants. We find that the classic grounded cognition theories, simulation and conceptual metaphor theory, have not considered physical invariants, while conceptual spaces and predictive processing have. We conclude that physical invariants should be included into grounded cognition theories, and that the core mechanisms of simulation and conceptual metaphor theory are well suited to do this. Furthermore, conceptual spaces and predictive processing are very promising and should also be integrated with grounded cognition in the future.

Rapid purification and scale-up of tilianin using counter-current chromatography with rectangular horizontal tubing

In counter-current chromatography (CCC), linear scale-up is an ideal amplification strategy. However, when transferring from analytical to predictable preparative processes with high throughput, linear scale-up would be challenging due to limitations imposed by differences in instrument parameters, such as gravitational forces, tubing cross-section area, tubing length, column volume and flow rate. Some effective scale-up strategies have been studied for different instrument parameters, but so far, these scale-up works have only been tested on standard circular (SC) tubing.The previous research of our group found that rectangular horizontal (RH) tubing can double the separation efficiency compared with conventional SC tubing, and has industrial production potential. This paper used the separation of tilianin from Dracocephalum moldavica L. as an example to demonstrate how to scale up the optimized process from analytical SC tubing to preparative RH tubing. After systematic optimization of solvent systems, sample concentration and flow rate on the analytical CCC, the optimized parameters obtained were successfully transferred to the preparative CCC. The results showed that a crude sample of 2.07 g was successfully separated using a solvent system of n-hexane - ethyl acetate - ethanol - water (1:4:1:5, v/v/v/v) in reversed phase mode, and the three consecutive separations produced a total of 380 mg tilianin in 75 min with high purities of 98.3%, as analyzed by HPLC. The total throughput achieved from the analytical to semi-preparative scale was improved by 138 times (from 12 mg/h to 1.66 g/h), while the column volume was increased by only 46.5 times (from 15.5 mL to 720 mL). This is the successful application of CCC for the separation and purification of tilianin. Given that SC tubing is the traditional configuration for CCC columns, this study is a necessary step to prove the applicability of RH tubing columns for routine use and potential large-scale industrial applications.

Binocularly suppressed stimuli induce brain activities related to aesthetic emotions

IntroductionAesthetic emotions are a class of emotions aroused by evaluating aesthetically appealing objects or events. While evolutionary aesthetics suggests the adaptive roles of these emotions, empirical assessments are lacking. Previous neuroscientific studies have demonstrated that visual stimuli carrying evolutionarily important information induce neural responses even when presented non-consciously. To examine the evolutionary importance of aesthetic emotions, we conducted a neuroscientific study using magnetoencephalography (MEG) to measure induced neural responses to non-consciously presented portrait paintings categorised as biological and non-biological and examined associations between the induced responses and aesthetic ratings.MethodsMEG and pre-rating data were collected from 23 participants. The pre-rating included visual analogue scales for object saliency, facial saliency, liking, and beauty scores, in addition to ‘biologi-ness,’ which was used for subcategorising stimuli into biological and non-biological. The stimuli were presented non-consciously using a continuous flash suppression paradigm or consciously using binocular presentation without flashing masks, while dichotomic behavioural responses were obtained (beauty or non-beauty). Time-frequency decomposed MEG data were used for correlation analysis with pre-rating scores for each category.ResultsBehavioural data revealed that saliency scores of non-consciously presented stimuli influenced dichotomic responses (beauty or non-beauty). MEG data showed that non-consciously presented portrait paintings induced spatiotemporally distributed low-frequency brain activities associated with aesthetic ratings, which were distinct between the biological and non-biological categories and conscious and non-conscious conditions.ConclusionAesthetic emotion holds evolutionary significance for humans. Neural pathways are sensitive to visual images that arouse aesthetic emotion in distinct ways for biological and non-biological categories, which are further influenced by consciousness. These differences likely reflect the diversity in mechanisms of aesthetic processing, such as processing fluency, active elaboration, and predictive processing. The aesthetic processing of non-conscious stimuli appears to be characterised by fluency-driven affective processing, while top-down regulatory processes are suppressed. This study provides the first empirical evidence supporting the evolutionary significance of aesthetic processing.

An interpretable framework for high-precision flatness prediction in strip cold rolling

Flatness plays a crucial role in determining the quality of products in strip cold rolling. Data driven methods have shown promise in flatness prediction by effectively capturing the nonlinearities and strong coupling present in cold rolling processing, surpassing the capability of conventional methods. However, existing data driven models remain restricted by a lack of rolling theory guidance, a black-box nature of predictive processes, and gradient conflict of multi-channel flatness. To overcome these limitations, this paper proposes an interpretable mechanism guided multi-channel distributed meta learning framework for flatness prediction. Initially, significant physic-based parameters, such as theoretical rolling force deviation and tension deviation, and controller parameters are calculated to guide data driven modeling. Subsequently, a distributed meta learning framework is modeled for multi-channel flatness to eliminate gradient conflict. Furthermore, eXplainable Artificial Intelligence (XAI) technique is implemented to ensure the transparent predictive processes of multi-channel flatness. The analysis results present that theoretical parameters and controller parameters effectively improve the performance of flatness prediction. In addition, the comparative results demonstrate that the proposed framework outperforms the existing flatness prediction methods and other state-of-the-art machine learning methods by 4.24%. Importantly, the XAI-based explanation of the proposed framework effectively enhances the credibility of data driven flatness prediction.

Investigating the neural effects of typicality and predictability for face and object stimuli.

The brain calibrates itself based on the past stimulus diet, which makes frequently observed stimuli appear as typical (as opposed to uncommon stimuli, which appear as distinctive). Based on predictive processing theory, the brain should be more "prepared" for typical exemplars, because these contain information that has been encountered frequently, allowing it to economically represent items of that category. Thus, one could ask whether predictability and typicality of visual stimuli interact, or rather act in an additive manner. We adapted the design by Egner and colleagues (2010), who used cues to induce expectations about stimulus category (face vs. chair) occurrence during an orthogonal inversion detection task. We measured BOLD responses with fMRI in 35 participants. First, distinctive stimuli always elicited stronger responses than typical ones in all ROIs, and our whole-brain directional contrasts for the effects of typicality and distinctiveness converge with previous findings. Second and importantly, we could not replicate the interaction between category and predictability reported by Egner et al. (2010), which casts doubt on whether cueing designs are ideal to elicit reliable predictability effects. Third, likely as a consequence of the lack of predictability effects, we found no interaction between predictability and typicality in any of the four tested regions (bilateral fusiform face areas, lateral occipital complexes) when considering both categories, nor in the whole brain. We discuss the issue of replicability in neuroscience and sketch an agenda for how future studies might address the same question.

Predictive language processing: integrating comprehension and production, and what atypical populations can tell us.

Predictive processing, a crucial aspect of human cognition, is also relevant for language comprehension. In everyday situations, we exploit various sources of information to anticipate and therefore facilitate processing of upcoming linguistic input. In the literature, there are a variety of models that aim at accounting for such ability. One group of models propose a strict relationship between prediction and language production mechanisms. In this review, we first introduce very briefly the concept of predictive processing during language comprehension. Secondly, we focus on models that attribute a prominent role to language production and sensorimotor processing in language prediction ("prediction-by-production" models). Contextually, we provide a summary of studies that investigated the role of speech production and auditory perception on language comprehension/prediction tasks in healthy, typical participants. Then, we provide an overview of the limited existing literature on specific atypical/clinical populations that may represent suitable testing ground for such models-i.e., populations with impaired speech production and auditory perception mechanisms. Ultimately, we suggest a more widely and in-depth testing of prediction-by-production accounts, and the involvement of atypical populations both for model testing and as targets for possible novel speech/language treatment approaches.

Spatiotemporal brain hierarchies of auditory memory recognition and predictive coding

Our brain is constantly extracting, predicting, and recognising key spatiotemporal features of the physical world in order to survive. While neural processing of visuospatial patterns has been extensively studied, the hierarchical brain mechanisms underlying conscious recognition of auditory sequences and the associated prediction errors remain elusive. Using magnetoencephalography (MEG), we describe the brain functioning of 83 participants during recognition of previously memorised musical sequences and systematic variations. The results show feedforward connections originating from auditory cortices, and extending to the hippocampus, anterior cingulate gyrus, and medial cingulate gyrus. Simultaneously, we observe backward connections operating in the opposite direction. Throughout the sequences, the hippocampus and cingulate gyrus maintain the same hierarchical level, except for the final tone, where the cingulate gyrus assumes the top position within the hierarchy. The evoked responses of memorised sequences and variations engage the same hierarchical brain network but systematically differ in terms of temporal dynamics, strength, and polarity. Furthermore, induced-response analysis shows that alpha and beta power is stronger for the variations, while gamma power is enhanced for the memorised sequences. This study expands on the predictive coding theory by providing quantitative evidence of hierarchical brain mechanisms during conscious memory and predictive processing of auditory sequences.

Optimization and predictive modeling of membrane based produced water treatment using machine learning models

Membrane technology has become a crucial solution for treating produced water, playing a vital role in resource recovery and environmental preservation. The use of advanced Artificial Intelligence (AI) technique such as Machine Learning (ML) has introduced an innovative approach to define and model these treatment procedures such as trans-membrane pressure (TMP) and total membrane resistance. The research focuses specifically on the utilization of Artificial Neural Network (ANN) and Extreme Gradient Boosting (XGBoost) model to improve the precision and effectiveness of produced water treatment using membrane technology. Also, the study presented explores how the Artificial Neural Network (ANN) framework excels in predictive insights and process modeling, optimizing operational parameters in membrane-based treatment setups. Furthermore, integrating the Extreme Gradient Boosting (XGBoost) model highlights its ability to enhance system performance, providing accurate predictions and better control in complex treatment processes. It presents comprehensive research of combining Artificial Neural Network (ANN) and Extreme Gradient Boosting (XGBoost) model with membrane technology for produced water treatment, emphasizing their potential to improve optimization techniques and modeling approaches, ultimately enhancing water treatment efficiency and sustainability. Both the Artificial Neural Network (ANN) and Extreme Gradient Boosting (XGBoost) models achieved an impressive R-squared (R2) scores of 0.754, and 0.97 for Transmembrane Pressure (TMP), and 0.95 and 0.81 for Total Resistance prediction respectively.

Federated Variational Inference: Towards Improved Personalization and Generalization

Conventional federated learning algorithms train a single global model by leveraging all participating clients’ data. However, due to heterogeneity in client generative distributions and predictive models, these approaches may not appropriately approximate the predictive process, converge to an optimal state, or generalize to new clients. We study personalization and generalization in stateless cross-device federated learning setups assuming heterogeneity in client data distributions and predictive models. We first propose a hierarchical generative model and formalize it using Bayesian Inference. We then approximate this process using Variational Inference to train our model efficiently. We call this algorithm Federated Variational Inference (FedVI). We use PAC-Bayes analysis to provide generalization bounds for FedVI. We evaluate our model on FEMNIST and CIFAR-100 image classification and show that FedVI beats the state-of-the-art on both tasks.

Suppression and omission effects in auditory predictive processing-Two of the same?

Recent theories describe perception as an inferential process based on internal predictive models that are adjusted by prediction violations (prediction error). Two different modulations of the auditory N1 event-related brain potential component are often discussed as an expression of auditory predictive processing. The sound-related N1 component is attenuated for self-generated sounds compared to the N1 elicited by externally generated sounds (N1 suppression). An omission-related component in the N1 time-range is elicited when the self-generated sounds are occasionally omitted (omission N1). Both phenomena were explained by action-related forward modelling, which takes place when the sensory input is predictable: prediction error signals are reduced when predicted sensory input is presented (N1 suppression) and elicited when predicted sensory input is omitted (omission N1). This common theoretical account is appealing but has not yet been directly tested. We manipulated the predictability of a sound in a self-generation paradigm in which, in two conditions, either 80% or 50% of the button presses did generate a sound, inducing a strong or a weak expectation for the occurrence of the sound. Consistent with the forward modelling account, an omission N1 was observed in the 80% but not in the 50% condition. However, N1 suppression was highly similar in both conditions. Thus, our results demonstrate a clear effect of predictability for the omission N1 but not for the N1 suppression. These results imply that the two phenomena rely (at least in part) on different mechanisms and challenge prediction related accounts of N1 suppression.

Decoding predicted musical notes from omitted stimulus potentials

Electrophysiological studies have investigated predictive processing in music by examining event-related potentials (ERPs) elicited by the violation of musical expectations. While several studies have reported that the predictability of stimuli can modulate the amplitude of ERPs, it is unclear how specific the representation of the expected note is. The present study addressed this issue by recording the omitted stimulus potentials (OSPs) to avoid contamination of bottom-up sensory processing with top-down predictive processing. Decoding of the omitted content was attempted using a support vector machine, which is a type of machine learning. ERP responses to the omission of four target notes (E, F, A, and C) at the same position in familiar and unfamiliar melodies were recorded from 25 participants. The results showed that the omission N1 were larger in the familiar melody condition than in the unfamiliar melody condition. The decoding accuracy of the four omitted notes was significantly higher in the familiar melody condition than in the unfamiliar melody condition. These results suggest that the OSPs contain discriminable predictive information, and the higher the predictability, the more the specific representation of the expected note is generated.

Bolstering the adaptive information processing model: a narrative review.

In recent years, several theoretical models have been suggested as complementary to the adaptative information processing model of eye movement desensitization and reprocessing therapy. A narrative review of such models was conducted to assess the contributions of each, as well as their convergences, contradictions, and potential complementarity. Seven theoretical models were identified. All focus on the effects of EMDR therapy as a comprehensive psychotherapy approach with its principles, procedures, and protocols. Several refer to concepts related to propositional or predictive processing theories. Overall, the contribution of these proposals does appear to bolster Shapiro's original AIP model, potentially offering additional depth and breadth to case conceptualization and treatment planning in clinical practice, as well as a more precise theoretical understanding. The current exploratory comparative analysis may serve as a preliminary baseline to guide research into the relative merit of suggested theoretical proposals to enhance current standards for the clinical practice and teaching of EMDR therapy.

Systematic review and meta-analysis of the visual mismatch negativity in schizophrenia.

Mismatch negativity (MMN) is an event-related potential component automatically elicited by events that violate predictions based on prior events. To elicit this component, researchers use stimulus repetition to induce predictions, and the MMN is obtained by subtracting the brain response to rare or unpredicted stimuli from that of frequent stimuli. Under the Predictive Processing framework, one increasingly popular interpretation of the mismatch response postulates that MMN represents a prediction error. In this context, the reduced MMN amplitude to auditory stimuli has been considered a potential biomarker of Schizophrenia, representing a reduced prediction error and the inability to update the mental model of the world based on the sensory signals. It is unclear, however, whether this amplitude reduction is specific for auditory events or if the visual MMN reveals a similar pattern in schizophrenia spectrum disorder. This review and meta-analysis aimed to summarise the available literature on the vMMN in schizophrenia. A systematic literature search resulted in 10 eligible studies that resulted in a combined effect size of g = -.63, CI [-.86, -.41], reflecting lower vMMN amplitudes in patients. These results are in line with the findings in the auditory domain. This component offers certain advantages, such as less susceptibility to overlap with components generated by attentional demands. Future studies should use vMMN to explore abnormalities in the Predictive Processing framework in different stages and groups of the SSD and increase the knowledge in the search for biomarkers in schizophrenia.

Energy-information trade-off induces continuous and discontinuous phase transitions in lateral predictive coding

Lateral predictive coding is a recurrent neural network that creates energy-efficient internal representations by exploiting statistical regularity in sensory inputs. Here, we analytically investigate the trade-off between information robustness and energy in a linear model of lateral predictive coding and numerically minimize a free energy quantity. We observed several phase transitions in the synaptic weight matrix, particularly a continuous transition that breaks reciprocity and permutation symmetry and builds cyclic dominance and a discontinuous transition with the associated sudden emergence of tight balance between excitatory and inhibitory interactions. The optimal network follows an ideal gas law over an extended temperature range and saturates the efficiency upper bound of energy use. These results provide theoretical insights into the emergence and evolution of complex internal models in predictive processing systems.

Framing the predictive mind: why we should think again about Dreyfus

AbstractIn this paper I return to Hubert Dreyfus’ old but influential critique of artificial intelligence, redirecting it towards contemporary predictive processing models of the mind (PP). I focus on Dreyfus’ arguments about the “frame problem” for artificial cognitive systems, and his contrasting account of embodied human skills and expertise. The frame problem presents as a prima facie problem for practical work in AI and robotics, but also for computational views of the mind in general, including for PP. Indeed, some of the issues it presents seem more acute for PP, insofar as it seeks to unify all cognition and intelligence, and aims to do so without admitting any cognitive processes or mechanisms outside of the scope of the theory. I contend, however, that there is an unresolved problem for PP concerning whether it can both explain all cognition and intelligent behavior as minimizing prediction error with just the core formal elements of the PP toolbox, and also adequately comprehend (or explain away) some of the apparent cognitive differences between biological and prediction-based artificial intelligence, notably in regard to establishing relevance and flexible context-switching, precisely the features of interest to Dreyfus’ work on embodied indexicality, habits/skills, and abductive inference. I address several influential philosophical versions of PP, including the work of Jakob Hohwy and Andy Clark, as well as more enactive-oriented interpretations of active inference coming from a broadly Fristonian perspective.

An adversarial environment reinforcement learning-driven intrusion detection algorithm for Internet of Things

The increasing prevalence of Internet of Things (IoT) systems has made them attractive targets for malicious actors. To address the evolving threats and the growing complexity of detection, there is a critical need to search for and develop new algorithms that are fast and robust in detecting and classifying dangerous network traffic. In this context, deep reinforcement learning (DRL) is gaining recognition as a prospective solution in numerous fields as it enables autonomous agents to cooperate with their environment for decision-making without relying on human experts. This article presents an innovative approach to intrusion detection in IoT systems using an adversarial reinforcement learning (RL) algorithm known for its exceptional predictive capabilities. The predictive process relies on a classifier, implemented as a streamlined and highly efficient neural network. Embedded within this classifier is a policy function meticulously trained using an innovative RL model. Importantly, this model ensures that the environment’s behavior is dynamically fine-tuned simultaneously with the learning process, improving the overall effectiveness of the intrusion detection approach. The efficiency of our proposal was assessed using the Bot-IoT database, consisting of a mixture of legitimate IoT network traffic and simulated attack scenarios. Our scheme shows superior performance compared to existing ones. Therefore, our approach to IoT intrusion detection can be considered a valuable alternative to existing methods, capable of significantly improving the IoT systems’ security.

Expectation Modulates Repetition Suppression at Late But Not Early Stages during Visual Word Recognition: Evidence from Event-related Potentials.

Visual word recognition is commonly rapid and efficient, incorporating top-down predictive processing mechanisms. Neuroimaging studies with face stimuli suggest that repetition suppression (RS) reflects predictive processing at the neural level, as this effect is larger when repetitions are more frequent, that is, more expected. It remains unclear, however, at the temporal level whether and how RS and its modulation by expectation occur in visual word recognition. To address this gap, the present study aimed to investigate the presence and time course of these effects during visual word recognition using EEG. Thirty-six native Cantonese speakers were presented with pairs of Chinese written words and performed a nonlinguistic oddball task. The second word of a pair was either a repetition of the first or a different word (alternation). In repetition blocks, 75% of trials were repetitions and 25% were alternations, whereas the reverse was true in alternation blocks. Topographic analysis of variance of EEG at each time point showed robust RS effects in three time windows (141-227 msec, 242-445 msec, and 467-513 msec) reflecting facilitation of visual word recognition. Importantly, the modulation of RS by expectation was observed at the late rather than early intervals (334-387 msec, 465-550 msec, and 559-632 msec) and more than 100msec after the first RS effects. In the predictive coding view of RS, only late repetition effects are modulated by expectation, whereas early RS effects may be mediated by lower-level predictions. Taken together, our findings provide the first EEG evidence revealing distinct temporal dynamics of RS effects and repetition probability on RS effects in visual processing of Chinese words.

Predictive processing: Layer-specific prediction error signals in human cortex

A new study leveraging advances in high-field fMRI provides evidence that superficial cortical layers in humans play a crucial role in signaling prediction errors, a finding that is consistent with the predictive processing framework.

Machine Learning Exploration of Bank Marketing Data with Apache Spark

Abstract: Banks use the advanced analytics provided by Apache Spark to improve customer service and optimize marketing. By integrating machine learning, insights into consumer behavior can be gained through predictive modeling and efficient data processing. The main topics of this study are customer segmentation, predictive modeling and personalized marketing. PySpark's user-friendly interface and Spark's scalability support tactics related to growth, customer acquisition, and retention.

Effect of contextual prediction on emotional word processing: an evidence from ERPNR-D-24-00189.

This study examined the effect of context on the prediction of emotional words with varying valences. It investigated the neural mechanisms underlying the processing differences of emotion words with different valences in both predictable and unpredictable contexts. Additionally, it aimed to address the conflicting results regarding the processing time in predictive contexts reported in previous studies. Participants were instructed to carefully read the text that included the specified emotion words. Event-related potentials elicited by emotional words were measured. To ensure that the participants can read the text carefully, 33% of the texts are followed by comprehension problems. After reading the text, the comprehension questions were answered based on the text content. The study revealed that the N400 amplitude elicited by an unpredictable context was greater than that elicited by a predictable context. Additionally, the N400 amplitude triggered by positive emotion words was larger than that triggered by negative emotion words. However, there was no significant difference in late positive component amplitude observed between contextual prediction and emotional word valence. The present study suggests that predictive processing takes place at an intermediate stage of speech processing, approximately 400 ms after stimulus onset. Furthermore, the presence of a predictive context enhances the processing of emotional information. Notably, brain activity is more pronounced during the processing of positive emotional stimuli compared to negative emotional stimuli. Additionally, the facilitative effect of a predictable context diminishes in the advanced phase of Chinese speech comprehension.