Computational Learning Model Research Articles

Role of NMDAR plasticity in a computational model of synaptic memory

A largely unexplored question in neuronal plasticity is whether synapses are capable of encoding and learning the timing of synaptic inputs. We address this question in a computational model of synaptic input time difference learning (SITDL), where N‐methyl‐d‐aspartate receptor (NMDAR) isoform expression in silent synapses is affected by time differences between glutamate and voltage signals. We suggest that differences between NMDARs’ glutamate and voltage gate conductances induce modifications of the synapse’s NMDAR isoform population, consequently changing the timing of synaptic response. NMDAR expression at individual synapses can encode the precise time difference between signals. Thus, SITDL enables the learning and reconstruction of signals across multiple synapses of a single neuron. In addition to plausibly predicting the roles of NMDARs in synaptic plasticity, SITDL can be usefully applied in artificial neural network models.

Modeling incentive salience in Pavlovian learning more parsimoniously using a multiple attribute model.

We present a multi-attribute incentive salience (MAIS) model as a computational account of incentive salience in model-based Pavlovian learning. A model of incentive salience as a joint function of reward value and physiological state has been previously proposed by Zhang et al. (2009). In that model, the function takes additive or multiplicative forms depending on whether a preference shifts from positive to negative or vice versa. We demonstrate that arbitrarily varying this function is unnecessary to explain observed data. A multiplicative function is sufficient if one takes into account empirical data suggesting the incentive salience function for an incentive is comprised of multiple physiological signals. We compare our model to the previously proposed model on two datasets. We find the MAIS model predicts the outcomes equally well, fits empirical data describing multiple sensory representations of a single stimulus, better approximates the dual-structure appetitive-aversive nature of the reward system, is compatible with existing knowledge about incentive salience in Pavlovian learning, and better describes revaluation in Pavlovian learning. This model addresses a call (Dayan & Berridge, 2014) for algorithmic and computational models of model-based Pavlovian learning that consistently and fully explain empirical observations. Because a multi-attribute model is relevant even for simple Pavlovian associations, it should be useful in a wide variety of decision-making contexts, including agent modeling and addiction research.

Supplemental Material for A Computational Model of Infant Learning and Reasoning With Probabilities

Supplemental Material for A Computational Model of Infant Learning and Reasoning With Probabilities

How children learn to communicate discriminatively.

How do children learn to communicate, and what do they learn? Traditionally, most theories have taken an associative, compositional approach to these questions, supposing children acquire an inventory of form-meaning associations, and procedures for composing / decomposing them; into / from messages in production and comprehension. This paper presents an alternative account of human communication and its acquisition based on the systematic, discriminative approach embodied in psychological and computational models of learning, and formally described by communication theory. It describes how discriminative learning theory offers an alternative perspective on the way that systems of semantic cues are conditioned onto communicative codes, while information theory provides a very different view of the nature of the codes themselves. It shows how the distributional properties of languages satisfy the communicative requirements described in information theory, enabling language learners to align their expectations despite the vastly different levels of experience among language users, and to master communication systems far more abstract than linguistic intuitions traditionally assume. Topics reviewed include morphological development, the acquisition of verb argument structures, and the functions of linguistic systems that have proven to be stumbling blocks for compositional theories: grammatical gender and personal names.

Ageing is associated with disrupted reinforcement learning whilst learning to help others is preserved

Reinforcement learning is a fundamental mechanism displayed by many species. However, adaptive behaviour depends not only on learning about actions and outcomes that affect ourselves, but also those that affect others. Using computational reinforcement learning models, we tested whether young (age 18–36) and older (age 60–80, total n = 152) adults learn to gain rewards for themselves, another person (prosocial), or neither individual (control). Detailed model comparison showed that a model with separate learning rates for each recipient best explained behaviour. Young adults learned faster when their actions benefitted themselves, compared to others. Compared to young adults, older adults showed reduced self-relevant learning rates but preserved prosocial learning. Moreover, levels of subclinical self-reported psychopathic traits (including lack of concern for others) were lower in older adults and the core affective-interpersonal component of this measure negatively correlated with prosocial learning. These findings suggest learning to benefit others is preserved across the lifespan with implications for reinforcement learning and theories of healthy ageing.

Modeling changes in probabilistic reinforcement learning during adolescence.

In the real world, many relationships between events are uncertain and probabilistic. Uncertainty is also likely to be a more common feature of daily experience for youth because they have less experience to draw from than adults. Some studies suggest probabilistic learning may be inefficient in youths compared to adults, while others suggest it may be more efficient in youths in mid adolescence. Here we used a probabilistic reinforcement learning task to test how youth age 8-17 (N = 187) and adults age 18-30 (N = 110) learn about stable probabilistic contingencies. Performance increased with age through early-twenties, then stabilized. Using hierarchical Bayesian methods to fit computational reinforcement learning models, we show that all participants’ performance was better explained by models in which negative outcomes had minimal to no impact on learning. The performance increase over age was driven by 1) an increase in learning rate (i.e. decrease in integration time scale); 2) a decrease in noisy/exploratory choices. In mid-adolescence age 13-15, salivary testosterone and learning rate were positively related. We discuss our findings in the context of other studies and hypotheses about adolescent brain development.

Learning how to behave: cognitive learning processes account for asymmetries in adaptation to social norms.

Changes to social settings caused by migration, cultural change or pandemics force us to adapt to new social norms. Social norms provide groups of individuals with behavioural prescriptions and therefore can be inferred by observing their behaviour. This work aims to examine how cognitive learning processes affect adaptation and learning of new social norms. Using a multiplayer game, I found that participants initially complied with various social norms exhibited by the behaviour of bot-players. After gaining experience with one norm, adaptation to a new norm was observed in all cases but one, where an active-harm norm was resistant to adaptation. Using computational learning models, I found that active behaviours were learned faster than omissions, and harmful behaviours were more readily attributed to all group members than beneficial behaviours. These results provide a cognitive foundation for learning and adaptation to descriptive norms and can inform future investigations of group-level learning and cross-cultural adaptation.

Sequence learning recodes cortical representations instead of strengthening initial ones.

We contrast two computational models of sequence learning. The associative learner posits that learning proceeds by strengthening existing association weights. Alternatively, recoding posits that learning creates new and more efficient representations of the learned sequences. Importantly, both models propose that humans act as optimal learners but capture different statistics of the stimuli in their internal model. Furthermore, these models make dissociable predictions as to how learning changes the neural representation of sequences. We tested these predictions by using fMRI to extract neural activity patterns from the dorsal visual processing stream during a sequence recall task. We observed that only the recoding account can explain the similarity of neural activity patterns, suggesting that participants recode the learned sequences using chunks. We show that associative learning can theoretically store only very limited number of overlapping sequences, such as common in ecological working memory tasks, and hence an efficient learner should recode initial sequence representations.

Decision Support Systems in Temporomandibular Joint Osteoarthritis: A review of Data Science and Artificial Intelligence Applications

With the exponential growth of computational systems and increased patient data acquisition, dental research faces new challenges to manage a large quantity of information. For this reason, data science approaches are needed for the integrative diagnosis of multifactorial diseases, such as Temporomandibular joint (TMJ) Osteoarthritis (OA). The Data science spectrum includes data capture/acquisition, data processing with optimized web-based storage and management, data analytics involving in-depth statistical analysis, machine learning (ML) approaches, and data communication. Artificial intelligence (AI) plays a crucial role in this process. It consists of developing computational systems that can perform human intelligence tasks, such as disease diagnosis, using many features to help in the decision-making support. Patient's clinical parameters, imaging exams, and molecular data are used as the input in cross-validation tasks, and human annotation/diagnosis is also used as the gold standard to train computational learning models and automatic disease classifiers. This paper aims to review and describe AI and ML techniques to diagnose TMJ OA and data science approaches for imaging processing. We used a web-based system for multi-center data communication, algorithms integration, statistics deployment, and process the computational machine learning models. We successfully show AI and data-science applications using patients' data to improve the TMJ OA diagnosis decision-making towards personalized medicine.

Minimal interplay between explicit knowledge, dynamics of learning and temporal expectations in different, complex uni- and multisensory contexts

While temporal expectations (TE) generally improve reactions to temporally predictable events, it remains unknown how the learning of temporal regularities (one time point more likely than another time point) and explicit knowledge about temporal regularities contribute to performance improvements; and whether any contributions generalise across modalities. Here, participants discriminated the frequency of diverging auditory, visual or audio-visual targets embedded in auditory, visual or audio-visual distractor sequences. Temporal regularities were manipulated run-wise (early vs. late target within sequence). Behavioural performance (accuracy, RT) plus measures from a computational learning model all suggest that learning of temporal regularities occurred but did not generalise across modalities, and that dynamics of learning (size of TE effect across runs) and explicit knowledge have little to no effect on the strength of TE. Remarkably, explicit knowledge affects performance—if at all—in a context-dependent manner: Only under complex task regimes (here, unknown target modality) might it partially help to resolve response conflict while it is lowering performance in less complex environments.

A small-scale robotic manipulandum for motor control study with rodents

Computational models of motor learning have been examined experimentally with human participants using a robotic manipulandum that produces force perturbations on their reach trajectories. Here, we developed a small-scale robotic manipulandum for rodent studies. This device enabled us to conduct the same motor control experiments typically used for human study on rodents in order to examine the neural basis of the computational process of motor control and learning. We present the design of the manipulandum and controller system in addition to kinematic analysis of the link structures. We also confirmed the feasibility of the developed system for rodent studies.

Interventional Radiology ex-machina: impact of Artificial Intelligence on practice.

Artificial intelligence (AI) is a branch of Informatics that uses algorithms to tirelessly process data, understand its meaning and provide the desired outcome, continuously redefining its logic. AI was mainly introduced via artificial neural networks, developed in the early 1950s, and with its evolution into "computational learning models." Machine Learning analyzes and extracts features in larger data after exposure to examples; Deep Learning uses neural networks in order to extract meaningful patterns from imaging data, even deciphering that which would otherwise be beyond human perception. Thus, AI has the potential to revolutionize the healthcare systems and clinical practice of doctors all over the world. This is especially true for radiologists, who are integral to diagnostic medicine, helping to customize treatments and triage resources with maximum effectiveness. Related in spirit to Artificial intelligence are Augmented Reality, mixed reality, or Virtual Reality, which are able to enhance accuracy of minimally invasive treatments in image guided therapies by Interventional Radiologists. The potential applications of AI in IR go beyond computer vision and diagnosis, to include screening and modeling of patient selection, predictive tools for treatment planning and navigation, and training tools. Although no new technology is widely embraced, AI may provide opportunities to enhance radiology service and improve patient care, if studied, validated, and applied appropriately.

Reward and fictive prediction error signals in ventral striatum: asymmetry between factual and counterfactual processing.

Reward prediction error, the difference between the expected and obtained reward, is known to act as a reinforcement learning neural signal. In the current study, we propose a model fitting approach that combines behavioral and neural data to fit computational models of reinforcement learning. Briefly, we penalized subject-specific fitted parameters that moved away too far from the group median, except when that deviation led to an improvement in the model's fit to neural responses. By means of a probabilistic monetary learning task and fMRI, we compared our approach with standard model fitting methods. Q-learning outperformed actor-critic at both behavioral and neural level, although the inclusion of neuroimaging data into model fitting improved the fit of actor-critic models. We observed both action-value and state-value prediction error signals in the striatum, while standard model fitting approaches failed to capture state-value signals. Finally, left ventral striatum correlated with reward prediction error while right ventral striatum with fictive prediction error, suggesting a functional hemispheric asymmetry regarding prediction-error driven learning.

Caregivers use gesture contingently to support word learning.

Children learn words in environments where there is considerable variability, both in terms of the number of possible referents for novel words, and the availability of cues to support word-referent mappings. How caregivers adapt their gestural cues to referential uncertainty has not yet been explored. We tested a computational model of cross-situational word learning that examined the value of a variable gesture cue during training across conditions of varying referential uncertainty. We found that gesture had a greater benefit for referential uncertainty, but unexpectedly also found that learning was best when there was variability in both the environment (number of referents) and gestural cue use. We demonstrated that these results are reflected behaviourally in an experimental word-learning study involving children aged 18-24-month-olds and their caregivers. Under similar conditions to the computational model, caregivers not only used gesture more when there were more potential referents for novel words, but children also learned best when there was some referential ambiguity for words. Thus, caregivers are sensitive to referential uncertainty in the environment and adapt their gestures accordingly, and children are able to respond to environmental variability to learn more robustly. These results imply that training under variable circumstances may actually benefit learning, rather than hinder it.

Variety-seeking, learning and performance

According to the variance hypothesis, variety-seeking or exploration is a critical condition for improving learning and performance over time. Extant computational learning models support this hypothesis by showing how individuals who are exposed to diverse knowledge sources are more likely to find superior solutions to a particular problem. Yet this research provides no precise guidelines about how broadly individuals should search. Our goal in this paper is to elucidate the conditions under which variety-seeking in organizations is beneficial. To this end, we developed a computational model in which individuals learn as they interact with other individuals, and update their knowledge as a result of this interaction. The model reveals how the type of learning environment (performance landscape) in which the learning dynamic unfolds determines when the benefits of variety-seeking outweigh the costs. Variety-seeking is performance-enhancing only when the knowledge of the chosen learning targets (i.e., individuals to learn from) provide useful information about the features of the performance landscape. The results further suggest that superior knowledge might be available locally, i.e., in the proximity of an individual’s current location. We also identify the point beyond which variety-seeking causes a sharp performance decline and show how this point depends on the type of landscape in which the learning dynamic unfolds and the degree of specialization of individual knowledge. The presence of this critical point explains why exploration becomes very costly. The implications of our findings for establishing the boundaries of exploration are discussed.

MLMD: Metric Learning for Predicting MiRNA-Disease Associations

The crucial roles played by microRNAs (miRNAs) in regulating various biological functions and in disease incidence have been reported continuously over the past decades. Therefore, the identification of novel disease-related miRNAs could help in understanding human disease etiology and pathogenesis further. Due to the involvement of high cost and more time in clinical experiments, development of accurate and feasible computational models is considered highly significant. Here, we aim to present a novel computational model of metric learning for predicting miRNA-disease associations (MLMD). MLMD aims at learning miRNA-disease metric to unravel not only novel disease-related miRNAs but also the miRNA-miRNA and disease-disease similarities. Comprehensive experimental results clearly proved the outstanding performance of MLMD compared to several state-of-the-art methods. MLMD achieved a reliable AUC score of 0.9106 and 0.8786 in the framework of global and local leave-one-out cross validations (LOOCV), respectively. Furthermore, we implemented case studies on two major human cancers (breast cancer and lung cancer) for comparative analysis with already known disease-related miRNAs. Results revealed the top 50 potential candidates were all disease-related miRNAs based on human public databases and literature analysis. We conclude that MLMD could not only serve as practical and feasible framework for inferring potential miRNA-disease associations, but also provide clues for understanding the human complex diseases.

Implementation of machine learning model-based decision support system for healthcare professionals to predict T2DM risk using heart rate variability features

Non-invasive early diabetes prediction has been gaining much premarkable over the last decade. Heart rate variability (HRV) is the only non-invasive technique that can predict the future occurrence of the disease. Early prediction of diabetes can help doctors start an early intervention. To this end, the authors have developed a computational machine learning model to predict type 2 diabetes mellitus (T2DM) risk using heart rate variability features and have evaluated its robustness against the HRV of 50 patients data. The electrocardiogram (ECG) signal of the control population (n=40) and T2DM population (n=120) have been recorded in the supine position for 5 minutes, and HRV signals have been obtained. The time domain, frequency domain, and non-linear features have been extracted from the HRV signal. A decision support system has been developed based on a machine learning algorithm. Finally, the decision support system has been validated using the HRV features of 50 patients (Control n=10 and T2DM n=40). HRV features are selected for the prediction of T2DM. The decision support system has been designed using three machine learning models: Gradient boosting decision tree (GBDT), Extreme Gradient boosting (XGBoost), Categorical boosting (CatBoost), and their performance have been evaluated based on the Accuracy (ACC), Sensitivity (SEN), Specificity (SPC), Positive predicted value (PPV), Negative predicted value (NPV), False-positive rate (FPR), False-negative rate (FNR), F1 score, and Area under the receiver operating characteristic curve (AUC) metrics. The CatBoost model offers the best performance outcomes, and its results have been validated on 50 patients. Thus the CatBoost model can be use as a decision support system in hospitals to predict the risk of T2DM.

Computational Learning Model for Prediction of Heart Disease Using Machine Learning Based on a New Regularizer.

Heart diseases are characterized as heterogeneous diseases comprising multiple subtypes. Early diagnosis and prognosis of heart disease are essential to facilitate the clinical management of patients. In this research, a new computational model for predicting early heart disease is proposed. The predictive model is embedded in a new regularization based on decaying the weights according to the weight matrices' standard deviation and comparing the results against its parents (RSD-ANN). The performance of RSD-ANN is far better than that of the existing methods. Based on our experiments, the average validation accuracy computed was 96.30% using either the tenfold cross-validation or holdout method.

Интеллектуальный робастный регулятор на технологиях когнитивных вычислений. Ч. 1: Модели когнитивного управления с эмоциональным обучением мозга

n on-linecontrol and decision-making systems, emotional brain training is a preferred methodology (compared to stochastic gradient-based and evolutionary algorithms) due to its low computational complexity and fast robust learning. To describe the emotional learning of the brain, a mathematical model was created —the brain emotional learning controller (BELC). The design of intelligent systems based on emotional signals basedoncontrol methods assoft computing technologies: artificial neural networks, fuzzy control and genetic algorithms. Based on the simulated mathematical model of mammals BEL, a controller architecture has been developed. Applied approachcalled “Brain Emotional Learning Based Intelligent Controller” (BELBIC) —a neurobiologically motivated intelligent controller based on a computational model of emotional learning in the mammalian limbic system. The article describes applied models of intelligent regulators based on emotional learning of the brain. BELBIC's learning capabilities;versatility and low computational complexity make it a very promising toolkitfor on-lineapplications.

Dynamic social learning in temporally and spatially variable environments.

Cultural evolution is partly driven by the strategies individuals use to learn behaviour from others. Previous experiments on strategic learning let groups of participants engage in repeated rounds of a learning task and analysed how choices are affected by individual payoffs and the choices of group members. While groups in such experiments are fixed, natural populations are dynamic, characterized by overlapping generations, frequent migrations and different levels of experience. We present a preregistered laboratory experiment with 237 mostly German participants including migration, differences in expertise and both spatial and temporal variation in optimal behaviour. We used simulation and multi-level computational learning models including time-varying parameters to investigate adaptive time dynamics in learning. Confirming theoretical predictions, individuals relied more on (conformist) social learning after spatial compared with temporal changes. After both types of change, they biased decisions towards more experienced group members. While rates of social learning rapidly declined in rounds following migration, individuals remained conformist to group-typical behaviour. These learning dynamics can be explained as adaptive responses to different informational environments. Summarizing, we provide empirical insights and introduce modelling tools that hopefully can be applied to dynamic social learning in other systems.