Computational Learning Model Research Articles

Adaptive control of synaptic plasticity integrates micro- and macroscopic network function.

Synaptic plasticity configures interactions between neurons and is therefore likely to be a primary driver of behavioral learning and development. How this microscopic-macroscopic interaction occurs is poorly understood, as researchers frequently examine models within particular ranges of abstraction and scale. Computational neuroscience and machine learning models offer theoretically powerful analyses of plasticity in neural networks, but results are often siloed and only coarsely linked to biology. In this review, we examine connections between these areas, asking how network computations change as a function of diverse features of plasticity and vice versa. We review how plasticity can be controlled at synapses by calcium dynamics and neuromodulatory signals, the manifestation of these changes in networks, and their impacts in specialized circuits. We conclude that metaplasticity-defined broadly as the adaptive control of plasticity-forges connections across scales by governing what groups of synapses can and can't learn about, when, and to what ends. The metaplasticity we discuss acts by co-opting Hebbian mechanisms, shifting network properties, and routing activity within and across brain systems. Asking how these operations can go awry should also be useful for understanding pathology, which we address in the context of autism, schizophrenia and Parkinson's disease.

Optimizing the human learnability of abstract network representations

Precisely how humans process relational patterns of information in knowledge, language, music, and society is not well understood. Prior work in the field of statistical learning has demonstrated that humans process such information by building internal models of the underlying network structure. However, these mental maps are often inaccurate due to limitations in human information processing. The existence of such limitations raises clear questions: Given a target network that one wishes for a human to learn, what network should one present to the human? Should one simply present the target network as-is, or should one emphasize certain parts of the network to proactively mitigate expected errors in learning? To investigate these questions, we study the optimization of network learnability in a computational model of human learning. Evaluating an array of synthetic and real-world networks, we find that learnability is enhanced by reinforcing connections within modules or clusters. In contrast, when networks contain significant core-periphery structure, we find that learnability is best optimized by reinforcing peripheral edges between low-degree nodes. Overall, our findings suggest that the accuracy of human network learning can be systematically enhanced by targeted emphasis and de-emphasis of prescribed sectors of information.

Heuristic Method for Minimizing Model Size of CNN by Combining Multiple Pruning Techniques.

Network pruning techniques have been widely used for compressing computational and memory intensive deep learning models through removing redundant components of the model. According to the pruning granularity, network pruning can be categorized into structured and unstructured methods. The structured pruning removes the large components in a model such as channels or layers, which might reduce the accuracy. The unstructured pruning directly removes mainly the parameters in a model as well as the redundant channels or layers, which might result in an inadequate pruning. To address the limitations of the pruning methods, this paper proposes a heuristic method for minimizing model size. This paper implements an algorithm to combine both the structured and the unstructured pruning methods while maintaining the target accuracy that is configured by its application. We use network slimming for the structured pruning method and deep compression for the unstructured one. Our method achieves a higher compression ratio than the case when the individual pruning method is applied. To show the effectiveness of our proposed method, this paper evaluates our proposed method with actual state-of-the-art CNN models of VGGNet, ResNet and DenseNet under the CIFAR-10 dataset. This paper discusses the performance of the proposed method with the cases of individual usage of the structured and unstructured pruning methods and then proves that our method achieves better performance with higher compression ratio. In the best case of the VGGNet, our method results in a 13× reduction ratio in the model size, and also gives a 15× reduction ratio regarding the pruning time compared with the brute-force search method.

The Dual Challenge of Search and Coordination for Organizational Adaptation: How Structures of Influence Matter

Organizations increasingly need to adapt to challenges in which search and coordination cannot be decoupled. In response, many have experimented with “agile” and “flat” designs that dismantle traditional forms of hierarchy to harness the distributed knowledge of specialized individuals. Despite the popularity of such practices, there is considerable variation in their implementation as well as conceptual ambiguity about the underlying premise. Does effective rapid experimentation necessarily imply the repudiation of hierarchical structures of influence? We use computational models of multiagent reinforcement learning to study the effectiveness of coordinated search in groups that vary in how they influence each other’s beliefs. We compare the behavior of flat and hierarchical teams with a baseline structure without any influence on beliefs (a “crowd”) when all three are placed in the same task environments. We find that influence on beliefs—whether it is hierarchical or not—makes it less likely that agents stabilize prematurely around their own experiences. However, flat teams can engage in excessive exploration, finding it difficult to converge on good alternatives, whereas hierarchical influence on beliefs reduces simultaneous uncoordinated exploration, introducing a degree of rapid exploitation. As a result, teams that need to achieve agility (i.e., rapid satisfactory results) in environments that require coordinated search may benefit from a hierarchical structure of influence—even when the apex actor has no superior knowledge, foresight, or capacity to control subordinates’ actions.Funding: Puranam thanks the Desmairis Fund at INSEAD for supporting the Organizations and Algorithms project.Supplemental Material: The online appendix is available at https://doi.org/10.1287/orsc.2022.1601 .

Learning from Ingroup Experiences Changes Intergroup Impressions.

Humans form impressions toward individuals of their own social groups (ingroup members) and of different social groups (outgroup members). Outgroup-focused theories predict that intergroup impressions are mainly shaped by experiences with outgroup individuals, while ingroup-focused theories predict that ingroup experiences play a dominant role. Here we test predictions from these two psychological theories by estimating how intergroup impressions are dynamically shaped when people learn from both ingroup and outgroup experiences. While undergoing fMRI, male participants had identical experiences with different ingroup or outgroup members and rated their social closeness and impressions toward the ingroup and the outgroup. Behavioral results showed an initial ingroup bias in impression ratings which was significantly reduced over the course of learning, with larger effects in individuals with stronger ingroup identification. Computational learning models revealed that these changes in intergroup impressions were predicted by the weight given to ingroup prediction errors. Neurally, the individual weight for ingroup prediction errors was related to the coupling between the left inferior parietal lobule and the left anterior insula, which, in turn, predicted learning-related changes in intergroup impressions. Our findings provide computational and neural evidence for ingroup-focused theories, highlighting the importance of ingroup experiences in shaping social impressions in intergroup settings.SIGNIFICANCE STATEMENT Living in multicultural societies, humans interact with individuals of their own social groups (ingroup members) and of different social groups (outgroup members). However, little is known about how people learn from the mixture of ingroup and outgroup interactions, the most natural experiences in current societies. Here, participants had identical, intermixed experiences with different ingroup and outgroup individuals and rated their closeness and impressions toward the ingroup and the outgroup. Combining computational models and fMRI, we find that the weight given to ingroup experiences (ingroup prediction errors) is the main source of intergroup impression change, captured by changes in connectivity between the parietal lobe and insula. These findings highlight the importance of ingroup experiences in shaping intergroup impressions in complex social environments.

Asymmetric and adaptive reward coding via normalized reinforcement learning.

Learning is widely modeled in psychology, neuroscience, and computer science by prediction error-guided reinforcement learning (RL) algorithms. While standard RL assumes linear reward functions, reward-related neural activity is a saturating, nonlinear function of reward; however, the computational and behavioral implications of nonlinear RL are unknown. Here, we show that nonlinear RL incorporating the canonical divisive normalization computation introduces an intrinsic and tunable asymmetry in prediction error coding. At the behavioral level, this asymmetry explains empirical variability in risk preferences typically attributed to asymmetric learning rates. At the neural level, diversity in asymmetries provides a computational mechanism for recently proposed theories of distributional RL, allowing the brain to learn the full probability distribution of future rewards. This behavioral and computational flexibility argues for an incorporation of biologically valid value functions in computational models of learning and decision-making.

Action suppression reveals opponent parallel control via striatal circuits.

The direct and indirect pathways of the basal ganglia are classically thought to promote and suppress action, respectively1. However, the observed co-activation of striatal direct and indirect medium spiny neurons2 (dMSNs and iMSNs, respectively) has challenged this view. Here we study these circuits in mice performing an interval categorization task that requires a series of self-initiated and cued actions and, critically, a sustained period of dynamic action suppression. Although movement produced the co-activation of iMSNs and dMSNs in the sensorimotor, dorsolateral striatum (DLS), fibre photometry and photo-identified electrophysiological recordings revealed signatures of functional opponency between the two pathways during action suppression. Notably, optogenetic inhibition showed that DLS circuits were largely engaged to suppress-and not promote-action. Specifically, iMSNs on a given hemisphere were dynamically engaged to suppress tempting contralateral action. To understand how such regionally specific circuit function arose, we constructed a computational reinforcement learning model that reproduced key features of behaviour, neural activity and optogenetic inhibition. The model predicted that parallel striatal circuits outside the DLS learned the action-promoting functions, generating the temptation to act. Consistent with this, optogenetic inhibition experiments revealed that dMSNs in the associative, dorsomedial striatum, in contrast to those in the DLS, promote contralateral actions. These data highlight how opponent interactions between multiple circuit- and region-specific basal ganglia processes can lead to behavioural control, and establish a critical role for the sensorimotor indirect pathway in the proactive suppression of tempting actions.

Oxytocin modulates neurocomputational mechanisms underlying prosocial reinforcement learning.

Humans often act in the best interests of others. However, how we learn which actions result in good outcomes for other people and the neurochemical systems that support this 'prosocial learning' remain poorly understood. Using computational models of reinforcement learning, functional magnetic resonance imaging and dynamic causal modelling, we examined how different doses of intranasal oxytocin, a neuropeptide linked to social cognition, impact how people learn to benefit others (prosocial learning) and whether this influence could be dissociated from how we learn to benefit ourselves (self-oriented learning). We show that a low dose of oxytocin prevented decreases in prosocial performance over time, despite no impact on self-oriented learning. Critically, oxytocin produced dose-dependent changes in the encoding of prediction errors (PE) in the midbrain-subgenual anterior cingulate cortex (sgACC) pathway specifically during prosocial learning. Our findings reveal a new role of oxytocin in prosocial learning by modulating computations of PEs in the midbrain-sgACC pathway.

PO-617-07 MODELS OF SEX-DEPENDENT EPIGENETIC TRANSCRIPTIONAL CONTROL OF CARDIAC ELECTROPHYSIOLOGY IN THE ADULT HUMAN HEART BASED ON THE GTEX DATABASE

The gene-tissue-expression database (GTEx) provides well-organized data sets for inferring links between gene expression patterns in adult humans (21 to 70 years old). These relationships can be parsed by organ, sex, age and other phenotypes of the donors. Computational machine learning models can help quantify such relationships.

Computational modeling of threat learning reveals links with anxiety and neuroanatomy in humans.

Influential theories implicate variations in the mechanisms supporting threat learning in the severity of anxiety symptoms. We use computational models of associative learning in conjunction with structural imaging to explicate links among the mechanisms underlying threat learning, their neuroanatomical substrates, and anxiety severity in humans. We recorded skin-conductance data during a threat-learning task from individuals with and without anxiety disorders (N=251; 8-50 years; 116 females). Reinforcement-learning model variants quantified processes hypothesized to relate to anxiety: threat conditioning, threat generalization, safety learning, and threat extinction. We identified the best-fitting models for these processes and tested associations among latent learning parameters, whole-brain anatomy, and anxiety severity. Results indicate that greater anxiety severity related specifically to slower safety learning and slower extinction of response to safe stimuli. Nucleus accumbens gray-matter volume moderated learning-anxiety associations. Using a modeling approach, we identify computational mechanisms linking threat learning and anxiety severity and their neuroanatomical substrates.

Probabilistic modeling of orthographic learning based on visuo-attentional dynamics.

How is orthographic knowledge acquired? In line with the self-teaching hypothesis, most computational models assume that phonological recoding has a pivotal role in orthographic learning. However, these models make simplifying assumptions on the mechanisms involved in visuo-orthographic processing. Against evidence from eye movement data during orthographic learning, they assume that orthographic information on novel words is immediately available and accurately encoded after a single exposure. In this paper, we describe BRAID-Learn, a new computational model of orthographic learning. BRAID-Learn is a probabilistic and hierarchical model that incorporates the mechanisms of visual acuity, lateral interference, and visual attention involved in word recognition. Orthographic learning in the model rests on three main mechanisms: first, visual attention moves over the input string to optimize the gain of information on letter identity at each fixation; second, top-down lexical influence is modulated as a function of stimulus familiarity; third, after exploration, perceived information is used to create a new orthographic representation or stabilize a better-specified representation of the input word. BRAID-Learn was challenged on its capacity to simulate the eye movement patterns reported in humans during incidental orthographic learning. In line with the behavioral data, the model predicts a larger decline with exposures in number of fixations and processing time for novel words than for known words. For novel words, most changes occur between the first and second exposure, that is to say, after creation in memory of a new orthographic representation. Beyond phonological recoding, our results suggest that visuo-attentional exploration is an intrinsic portion of orthographic learning seldom taken into consideration by models or theoretical accounts.

Impulsivity Relates to Multi-Trial Choice Strategy in Probabilistic Reversal Learning.

Impulsivity is defined as a trait-like tendency to engage in rash actions that are poorly thought out or expressed in an untimely manner. Previous research has found that impulsivity relates to deficits in decision making, in particular when it necessitates executive control or reward outcomes. Reinforcement learning (RL) relies on the ability to integrate reward or punishment outcomes to make good decisions, and has recently been shown to often recruit executive function; as such, it is unsurprising that impulsivity has been studied in the context of RL. However, how impulsivity relates to the mechanisms of RL remains unclear. We aimed to investigate the relationship between impulsivity and learning in a reward-driven learning task with probabilistic feedback and reversal known to recruit executive function. Based on prior literature in clinical populations, we predicted that higher impulsivity would be associated with poorer performance on the task, driven by more frequent switching following unrewarded outcomes. Our results did not support this prediction, but more advanced, trial-history dependent analyses revealed specific effects of impulsivity on switching behavior following consecutive unrewarded trials. Computational modeling captured group-level behavior, but not impulsivity results. Our results support previous findings highlighting the importance of sensitivity to negative outcomes in understanding how impulsivity relates to learning, but indicate that this may stem from more complex strategies than usually considered in computational models of learning. This should be an important target for future research.

Development of new computational machine learning models for longitudinal dispersion coefficient determination: case study of natural streams, United States.

Natural streams longitudinal dispersion coefficient (Kx) is an essential indicator for pollutants transport and its determination is very important. Kx is influenced by several parameters, including river hydraulic geometry, sediment properties, and other morphological characteristics, and thus its calculation is a highly complex engineering problem. In this research, three relatively explored machine learning (ML) models, including Random Forest (RF), Gradient Boosting Decision Tree (GTB), and XGboost-Grid, were proposed for the Kx determination. The modeling scheme on building the prediction matrix was adopted from the well-established literature. Several input combinations were tested for better predictability performance for the Kx. The modeling performance was tested based on the data division for the training and testing (70-30% and 80-20%). Based on the attained modeling results, XGboost-Grid reported the best prediction results over the training and testing phase compared to RF and GTB models. The development of the newly established machine learning model revealed an excellent computed-aided technology for the Kx simulation.

A Novel Integrated Learning Scheme for Predictive Diagnosis of Critical Care燩atient

Machine learning has proven to be one of the efficient solutions for analyzing complex data to perform identification and classification. With a large number of learning tools and techniques, the health section has significantly benefited from solving the diagnosis problems. This paper has reviewed some of the recent scientific implementations on learning-based schemes to find that existing studies of learning have mainly focused on predictive analysis with less emphasis on preprocessing and more inclination towards adopting sophisticated learning schemes that offer higher accuracy at the cost of the higher computational burden. Therefore, the proposed method addresses the concern mentioned above by a novel computational learning model that emphasizes fine-tuning complex medical data and makes it suitable for learning to balance better classification performance and computational complexity. The implementation is carried out using the MIMIC-III dataset, where the proposed system discretizes, the complete model using physician reports and furnished patient information as the first step. It also prepares the data by choosing a specific tuple and its associated field. The second step introduces a novel relatedness function where preprocessing is carried out using word quantization while adopting auto-encoders in deep learning followed by a novel learning-based diagnosis. The outcome exhibits that the proposed system offers better classification performance in reduced processing time in comparison to existing learning schemes.

Modeling the Social Reinforcement of Misinformation Dissemination on Social Media

Despite the salience of misinformation and its consequences, there still lies a tremendous gap in research on the broader tendencies in collective cognition that compels individuals to spread misinformation so excessively. This study examined social learning as an antecedent of engaging with misinformation online. Using data released by Twitter for academic research in 2018, Tweets that included URL news links of both known misinformation and reliable domains were analyzed. Lindström’s computational reinforcement learning model was adapted as an expression of social learning, where a Twitter user’s posting frequency of news links is dependent on the relative engagement they receive in consequence. The research found that those who shared misinformation were highly sensitive to social reward. Inflation of positive social feedback was associated with a decrease in posting latency, indicating that users that posted misinformation were strongly influenced by social learning. However, the posting frequency of authentic news sharers remained fixed, even after receiving an increase in relative and absolute engagement. The results identified social learning is a contributor to the spread of misinformation online. In addition, behavior driven by social validation suggests a positive correlation between posting frequency, gratification received from posting, and a growing mental health dependency on social media. Developing interventions for spreading misinformation online may profit by assessing which online environments amplify social learning, particularly the conditions under which misinformation proliferates.

A Computational Deep Learning Model of Animacy Perception in the Human Brain

A Computational Deep Learning Model of Animacy Perception in the Human Brain

Electrochemical-Kinetics, Computational and Machine Learning Model for Prediction of Azo Dye as Eco-Friendly Corrosion Inhibitor for Stainless Steel in Chloride Solution

Electrochemical-Kinetics, Computational and Machine Learning Model for Prediction of Azo Dye as Eco-Friendly Corrosion Inhibitor for Stainless Steel in Chloride Solution

Asymmetric effects of acute stress on cost and benefit learning

BackgroundHumans are continuously exposed to stressful challenges in everyday life. Such stressful events trigger a complex physiological reaction – the fight-or-flight response – that can hamper flexible decision-making and learning. Inspired by key neural and peripheral characteristics of the fight-or-flight response, here, we ask whether acute stress changes how humans learn about costs and benefits. MethodsHealthy adults were randomly exposed to an acute stress (age mean=23.48, 21/40 female) or no-stress control (age mean=23.80, 22/40 female) condition, after which they completed a reinforcement learning task in which they minimize cost (physical effort) and maximize benefits (monetary rewards). During the task pupillometry data were collected. A computational model of cost-benefit reinforcement learning was employed to investigate the effect of acute stress on cost and benefit learning and decision-making. ResultsAcute stress improved learning to maximize rewards relative to minimizing physical effort (Condition-by-Trial Type interaction: F(1,78)= 6.53, p = 0.01, n2G= 0.04; reward > effort in stress condition: t(39) = 5.40, p < 0.01). Computational modelling revealed that asymmetric learning could be explained by changes in the learning rates of reward value and action cost [condition-by-learning rate (αR, αE) interaction: F(1,78)= 6.42, p = 0.01, n2G= 0.03; αE> αR in control condition: t(39) = −4.75, p < 0.001]. This process was associated with distinct alterations in pupil size fluctuations. Data and scripts are available (https://osf.io/ydv2q/). ConclusionsHere we demonstrate that acute stress is associated with asymmetric learning about reward value versus action cost, thereby providing new insights into learning strategies under acute stress, which, depending on the context, may be maladaptive or beneficial. Our pupillometry and physiological results tentatively link asymmetric cost and benefit learning to stress-related changes in catecholamine activity.

Toward a “Standard Model” of Early Language Learning

A standard model is a theoretical framework that synthesizes observables into a quantitative consensus. Have researchers made progress toward this kind of synthesis for children’s early language learning? Many computational models of early vocabulary learning assume that individual words are learned through an accumulation of environmental input. This assumption is also implicit in empirical work that emphasizes links between language input and learning outcomes. However, models have typically focused on average performance, whereas empirical work has focused on variability. To model individual variability, we relate the tradition of research on accumulator models to item response theory models from psychometrics. This formal connection reveals that currently available data sets do not allow researchers to test the resulting models fully, illustrating a critical need for theory to contribute to shaping new data collection and creating and testing an eventual standard model.

Altered Reinforcement Learning from Reward and Punishment in Anorexia Nervosa: Evidence from Computational Modeling.

Anorexia nervosa (AN) is associated with altered sensitivity to reward and punishment. Few studies have investigated whether this results in aberrant learning. The ability to learn from rewarding and aversive experiences is essential for flexibly adapting to changing environments, yet individuals with AN tend to demonstrate cognitive inflexibility, difficulty set-shifting and altered decision-making. Deficient reinforcement learning may contribute to repeated engagement in maladaptive behavior. This study investigated learning in AN using a probabilistic associative learning task that separated learning of stimuli via reward from learning via punishment. Forty-two individuals with Diagnostic and Statistical Manual of Mental Disorders (DSM)-5 restricting-type AN were compared to 38 healthy controls (HCs). We applied computational models of reinforcement learning to assess group differences in learning, thought to be driven by violations in expectations, or prediction errors (PEs). Linear regression analyses examined whether learning parameters predicted BMI at discharge. AN had lower learning rates than HC following both positive and negative PE (p<.02), and were less likely to exploit what they had learned. Negative PE on punishment trials predicted lower discharge BMI (p<.001), suggesting individuals with more negative expectancies about avoiding punishment had the poorest outcome. This is the first study to show lower rates of learning in AN following both positive and negative outcomes, with worse punishment learning predicting less weight gain. An inability to modify expectations about avoiding punishment might explain persistence of restricted eating despite negative consequences, and suggests that treatments that modify negative expectancy might be effective in reducing food avoidance in AN.