Instance-based Learning Theory Research Articles

Turing-like Experiment in a Cyber Defense Game

During the past decade, researchers of behavioral cyber security have created cognitive agents that are able to learn and make decisions in dynamic environments in ways that assimilate human decision processes. However, many of these efforts have been limited to simple detection tasks and represent basic cognitive functions rather than a whole set of cognitive capabilities required in dynamic cyber defense scenarios. Our current work aims at advancing the development of cognitive agents that learn and make defense-dynamic decisions during cyber attacks by intelligent attack agents. We also aim to evaluate the capability of these cognitive models in ``Turing-like'' experiments, comparing the decisions and performance of these agents against human cyber defenders. In this paper, we present an initial demonstration of a cognitive model of the defender that relies on a cognitive theory of dynamic decision-making, Instance-Based Learning Theory (IBLT); we also demonstrate the execution of the same defense task by human defenders. We rely on OpenAI Gym and CybORG and adapt an existing CAGE scenario to generate a simulation experiment using an IBL defender. We also offer a new Interactive Defense Game (IDG), where \textit{human} defenders can perform the same CAGE scenario simulated with the IBL model. Our results suggest that the IBL model makes decisions against two intelligent attack agents that are similar to those observed in a subsequent human experiment. We conclude with a description of the cognitive foundations required to build autonomous intelligent cyber defense agents that can collaborate with humans in autonomous cyber defense teams.

Credit Assignment: Challenges and Opportunities in Developing Human-like Learning Agents

Temporal credit assignment is the process of distributing delayed outcomes to each action in a sequence, which is essential for learning to adapt and make decisions in dynamic environments. While computational methods in reinforcement learning, such as temporal difference (TD), have shown success in tackling this issue, it remains unclear whether these mechanisms accurately reflect how humans handle feedback delays. Furthermore, cognitive science research has not fully explored the credit assignment problem in humans and cognitive models. Our study uses a cognitive model based on Instance-Based Learning Theory (IBLT) to investigate various credit assignment mechanisms, including equal credit, exponential credit, and TD credit, using the IBL decision mechanism in a goal-seeking navigation task with feedback delays and varying levels of decision complexity. We compare the performance and process measures of the different models with human decision-making in two experiments. Our findings indicate that the human learning process cannot be fully explained by any of the mechanisms. We also observe that decision complexity affects human behavior but not model behavior. By examining the similarities and differences between human and model behavior, we summarize the challenges and opportunities for developing learning agents that emulate human decisions in dynamic environments.

Applying Generative Artificial Intelligence to cognitive models of decision making.

Generative Artificial Intelligence has made significant impacts in many fields, including computational cognitive modeling of decision making, although these applications have not yet been theoretically related to each other. This work introduces a categorization of applications of Generative Artificial Intelligence to cognitive models of decision making. This categorization is used to compare the existing literature and to provide insight into the design of an ablation study to evaluate our proposed model in three experimental paradigms. These experiments used for model comparison involve modeling human learning and decision making based on both visual information and natural language, in tasks that vary in realism and complexity. This comparison of applications takes as its basis Instance-Based Learning Theory, a theory of experiential decision making from which many models have emerged and been applied to a variety of domains and applications. The best performing model from the ablation we performed used a generative model to both create memory representations as well as predict participant actions. The results of this comparison demonstrates the importance of generative models in both forming memories and predicting actions in decision-modeling research. In this work, we present a model that integrates generative and cognitive models, using a variety of stimuli, applications, and training methods. These results can provide guidelines for cognitive modelers and decision making researchers interested in integrating Generative AI into their methods.

Generative Environment-Representation Instance-Based Learning: A Cognitive Model

Instance-Based Learning Theory (IBLT) suggests that humans learn to engage in dynamic decision making tasks through the accumulation of experiences, represented by the decision task features, the actions performed, and the utility of decision outcomes. This theory has been applied to the design of Instance-Based Learning (IBL) models of human behavior in a variety of contexts. One key feature of all IBL model applications is the method of accumulating instance-based memory and performing recognition-based retrieval. In simple tasks with few features, this knowledge representation and retrieval could hypothetically be done using all relevant information. However, these methods do not scale well to complex tasks when exhaustive enumeration of features is unfeasible. This requires cognitive modelers to design task-specific representations of state features, as well as similarity metrics, which can be time consuming and fail to generalize to related tasks. To address this issue, we leverage recent advancements in Artificial Neural Networks, specifically generative models (GMs), to learn representations of complex dynamic decision making tasks without relying on domain knowledge. We evaluate a range of GMs in their usefulness in forming representations that can be used by IBL models to predict human behavior in a complex decision making task. This work connects generative and cognitive models by using GMs to form representations and determine similarity.

Building Human-Like Artificial Agents: A General Cognitive Algorithm for Emulating Human Decision-Making in Dynamic Environments.

One of the early goals of artificial intelligence (AI) was to create algorithms that exhibited behavior indistinguishable from human behavior (i.e., human-like behavior). Today, AI has diverged, often aiming to excel in tasks inspired by human capabilities and outperform humans, rather than replicating human cogntion and action. In this paper, I explore the overarching question of whether computational algorithms have achieved this initial goal of AI. I focus on dynamic decision-making, approaching the question from the perspective of computational cognitive science. I present a general cognitive algorithm that intends to emulate human decision-making in dynamic environments, as defined in instance-based learning theory (IBLT). I use the cognitive steps proposed in IBLT to organize and discuss current evidence that supports some of the human-likeness of the decision-making mechanisms. I also highlight the significant gaps in research that are required to improve current models and to create higher fidelity in computational algorithms to represent human decision processes. I conclude with concrete steps toward advancing the construction of algorithms that exhibit human-like behavior with the ultimate goal of supporting human dynamic decision-making.

Learning in Cooperative Multiagent Systems Using Cognitive and Machine Models

Developing effective multi-agent systems (MASs) is critical for many applications requiring collaboration and coordination with humans. Despite the rapid advance of multi-agent deep reinforcement learning (MADRL) in cooperative MASs, one of the major challenges that remain is the simultaneous learning and interaction of independent agents in dynamic environments in the presence of stochastic rewards. State-of-the-art MADRL models struggle to perform well in Coordinated Multi-agent Object Transportation Problems (CMOTPs) wherein agents must coordinate with each other and learn from stochastic rewards. In contrast, humans often learn rapidly to adapt to non-stationary environments that require coordination among people. In this article, motivated by the demonstrated ability of cognitive models based on Instance-based Learning Theory (IBLT) to capture human decisions in many dynamic decision-making tasks, we propose three variants of multi-agent IBL models (MAIBLs). The idea of these MAIBL algorithms is to combine the cognitive mechanisms of IBLT and the techniques of MADRL models to deal with coordination MASs in stochastic environments from the perspective of independent learners. We demonstrate that the MAIBL models exhibit faster learning and achieve better coordination in a dynamic CMOTP task with various settings of stochastic rewards compared to current MADRL models. We discuss the benefits of integrating cognitive insights into MADRL models.

Fostering Collective Intelligence in Human-AI Collaboration: Laying the Groundwork for COHUMAIN.

Artificial Intelligence (AI) powered machines are increasingly mediating our work and many of our managerial, economic, and cultural interactions. While technology enhances individual capability in many ways, how do we know that the sociotechnical system as a whole, consisting of a complex web of hundreds of human-machine interactions, is exhibiting collective intelligence? Research on human-machine interactions has been conducted within different disciplinary silos, resulting in social science models that underestimate technology and vice versa. Bringing together these different perspectives and methods at this juncture is critical. To truly advance our understanding of this important and quickly evolving area, we need vehicles to help research connect across disciplinary boundaries. This paper advocates for establishing an interdisciplinary research domain-Collective Human-Machine Intelligence (COHUMAIN). It outlines a research agenda for a holistic approach to designing and developing the dynamics of sociotechnical systems. In illustrating the kind of approach, we envision in this domain, we describe recent work on a sociocognitive architecture, the transactive systems model of collective intelligence, that articulates the critical processes underlying the emergence and maintenance of collective intelligence and extend it to human-AI systems. We connect this with synergistic work on a compatible cognitive architecture, instance-based learning theory and apply it to the design of AI agents that collaborate with humans. We present this work as a call to researchers working on related questions to not only engage with our proposal but also develop their own sociocognitive architectures and unlock the real potential of human-machine intelligence.

SUBSTANTIAL COMPONENTS OF THE DECISION-MAKING PROCESS: CONCEPT, PSYCHOLOGICAL STRUCTURE, AND FACTORS

The article highlights the theoretical study results of the «decision-making» concept, the psychological structure and factors of the decision-making process. The main areas of research on the decision-making process are highlighted: the activity of a practical psychologist; pedagogical activity; marketing; situations of risk and uncertainty; managerial activity. In accordance with the tendency to move away from russian-language calques in Ukrainian scientific terminology, in this article emphasized the practicability of using the construct «ухвалення рішень», which corresponds to the context of the decision, as opposed to the previously used «прийняття рішень». As a result of the analysis the main components of the «decision-making» concept are identified: an integrative cognitive process, a selection process, a set of alternatives, a goal or a desired result, and criteria for evaluating alternatives in accordance with the formulated goal. The selection of the essential characteristics of the decision-making process in different approaches and theories allowed the interpretation of the «decision-making» concept as an integrative cognitive process of choosing the most optimal alternative from the available set, which is occurs in accordance with the formulated goal on the selected criteria basis. The analysis of the instance-based learning theory (IBLT), the layered reference model of the brain (LRMB) in combination with the cognitive model of memory «OAR», the psychological concept of decision-making «WRAP», etc. allowed to identify the decision-making process components (problem situation that leads to the need of finding an optimal solution; goal; reproduction of alternative solutions in memory or generation of new ones in case of the situation novelty; criteria for evaluating alternatives in accordance with the formulated goal; evaluation of alternatives; the choice of the most optimal alternative; implementation) and to characterize its psychological structure. The factors of the decision-making process are identified and characterized: socio-economic status; level of social support; peer environment; level of social risks; reflection; value of education; value of freedom; value of public recognition; personal resoluteness, which includes a number of components (tolerance, risk-taking, assertiveness, adventurousness, detaildness, reflexivity and foresight); motivation for success; urgent needs; heuristics and biases; feelings of regret; the value of different alternatives attributes in a decision situation; the willpower of the individual. Among these factors, a special role is played by reflection, reflexivity, free will and the value of different alternatives attributes in a decision situation. The theoretical analysis of the decision-making process factors has shown the absence of their classification and categorization, which determines the prospects for further research in this area.

SpeedyIBL: A comprehensive, precise, and fast implementation of instance-based learning theory.

Instance-based learning theory (IBLT) is a comprehensive account of how humans make decisions from experience during dynamic tasks. Since it was first proposed almost two decades ago, multiple computational models have been constructed based on IBLT (i.e., IBL models). These models have been demonstrated to be very successful in explaining and predicting human decisions in multiple decision-making contexts. However, as IBLT has evolved, the initial description of the theory has become less precise, and it is unclear how its demonstration can be expanded to more complex, dynamic, and multi-agent environments. This paper presents an updated version of the current theoretical components of IBLT in a comprehensive and precise form. It also provides an advanced implementation of the full set of theoretical mechanisms, SpeedyIBL, to unlock the capabilities of IBLT to handle a diverse taxonomy of individual and multi-agent decision-making problems. SpeedyIBL addresses a practical computational issue in past implementations of IBL models, the curse of exponential growth, that emerges from memory-based tabular computations. When more observations accumulate over time, there is an exponential growth of the memory of instances that leads directly to an exponential slowdown of the computational time. Thus, SpeedyIBL leverages parallel computation with vectorization to speed up the execution time of IBL models. We evaluate the robustness of SpeedyIBL over an existing implementation of IBLT in decision games of increased complexity. The results not only demonstrate the applicability of IBLT through a wide range of decision-making tasks, but also highlight the improvement of SpeedyIBL over its prior implementation as the complexity of decision features the of agents increase. The library is open sourced for the use of the broad research community.

Designing effective masking strategies for cyberdefense through human experimentation and cognitive models

Masking strategies for cyberdefense (i.e., disguising network attributes to hide the real state of the network) are predicted to be effective in simulated experiments. However, it is unclear how effective they are against human attackers. We address three factors that challenge the effectiveness of the masking strategies in practice: (1) we relax the assumption of rationality of the attackers made by Game Theory/Machine Learning defense algorithms; (2) we provide a cognitive model of human attackers that can inform these defense algorithms; and (3) we provide a way to generate data on attacker’s decisions through simulation with a cognitive model. Two masking strategies of defense were generated using Game Theory and Machine Learning (ML) algorithms. The effectiveness of these two masking strategies of defense, risk averse and rational, are compared in an experiment with human attackers. We collected attacker’s decisions against the two masking strategies. With the limited human participant’s data, the results indicate that the risk averse strategy can reduce the defense losses compared to the rational masking strategy. We also propose a cognitive model based on Instance-Based Learning Theory that accurately represents and predicts the attacker’s decisions in this task. We demonstrate the model’s process by generating simulated data and comparing it to the attacker’s actual actions in the experiment. The model is able to capture the data at the aggregate and at the individual levels of attackers making decisions in both rational and risk averse defense algorithms. We propose that this model can be used to inform game theoretic defense algorithms and to produce synthetic data that can be used by ML algorithms to generate new defense strategies.

Learning and Dynamic Decision Making.

Humans make decisions in dynamic environments (increasingly complex, highly uncertain, and changing situations) by searching for potential alternatives sequentially over time, to determine the best option at a precise moment. Surprisingly, the field of behavioral decision making has little to offer in terms of theoretical principles and practical guidelines on how people make decisions in dynamic situations. My research program aims to fill in this gap by developing theoretical understandings of decision processes as well as practical demonstrations of how these theoretical developments can improve human dynamic decision making. Throughout my research career, I have helped create, test, and improve a general theory of dynamic decision making, instance-based learning theory, IBLT. The methods I have used to contribute to IBLT are (1) laboratory experiments that rely on dynamic games in which humans make choices over time and space, individually and in teams, and from which we extrapolate robust phenomena and behavioral insights; and (2) computational, actionable cognitive models, which specify the decision-making process and the cognitive mechanisms involved into a computational algorithm. The combination of these methods spawned novel applications in areas such as cybersecurity, phishing, climate change, and human-machine interactions. In this paper, I will take you through my own intellectual exploratory experience of computational modeling of human decision processes, and how the integration of experimental work and cognitive modeling helped in discovering and uncovering the field of dynamic decisionmaking.

Impact of digital nudging on information security behavior: an experimental study on framing and priming in cybersecurity

PurposePhishing attacks are the most common cyber threats targeted at users. Digital nudging in the form of framing and priming may reduce user susceptibility to phishing. This research focuses on two types of digital nudging, framing and priming, and examines the impact of framing and priming on users' behavior (i.e. action) in a cybersecurity setting. It draws on prospect theory, instance-based learning theory and dual-process theory to generate the research hypotheses.Design/methodology/approachA 3 × 2 experimental study was carried out to test the hypotheses. The experiment consisted of three levels for framing (i.e. no framing, negative framing and positive framing) and two levels for priming (i.e. with and without priming).FindingsThe findings suggest that priming users to information security risks reduces their risk-taking behavior, whereas positive and negative framing of information security messages regarding potential consequences of the available choices do not change users' behavior. The results also indicate that risk-averse cybersecurity behavior is associated with greater confidence with the action, greater perceived severity of cybersecurity risks, lower perceived susceptibility to cybersecurity risks resulting from the action and lower trust in the download link.Originality/valueThis research shows that digital nudging in the form of priming is an effective way to reduce users' exposure to cybersecurity risks.

Theory of Mind From Observation in Cognitive Models and Humans.

A major challenge for research in artificial intelligence is to develop systems that can infer the goals, beliefs, and intentions of others (i.e., systems that have theory of mind, ToM). In this research, we propose a cognitive ToM framework that uses a well-known theory of decisions from experience to construct a computational representation of ToM. Instance-based learning theory (IBLT) is used to construct a cognitive model that generates ToM from the observation of other agents' behavior. The IBL model of the observer distinguishes itself from previous models of ToM that make unreasonable assumptions about human cognition, are hand-crafted for particular settings, complex, or unable to explain a cognitive development of ToM compared to human's ToM. The IBL model learns from the observation of goal-directed agents' behavior in a gridworld navigation task, and it infers and predicts the behaviors of the agents in new gridworlds across different degrees of decision complexity in similar ways to the way human observers do. We provide evidence for the alignment of the IBL observer's predictions under various levels of decision complexity. We also advance the demonstration of the IBL predictions using a classic test of false beliefs (the Sally-Anne test), which is commonly used to test ToM in humans. We discuss our results and the potential of the IBL observer model to improve human-machine interactions.

What Attackers Know and What They Have to Lose: Framing Effects on Cyber-attacker Decision Making

Many cybersecurity algorithms assume adversaries make perfectly rational decisions. However, human decisions are only boundedly rational and, according to Instance-Based Learning Theory, are based on the similarity of the present contextual features to past experiences. More must be understood about what available features are represented in the decision and how outcomes are evaluated. To these ends, we examined human behavior in a cybersecurity game designed to simulate an insider attack scenario. In a human-subjects experiment, we manipulated the information made available to participants (concealed or revealed decision probabilities) and the framing of the outcome (as losses or not). An endowment was given to frame negative outcomes as losses, but these were not framed as losses when no endowment was given. The results reveal differences in behavior when some information is concealed, but the framing of outcomes only affects behavior when all information is available. A cognitive model was developed to help understand the cognitive representation of these features and the implications of the behavioral results.

Computational cognitive modeling and validation of Dp140 induced alteration of working memory in Duchenne Muscular Dystrophy

Duchenne Muscular Dystrophy has emerged as a model to assess cognitive domains. The DMD gene variant location and its association with variable degrees of cognitive impairment necessitate identification of a common denominator. Computer architectures provide a framework to delineate the mechanisms involved in the cognitive functioning of the human brain. Copy number variations in the 79 exons of DMD gene were screened in 84 DMD subjects by Multiplex Ligation-dependent Probe Amplification (MLPA). DMD subjects were categorized based on the presence or absence of DP140 isoform. The cognitive and neuropsychological assessments were carried out as per inclusion criteria using standard scales. Instance-based learning theory (IBLT) based on the partial matching process was developed to mimic Stroop Color and Word Task (SCWT) performance on Adaptive Control of Thought-Rational (ACT-R) cognitive architecture based on IBLT. Genotype–phenotype correlation was conducted based on the mutation location in DMD gene. Assessment of specific cognitive domains in DP140 − ve group corresponded to the involvement of multiple brain lobes including temporal (verbal and visual learning and memory), parietal (visuo-conceptual and visuo-constructive abilities) and frontal (sustained and focused attention, verbal fluency, cognitive control). Working memory axis was found to be the central domain through tasks including RAVLT trial 1, recency effect, digit span backward, working memory index, arithmetic subtests in the Dp140 − ve group. IBLT validated the non-reliance of DMD subjects on recency indicating affected working memory domain. Modeling strategy revealed altered working memory processes in DMD cases with affected Dp140 isoform. DMD brain was observed to rely on primacy than the recency suggesting alterations in working memory capacity. Modeling revealed lowered activation of DMD brain with Dp140 − ve in order to retrieve the instances.

Cyber Security: Effects of Penalizing Defenders in Cyber-Security Games via Experimentation and Computational Modeling.

Cyber-attacks are deliberate attempts by adversaries to illegally access online information of other individuals or organizations. There are likely to be severe monetary consequences for organizations and its workers who face cyber-attacks. However, currently, little is known on how monetary consequences of cyber-attacks may influence the decision-making of defenders and adversaries. In this research, using a cyber-security game, we evaluate the influence of monetary penalties on decisions made by people performing in the roles of human defenders and adversaries via experimentation and computational modeling. In a laboratory experiment, participants were randomly assigned to the role of “hackers” (adversaries) or “analysts” (defenders) in a laboratory experiment across three between-subject conditions: Equal payoffs (EQP), penalizing defenders for false alarms (PDF) and penalizing defenders for misses (PDM). The PDF and PDM conditions were 10-times costlier for defender participants compared to the EQP condition, which served as a baseline. Results revealed an increase (decrease) and decrease (increase) in attack (defend) actions in the PDF and PDM conditions, respectively. Also, both attack-and-defend decisions deviated from Nash equilibriums. To understand the reasons for our results, we calibrated a model based on Instance-Based Learning Theory (IBLT) theory to the attack-and-defend decisions collected in the experiment. The model’s parameters revealed an excessive reliance on recency, frequency, and variability mechanisms by both defenders and adversaries. We discuss the implications of our results to different cyber-attack situations where defenders are penalized for their misses and false-alarms.

Modeling the social-influence-based route choice behavior in a two-route network

In this paper, we first propose an instance-based learning theory (IBLT) model with social learning to study the day-to-day route choice behavior in a two-route network. We then define four indexes (i.e., efficiency, stability, cooperation, and equity) to investigate the effects of social learning on each traffic participant’s route choice behavior in a two-route network. Numerical results show that social influence has some positive impacts on route choice behavior when more participants select the recommended routes. Cooperation among participants requires them to change their route choice behaviors against their natural tendencies. When each participant is very conscious of other participants’ route choice behaviors, it can alleviate the above phenomenon.

Unpacking buyer-seller differences in valuation from experience: A cognitive modeling approach

People often indicate a higher price for an object when they own it (i.e., as sellers) than when they do not (i.e., as buyers)—a phenomenon known as the endowment effect. We develop a cognitive modeling approach to formalize, disentangle, and compare alternative psychological accounts (e.g., loss aversion, loss attention, strategic misrepresentation) of such buyer-seller differences in pricing decisions of monetary lotteries. To also be able to test possible buyer-seller differences in memory and learning, we study pricing decisions from experience, obtained with the sampling paradigm, where people learn about a lottery’s payoff distribution from sequential sampling. We first formalize different accounts as models within three computational frameworks (reinforcement learning, instance-based learning theory, and cumulative prospect theory), and then fit the models to empirical selling and buying prices. In Study 1 (a reanalysis of published data with hypothetical decisions), models assuming buyer-seller differences in response bias (implementing a strategic-misrepresentation account) performed best; models assuming buyer-seller differences in choice sensitivity or memory (implementing a loss-attention account) generally fared worst. In a new experiment involving incentivized decisions (Study 2), models assuming buyer-seller differences in both outcome sensitivity (as proposed by a loss-aversion account) and response bias performed best. In both Study 1 and 2, the models implemented in cumulative prospect theory performed best. Model recovery studies validated our cognitive modeling approach, showing that the models can be distinguished rather well. In summary, our analysis supports a loss-aversion account of the endowment effect, but also reveals a substantial contribution of simple response bias.

Cyber security: testing the effects of attack strategy, similarity, and experience on cyber attack detection

Cyber attacks, the disruption of normal functioning of computers in a network due to malicious events (threats), are becoming widespread and the role of security analysts is becoming important in protecting networks by accurately and timely detecting cyber attacks. In this paper, we investigate the role of two internal factors, similarity and experience, and an external factor, strategy of an attacker, to influence a simulated analyst’s detection of cyber attacks. We use an existing cognitive model, based upon instance-based learning theory, which represents the decision-making process of a security analyst. We manipulate the attack strategy, experience, and similarity assumptions and evaluate their effects on model’s accurate and timely detection of cyber attacks. Results revealed that although experience and strategy played a significant role in cyber attack detection; the role of similarity was much smaller. We highlight the implications of our findings for training human security analysts in their job.

Cyber Situation Awareness

To determine the effects of an adversary's behavior on the defender's accurate and timely detection of network threats. Cyber attacks cause major work disruption. It is important to understand how a defender's behavior (experience and tolerance to threats), as well as adversarial behavior (attack strategy), might impact the detection of threats. In this article, we use cognitive modeling to make predictions regarding these factors. Different model types representing a defender, based on Instance-Based Learning Theory (IBLT), faced different adversarial behaviors. A defender's model was defined by experience of threats: threat-prone (90% threats and 10% nonthreats) and nonthreat-prone (10% threats and 90% nonthreats); and different tolerance levels to threats: risk-averse (model declares a cyber attack after perceiving one threat out of eight total) and risk-seeking (model declares a cyber attack after perceiving seven threats out of eight total). Adversarial behavior is simulated by considering different attack strategies: patient (threats occur late) and impatient (threats occur early). For an impatient strategy, risk-averse models with threat-prone experiences show improved detection compared with risk-seeking models with nonthreat-prone experiences; however, the same is not true for a patient strategy. Based upon model predictions, a defender's prior threat experiences and his or her tolerance to threats are likely to predict detection accuracy; but considering the nature of adversarial behavior is also important. Decision-support tools that consider the role of a defender's experience and tolerance to threats along with the nature of adversarial behavior are likely to improve a defender's overall threat detection.