Hypothesis-testing Behavior Research Articles

Using sequence mining to reveal the efficiency in scientific reasoning during STEM learning with a game-based learning environment

The goal of this study was to assess how metacognitive monitoring and scientific reasoning impacted the efficiency of game completion during learning with Crystal Island, a game-based learning environment that fosters self-regulated learning and scientific reasoning by having participants solve the mystery of what illness impacted inhabitants of the island. We conducted sequential pattern mining and differential sequence mining on 64 undergraduate participants’ hypothesis testing behavior. Patterns were coded based on the relevancy of what items were being tested for, and the items themselves. Results revealed that participants who were more efficient at solving the mystery tested significantly fewer partially-relevant and irrelevant items than less efficient participants. Additionally, more efficient participants had fewer sequences of testing items overall, and significantly lower instance support values of the PartiallyRelevant--Relevant to Relevant--Relevant and PartiallyRelevant--PartiallyRelevant to Relevant--Partially Relevant sequences compared to less efficient participants. These findings have implications for designing adaptive GBLEs that scaffold participants based on in-game behaviors.

The role of outcome and experience in hypothesis testing about food allergy

It is important to understand the reasoning strategies that health behaviours are based on. Croker and Buchanan (2011b) found that the strategies people use when choosing how to test a hypothesis about oral health are affected by whether the participant is seeking to reproduce a positive outcome (i.e. good health) or eliminate an unwanted outcome (i.e. bad health). The aim of this study was to investigate the effect of outcome on reasoning strategies in a food allergy context. Participants with and without food allergy were given hypothesis–testing tasks and asked to choose which of three alternative patterns of food consumption could be used to test a hypothesis that a person is allergic to a particular food. Participants were more likely to select a controlled test of the hypothesis that a specific food causes an allergic reaction when a reaction to a food had been observed after eating, than when a reaction had not been observed due to food avoidance. Although the potential severity of making an incorrect choice in a food allergy context is both greater and more proximal than in an oral health context, the same bias in reasoning strategy was found. Logically appropriate hypothesis–testing behaviour may not, therefore, underpin real–world decision making.

Cognitive development. Observing the unexpected enhances infants' learning and exploration.

Given the overwhelming quantity of information available from the environment, how do young learners know what to learn about and what to ignore? We found that 11-month-old infants (N = 110) used violations of prior expectations as special opportunities for learning. The infants were shown events that violated expectations about object behavior or events that were nearly identical but did not violate expectations. The sight of an object that violated expectations enhanced learning and promoted information-seeking behaviors; specifically, infants learned more effectively about objects that committed violations, explored those objects more, and engaged in hypothesis-testing behaviors that reflected the particular kind of violation seen. Thus, early in life, expectancy violations offer a wedge into the problem of what to learn.

Is it better to select or to receive? Learning via active and passive hypothesis testing.

People can test hypotheses through either selection or reception. In a selection task, the learner actively chooses observations to test his or her beliefs, whereas in reception tasks data are passively encountered. People routinely use both forms of testing in everyday life, but the critical psychological differences between selection and reception learning remain poorly understood. One hypothesis is that selection learning improves learning performance by enhancing generic cognitive processes related to motivation, attention, and engagement. Alternatively, we suggest that differences between these 2 learning modes derives from a hypothesis-dependent sampling bias that is introduced when a person collects data to test his or her own individual hypothesis. Drawing on influential models of sequential hypothesis-testing behavior, we show that such a bias (a) can lead to the collection of data that facilitates learning compared with reception learning and (b) can be more effective than observing the selections of another person. We then report a novel experiment based on a popular category learning paradigm that compares reception and selection learning. We additionally compare selection learners to a set of "yoked" participants who viewed the exact same sequence of observations under reception conditions. The results revealed systematic differences in performance that depended on the learner's role in collecting information and the abstract structure of the problem.

Mine or yours? Development of sharing in toddlers in relation to ownership understanding.

To examine early developments in other-oriented resource sharing, fifty-one 18- and 24-month-old children were administered 6 tasks with toys or food that could be shared with an adult playmate who had none. On each task the playmate communicated her desire for the items in a series of progressively more explicit cues. Twenty-four-month-olds shared frequently and spontaneously. Eighteen-month-olds shared when given multiple opportunities and when the partner provided enough communicative support. Younger children engaged in self-focused and hypothesis-testing behavior in lieu of sharing more often than did older children. Ownership understanding, separately assessed, was positively associated with sharing and negatively associated with non-sharing behavior, independent of age and language ability.

Utilities in the 2-4-6 Task

The Wason 2-4-6 task was embedded in a practical reasoning scenario where number sequences had well-defined utilities in the process of achieving a goal. Reasoners' hypothesis-testing behavior was clearly goal-driven and was significantly influenced by whether the utilities favored positive or negative sequences. In the version of the scenario where generating positive sequences had greater benefits than generating negative ones, participants performed poorly at the task as measured by their ability to guess the correct rule and by the nature and number of triples tested before making an announcement. In contrast, the scenario that assigned a greater utility to the production of negative sequences fostered significantly more diligent and creative hypothesis-testing behavior, and participants were significantly more likely to discover the rule. These results suggest that the poor performance observed in Wason's traditional 2-4-6 task reflects a hypothesis-testing process that by default assigns greater utility to the production of sequences that conform to the initial triple, and hence receive positive feedback. However, reasoners are not averse to producing negative sequences, and understand their implication, if their utility is made relevant in the process of achieving goals.

Exploring Explanation: Explaining Inconsistent Evidence Informs Exploratory, Hypothesis‐Testing Behavior in Young Children

Explaining inconsistency may serve as an important mechanism for driving the process of causal learning. But how might this process generate amended beliefs? One way that explaining inconsistency may promote discovery is by guiding exploratory, hypothesis-testing behavior. In order to investigate this, a study with young children ranging in age from 2 to 6 years (N = 80) examined the relation between explanation and exploratory behavior following consistent versus inconsistent outcomes. Results indicated that for inconsistent outcomes only, the kind of explanation children provided informed the kind of exploratory behavior they engaged in and the extent to which children modified and generated new hypotheses. In sum, the data provide insight into a mechanism by which explaining inconsistent evidence guides causal cognition.

You can't have your hypothesis and test it: The importance of utilities in theories of reasoning

AbstractMercier and Sperber's (M&S's) theory of reasoning cannot predict reasoning in the absence of an argumentative context. Applying the theory to hypothesis testing behavior, I propose that hypothesis testing is often motivated by determining the true inference and that reasoning models should account for utilities (affected by various motives, including the wish to convince) of reasoning outcomes.

Preferences for different questions when testing hypotheses in an abstract task: Positivity does play a role, asymmetry does not

Previous studies on hypothesis-testing behaviour have reported systematic preferences for posing positive questions (i.e., inquiries about features that are consistent with the truth of the hypothesis) and different types of asymmetric questions (i.e., questions where the hypothesis confirming and the hypothesis disconfirming responses have different evidential strength). Both tendencies can contribute – in some circumstances – to confirmation biases (i.e., the improper acceptance or maintenance of an incorrect hypothesis). The empirical support for asymmetric testing is, however, scarce and partly contradictory, and the relative strength of positive testing and asymmetric testing has not been empirically compared. In four studies where subjects were asked to select (Experiment 1) or evaluate (Experiments 2–4) questions for controlling an abstract hypothesis, we orthogonally balanced the positivity/negativity of questions by their symmetry/asymmetry (Experiments 1–3), or by the type of asymmetry (confirmatory vs disconfirmatory; Experiment 4). In all Experiments participants strongly preferred positive to negative questions. Their choices were on the other hand mostly unaffected by symmetry and asymmetry in general, or – more specifically – by different types of asymmetry. Other results indicated that participants were sensitive to the diagnosticity of the questions (Experiments 1–3), and that they preferred testing features with a high probability under the focal hypothesis (Experiment 4). In the discussion we argue that recourse to asymmetric testing – observed in some previous studies using more contextualized problems – probably depends on context-related motivations and prior knowledge. In abstract tasks, where that knowledge is not available, more simple strategies – such as positive testing – are prevalent.

Dual-goal facilitation in Wason's 2–4–6 task: What mediates successful rule discovery?

The standard 2-4-6 task requires discovery of a single rule and produces success rates of about 20%, whereas the dual-goal (DG) version requests discovery of two complementary rules and elevates success to over 60%. The experiment examined two explanations of DG superiority: Evans' (1989) positivity-bias account, and Wharton, Cheng, and Wickens' (1993) goal-complementarity theory. Two DG conditions were employed that varied the linguistic labelling of rules (either positively labelled Dax vs. Med, or mixed-valence "fits" vs. "does not fit"). Solution-success results supported the goal-complementarity theory since facilitation arose in both DG conditions relative to single-goal tasks, irrespective of the linguistic labelling of hypotheses. DG instructions also altered quantitative and qualitative aspects of hypothesis-testing behaviour, and analyses revealed the novel result that the production of at least a single descending triple mediates between DG instructions and task success. We propose that the identification of an appropriate contrast class that delimits the scope of complementary rules may be facilitated through the generation of a descending instance. Overall, our findings can best be accommodated by Oaksford and Chater's (1994) iterative counterfactual model of hypotheses testing, which can readily subsume key elements of the goal-complementarity theory.

The impact of external representation in a rule discovery task

The 2-4-6 rule discovery task introduced by Wason (1960) elicits generally poor hypothesis-testing behaviour as reflected by the underwhelming effort and creativity participants invest in the process before announcing their “best” guess. The traditional task departs from real-world hypothesis testing in a significant respect: Reasoners proceed to discover the rule solely from an internal representation of the problem space. Two studies reported here examined representational effects in the 2-4-6 task by developing task isomorphs that offered external, physically manipulable, representations of the dimensions of the problem space. These isomorphs led to a significantly higher incidence of successful rule discovery, driven by significantly greater diligence and creative exploration of the problem space, compared to other task isomorphs that did not offer such an external representation. These results confirm the fruitfulness of a research programme that explores the contextual determinants of sound hypothesis testing.

Optimal data selection: revision, review, and reevaluation.

Since it first appeared, there has been much research and critical discussion on the theory of optimal data selection as an explanation of Wason's (1966,1968) selection task (Oaksford & Chater, 1994). In this paper, this literature is reviewed, and the theory of optimal data selection is reevaluated in its light. The information gain model is first located in the current theoretical debate in the psychology of reasoning concerning dual processes in human reasoning. A model comparison exercise is then presented that compares a revised version of the model with its theoretical competitors. Tests of the novel predictions of the model are then reviewed. This section also reviews experiments claimed not to be consistent with optimal data selection. Finally, theoretical criticisms of optimal data selection are discussed. It is argued either that the revised model accounts for them or that they do not stand up under analysis. It is concluded that some version of the optimal data selection model still provides the best account of the selection task. Consequently, the conclusion of Oaksford and Chater's (1994) original rational analysis (Anderson, 1990), that people's hypothesis-testing behavior on this task is rational and well adapted to the environment, still stands.

Was early man caught knapping during the cognitive (r)evolution?

Wynn describes a revolution in cognitive abilities some 500,000 years ago, which added new sophistication to the curiosity of early man – the ability to form hypotheses. This derivative of archaic curiosity is a fundamental feature of learning, and it is our contention that the naive hypothesis testing behavior of early man will have left a distinctive trail in the archaeological record.

Hypothesis testing as risk behaviour with regard to beliefs

In this paper hypothesis-testing behaviour is compared to risk-taking behaviour. It is proposed that choosing a suitable test for a given hypothesis requires making a preposterior analysis of two aspects of such a test: the probability of obtaining supporting evidence and the evidential value of this evidence. This consideration resembles the one a gambler makes when choosing among bets, each having a probability of winning and an amount to be won. A confirmatory testing strategy can be defined within this framework as a strategy directed at maximizing either the probability or the value of a confirming outcome. Previous theories on testing behaviour have focused on the human tendency to maximize the probability of a confirming outcome. In this paper, two experiments are presented in which participants tend to maximize the confirming value of the test outcome. Motivational factors enhance this tendency dependent on the context of the testing situation. Both this result and the framework are discussed in relation to other studies in the field of testing behaviour. Copyright © 2000 John Wiley & Sons, Ltd.

Effects of test cost and cognitive demand on hypothesis-testing behaviors: Three experiments

Both positive and diagnostic tests of hypotheses have apparent survival value when viewed within the context of the everyday social environment. It was predicted, however, that commonly observed preferences for both types of tests would be sensitive to a number of identifiable situational factors. Three experiments were conducted in which changes in hypothesis-testing behaviors were measured as a function of the cost of conducting tests and the cognitive demands of tests with regard to processing negatives. All experiments employed a fully computerized procedure in which subjects explored ancient temples with the goal of discovering who built them. Test-cost was manipulated by making some tests time-consuming and tedious to conduct, and cognitive demand was manipulated by controlling the negative or positive nature of the initial given evidence. Subjects' preferences for positive and diagnostic tests were shown to change predictably whenever these factors conflicted with each other or with test-cost considerations. The ability to test diagnostically was also shown to be powerfully affected by the nature of the presented evidence. In one experiment, subjects were nearly unanimous in choosing diagnostic tests whenever the presented evidence was positive, whereas these same subjects did not even exhibit a preference for diagnostic tests when given equally informative negative evidence. Overall, the results argue strongly for the need to qualify descriptions of human hypothesis-testing behaviors by referring to the relevant task parameters under which the behaviors in question are exhibited.

Testing vs. Believing Hypotheses: Magical Ideation in the Judgement of Contingencies

This paper examines the idea that an important dimension of human cognition is the amount of objective evidence required for perception of meaningful patterns. At the clinical extreme of this dimension are patients with hallucinations and delusions who experience perception with no external evidence and see connections between objectively unrelated events. Also, normal individuals exhibit considerable variation along this continuum. The theory proposed here predicts that normal subjects with low evidential criteria will be more likely to accept causal explanations, not only for everyday ''paranormal'' coincidences, but also for random contingencies in a laboratory experiment. This prediction was confirmed when 40 students completed a differential reinforcement of low rates (DRL) task designed to induce superstitious behaviour and were then questioned about their hypotheses concerning the contingencies for successful performance. Participants scoring high on the Magical Ideation scale (indicating greater belief in paranormal phenomena) tested fewer hypotheses during the task, and they ended up believing in more hypotheses regarding illusory contingencies than did their low-scoring peers. We proposed that a continuum of hypothesis-testing behaviour underlies the schizotypy continuum, with ''positive'' schizotypal traits reflecting a Type I error bias and ''negative'' traits a Type II error bias. Differential activation patterns within frontal-limbic networks are tentatively suggested as a physiological correlate of the behavioural continuum.

Paradoxes of Falsification

This paper deals with the concept of falsification in hypothesis testing research. A theoretical analysis of assumptions about falsifying behaviour and hypothesis-falsifying observations is presented, with two experimental studies. Both the theoretical analysis and the experimental results point to a number of paradoxes underlying the normative principle of falsification in cognitive psychology. First, subjects experience the falsificatory testing strategy as an impossible strategy to conduct. Obtaining falsifying results is a consequence of the quality of the hypothesis rather than of specific testing behaviour (Experiment 1 and Experiment 2). Second, under some conditions falsifying results impede rather than facilitate discovery (Experiment 2). Confirmatory testing and falsificatory testing, which have been the crucial concepts in the study of hypothesis-testing behaviour, may actually be questionable approaches to testing behaviour. The theoretical analysis is related to the standard analyses of Popper (1963) and Klayman and Ha (1987). The empirical results are discussed in relation to previous studies on falsificatory testing behaviour.

Transitions between modes of inquiry in a rule discovery task.

Scientists obtain their knowledge about the world through one of two major modes of inquiry--observation of naturally occurring phenomena and active experimentation. Both modes of inquiry have been extensively researched in studies of hypothesis testing behaviour and scientific inquiry, the first in studies employing the reception paradigm, the second in studies employing the selection/generation paradigm. The prevalence of both paradigms and their correspondence to well-established modes of scientific inquiry led to the hypothesis that subjects, if allowed, would employ both modes of inquiry. The variables affecting the choice of one mode rather than the other and the transitions between the two modes were expected to shed new light on the process of scientific inquiry. To test this hypothesis, 27 adults and 27 eighth-graders were set three rule-discovery problems, in a computer environment that allowed free transitions between item reception and generation. Item generation was significantly more prevalent in the adult sample, but almost all the adults and approximately half the children employed both modes of inquiry in at least one problem. When both modes of inquiry were employed, the reception mode tended to precede the generation mode. An inverse relationship was observed between item generation and the proportion of positive instances supplied by the environment. Individual inquiry styles were evident in both age groups. These results call attention to the need for a theory incorporating both modes of inquiry as integral components of the inquiry process. They also demonstrate the utility of empirical investigations that let subjects freely choose which mode to employ at any point of the inquiry process.

A philosophical and empirical analysis of subjects' modes of inquiry in Wason's 2-4-6 task.

Investigations of hypothesis-testing behaviour typically conclude that subjects’ methods are characterized by “confirmation bias.” However, these studies (a) relied exclusively upon Karl Popper's philosophy of science, (b) only described subjects’ strategies via a falsificationist terminology, and (c) failed to determine subjects’ precise intentions in the task situations. Alternative philosophies of science can be used to describe hitherto ignored aspects of rule discovery in Wason's (1960) 2–4–6 task as well as reinterpret findings from previous research. Using Wason's task, Experiment 1 adopted J. S. Mill's inductive methodology in order to assess better the extent of elimination in experimentation. While it was found that subjects did eliminate a large proportion of their hypotheses rather than merely confirm them, Mill's eliminative methods applied to this task failed to account for the entirety of subjects’ behaviours. Experiment 2 examined subjects’ methods in more depth by asking subjects during the task either to: (1) describe their procedures, (2) select from a list of methodology terms those that best characterized their methods, and (3) state on each learning trial their preferred hypotheses and their subjective assessments that those hypotheses were correct. Although subjects were generally unable to describe their methods of inquiry, they were able to typify them via the methodology terms as if their methods were tacitly held skills. Contrary to presumptions of previous research, analyses indicated that subjects were not testing particular hypotheses on every trial but were often examining possible instances of the rule “at random” or “different from ones examined” to that trial. It is also suggested that subjects’ decisions to announce rules and their thematic lines of inquiry be given further consideration by researchers.

The effect of the temporal locus of the introtact probe on the hypothesis-testing strategies of kindergarten children

The hypothesis-testing behavior of kindergarten children in discrimination learning was studied in a factorial design with two temporal placements of introtact probes (pretrial and posttrial) and two types of pretraining (unidimensional and multidimensional). Pretrial probes consisting of a request for the child's current hypothesis were administered in a conventional manner at the beginning of each trial with the stimuli in view and prior to the choice response. Posttrial probes were administered also with the stimuli in view, but after the choice response had occurred and feedback information had been provided. Learning-to-learn experience in solving three pretraining problems was given either with unidimensional simultaneous problems or with more complex multidimensional problems like the criterion tasks. The results indicated that posttrial probes produced superior local consistency, win-stay, and lose-shift probabilities, relative to pretrial probes, for both types of pretraining. Posttrial probes, however, facilitated learning and the testing of valid hypotheses only for multidimensional pretraining. These results are consistent with the hypothesis that posttrial probes constrain the child to be locally consistent and therefore improve short-term efficiency in hypothesis testing under both pretraining conditions. However, posttrial probes produce an improvement in long-term efficiency, and therefore in learning, only when other components of a successful strategy are acquired as in multidimensional pretraining.