Behavior Research Methods Research Articles

Simulation studies for methodological research in psychology: A standardized template for planning, preregistration, and reporting.

Simulation studies are widely used for evaluating the performance of statistical methods in psychology. However, the quality of simulation studies can vary widely in terms of their design, execution, and reporting. In order to assess the quality of typical simulation studies in psychology, we reviewed 321 articles published in Psychological Methods, Behavior Research Methods, and Multivariate Behavioral Research in 2021 and 2022, among which 100/321 = 31.2% report a simulation study. We find that many articles do not provide complete and transparent information about key aspects of the study, such as justifications for the number of simulation repetitions, Monte Carlo uncertainty estimates, or code and data to reproduce the simulation studies. To address this problem, we provide a summary of the ADEMP (aims, data-generating mechanism, estimands and other targets, methods, performance measures) design and reporting framework from Morris et al. (2019) adapted to simulation studies in psychology. Based on this framework, we provide ADEMP-PreReg, a step-by-step template for researchers to use when designing, potentially preregistering, and reporting their simulation studies. We give formulae for estimating common performance measures, their Monte Carlo standard errors, and for calculating the number of simulation repetitions to achieve a desired Monte Carlo standard error. Finally, we give a detailed tutorial on how to apply the ADEMP framework in practice using an example simulation study on the evaluation of methods for the analysis of pre-post measurement experiments. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Development of a Marmoset Apparatus for Automated Pulling to study cooperative behaviors

In recent years, the field of neuroscience has increasingly recognized the importance of studying animal behaviors in naturalistic environments to gain deeper insights into ethologically relevant behavioral processes and neural mechanisms. The common marmoset (Callithrix jacchus), due to its small size, prosocial nature, and genetic proximity to humans, has emerged as a pivotal model toward this effort. However, traditional research methodologies often fail to fully capture the nuances of marmoset social interactions and cooperative behaviors. To address this critical gap, we developed the Marmoset Apparatus for Automated Pulling (MarmoAAP), a novel behavioral apparatus designed for studying cooperative behaviors in common marmosets. MarmoAAP addresses the limitations of traditional behavioral research methods by enabling high-throughput, detailed behavior outputs that can be integrated with video and audio recordings, allowing for more nuanced and comprehensive analyses even in a naturalistic setting. We also highlight the flexibility of MarmoAAP in task parameter manipulation which accommodates a wide range of behaviors and individual animal capabilities. Furthermore, MarmoAAP provides a platform to perform investigations of neural activity underlying naturalistic social behaviors. MarmoAAP is a versatile and robust tool for advancing our understanding of primate behavior and related cognitive processes. This new apparatus bridges the gap between ethologically relevant animal behavior studies and neural investigations, paving the way for future research in cognitive and social neuroscience using marmosets as a model organism.

Development of a Marmoset Apparatus for Automated Pulling to study cooperative behaviors.

In recent years, the field of neuroscience has increasingly recognized the importance of studying animal behaviors in naturalistic environments to gain deeper insights into ethologically relevant behavioral processes and neural mechanisms. The common marmoset (Callithrix jacchus), due to its small size, prosocial nature, and genetic proximity to humans, has emerged as a pivotal model toward this effort. However, traditional research methodologies often fail to fully capture the nuances of marmoset social interactions and cooperative behaviors. To address this critical gap, we developed the Marmoset Apparatus for Automated Pulling (MarmoAAP), a novel behavioral apparatus designed for studying cooperative behaviors in common marmosets. MarmoAAP addresses the limitations of traditional behavioral research methods by enabling high-throughput, detailed behavior outputs that can be integrated with video and audio recordings, allowing for more nuanced and comprehensive analyses even in a naturalistic setting. We also highlight the flexibility of MarmoAAP in task parameter manipulation which accommodates a wide range of behaviors and individual animal capabilities. Furthermore, MarmoAAP provides a platform to perform investigations of neural activity underlying naturalistic social behaviors. MarmoAAP is a versatile and robust tool for advancing our understanding of primate behavior and related cognitive processes. This new apparatus bridges the gap between ethologically relevant animal behavior studies and neural investigations, paving the way for future research in cognitive and social neuroscience using marmosets as a model organism.

Assessing computational reproducibility in Behavior Research Methods

Psychological science has thrived thanks to new methods and innovative practices. Journals, including Behavior Research Methods (BRM), continue to support the dissemination and evaluation of research assets including data, software/hardware, statistical code, and databases of stimuli. However, such research assets rarely allow for computational reproducibility, meaning they are difficult to reuse. Therefore, in this preregistered report, we explore how BRM’s authors and BRM structures shape the landscape of functional research assets. Our broad research questions concern: (1) How quickly methods and analytical techniques reported in BRM can be used and developed further by other scientists; (2) Whether functionality has improved following changes to BRM journal policy in support of computational reproducibility; (3) Whether we can disentangle such policy changes from changes in reproducibility over time. We randomly sampled equal numbers of papers (N = 204) published in BRM before and after the implementation of policy changes. Pairs of researchers recorded how long it took to ensure assets (data, software/hardware, statistical code, and materials) were fully operational. They also coded the completeness and reusability of the assets. While improvements were observed in all measures, only changes to completeness were altered significantly following the policy changes (d = .37). The effects varied between different types of research assets, with data sets from surveys/experiments showing the largest improvements in completeness and reusability. Perhaps more importantly, changes to policy do appear to have improved the life span of research products by reducing natural decline. We conclude with a discussion of how, in the future, research and policy might better support computational reproducibility within and beyond psychological science.

Development of a Marmoset Apparatus for Automated Pulling (MarmoAAP) to Study Cooperative Behaviors.

In recent years, the field of neuroscience has increasingly recognized the importance of studying animal behaviors in naturalistic environments to gain deeper insights into ethologically relevant behavioral processes and neural mechanisms. The common marmoset (Callithrix jacchus), due to its small size, prosocial nature, and genetic proximity to humans, has emerged as a pivotal model toward this effort. However, traditional research methodologies often fail to fully capture the nuances of marmoset social interactions and cooperative behaviors. To address this critical gap, we developed the Marmoset Apparatus for Automated Pulling (MarmoAAP), a novel behavioral apparatus designed for studying cooperative behaviors in common marmosets. MarmoAAP addresses the limitations of traditional behavioral research methods by enabling high-throughput, detailed behavior outputs that can be integrated with video and audio recordings, allowing for more nuanced and comprehensive analyses even in a naturalistic setting. We also highlight the flexibility of MarmoAAP in task parameter manipulation which accommodates a wide range of behaviors and individual animal capabilities. Furthermore, MarmoAAP provides a platform to perform investigations of neural activity underlying naturalistic social behaviors. MarmoAAP is a versatile and robust tool for advancing our understanding of primate behavior and related cognitive processes. This new apparatus bridges the gap between ethologically relevant animal behavior studies and neural investigations, paving the way for future research in cognitive and social neuroscience using marmosets as a model organism.

Affordance norms for 2825 concrete nouns.

Objects are commonly described based on their relations to other objects (e.g., associations, semantic similarity, etc.) or their physical features (e.g., birds have wings, feathers, etc.). However, objects can also be described in terms of their actionable properties (i.e., affordances), which reflect interactive relations between actors and objects. While several normed datasets have been developed to categorize various aspects of meaning (e.g., semantic features, cue-target associations, etc.), to date, norms for affordances have not been generated. We address this limitation by developing a set of affordance norms for 2825 concrete nouns. Using an open-response format, we computed affordance strength (AFS; i.e., the probability of an item eliciting a particular action response), affordance proportion (AFP; i.e., the proportion of participants who provided a specific action response), and affordance set size (AFSS; i.e., the total number of unique action responses) for each item. Because our stimuli overlapped with Pexman et al.'s, Behavior Research Methods, 51, 453-466, (2019) body-object interaction norms (BOI), we tested whether AFS, AFP, and AFSS were related to BOI, as objects with more perceived action properties may be viewed as being more interactive. Additionally, we tested the relationship between AFS and AFP and two separate measures of relatedness: cosine similarity (Buchanan et al., Behavior Research Methods, 51, 1849-1863, 2019a, Behavior Research Methods, 51, 1878-1888, 2019b) and forward associative strength (Nelson et al., Behavior Research Methods, Instruments, & Computers, 36(3), 402-407, 2004). All analyses, however, revealed weak relationships between affordance measures and existing semantic norms, suggesting that affordance properties reflect a separate construct.

Prospects of Electrocorticography in Neuropharmacological Studies in Small Laboratory Animals.

Electrophysiological methods of research are widely used in neurobiology. To assess the bioelectrical activity of the brain in small laboratory animals, electrocorticography (ECoG) is most often used, which allows the recording of signals directly from the cerebral cortex. To date, a number of methodological approaches to the manufacture and implantation of ECoG electrodes have been proposed, the complexity of which is determined by experimental tasks and logistical capabilities. Existing methods for analyzing bioelectrical signals are used to assess the functional state of the nervous system in test animals, as well as to identify correlates of pathological changes or pharmacological effects. The review presents current areas of applications of ECoG in neuropharmacological studies in small laboratory animals. Traditionally, this method is actively used to study the antiepileptic activity of new molecules. However, the possibility of using ECoG to assess the neuroprotective activity of drugs in models of traumatic, vascular, metabolic, or neurodegenerative CNS damage remains clearly underestimated. Despite the fact that ECoG has a number of disadvantages and methodological difficulties, the recorded data can be a useful addition to traditional molecular and behavioral research methods. An analysis of the works in recent years indicates a growing interest in the method as a tool for assessing the pharmacological activity of psychoactive drugs, especially in combination with classification and prediction algorithms.

Lowering the Stakes: Quasi-Experimental, Mixed-Methods Evaluation of a Restructured Grading Approach in a Graduate Public Health Research Methods Course

In schools and programs of public health, active learning and restructured feedback strategies may improve student learning and course performance compared to traditional lecture-based formats and grading systems. To promote student engagement and active learning, we implemented the team-based learning (TBL) model and a restructured grading approach based on principles of “ungrading” in the required behavioral research methods course of a master’s level public health curriculum. We conducted a quasi-experimental, mixed-methods evaluation of the implementation of restructured grading approach (two sections, n = 46) compared to the traditional grading approach (two sections, n = 34). For the restructured grading sections, numeric grades were removed for weekly team quizzes and team assignments. We administered and analyzed pre- and post-course surveys to compare perceived mastery of course learning objectives, intrinsic motivation, and perceptions of grading and instructor feedback by grading approach. Additionally, we compared course grades and final paper grades by grading approach and conducted two focus groups (one for each approach). The results indicate that students’ learning, satisfaction, and perceptions of the course were mainly equivalent across grading approaches. Students in the restructured grading sections demonstrated a few modestly improved outcomes compared to their peers in the traditional grading courses. Students in the restructured grading sections had slightly higher mean course grades, reported greater increase in confidence to critically evaluate research designs, and found team quizzes to be more helpful. Removing grades for team quizzes and assignments supported student learning and reduced student stress.

Neuropsychological interpretation of disorders of consciousness using data from instrumental (neurophysiological) methods for diagnosing brain activity

The relevance of the problem under consideration is determined by the need to develop and improve interdisciplinary approaches to the diagnosis and rehabilitation of disorders of consciousness in patients with brain pathology. The purpose of the article is an analytical review of the methods of neuropsychological and neurophysiological diagnostics and rehabilitation work with patients in reduced states of consciousness. It is noted that the neuropsychological content of the concept of "consciousness" is insufficiently developed and there is no unified point of view on the brain basis of consciousness, as well as on methodological and procedural limitations that arise when a neuropsychologist works with patients in a vegetative state of consciousness and in a state of minimal consciousness. The problem of consistency of the results of behavioral (neuropsychological) and instrumental (neurophysiological) methods for assessing the level of states of consciousness conducted by different specialists (neuropsychologists, neurologists, neurophysiologists) who are part of a multidisciplinary team is considered. The possibility of combining the procedure of neuropsychological examination and instrumental (neurophysiological) methods in the diagnosis of patients in a vegetative state of consciousness, in a state of minimal consciousness) and the prognosis of restoring the level of consciousness is analyzed. The possibility of an integrated approach to the diagnosis of a state of consciousness associated with a combination of behavioral (observation-based) and objective (instrumental) research methods is confirmed, and possible ways of its implementation are considered.

A comparison of word humor ratings across speakers of North American, British, and Singapore English.

Large-scale collection of lexical-semantic norms for words in a given language has been instrumental in the progress of psycholinguistic research. However, such norms tend to be collected from speakers of the dominant variant or dialect. This research aims to determine if there may be differences across speakers of various dialects of English in the humor of individual words. Engelthaler and Hills (2018, Behavior Research Methods, 50[3], 1116-1124) observed that their humor ratings were most strongly correlated with inverse word frequency: Less frequent words tended to be rated as more humorous. We hypothesized that words that are less frequently occurring in a given English dialect should be perceived as more humorous by speakers of the same dialect. We selected words of relatively higher and lower frequencies across various corpora of North American, British, or Singapore English, and presented these words to participants who were native English speakers of North American, British, or Singapore English. Study 1 compared humor ratings of North Americans and Singaporeans; Study 2 compared humor ratings of North Americans and the British. Analyses of participants' random slope coefficients of frequency extracted from cumulative link mixed-effects models indicated that humor ratings were more strongly (and inversely) associated with the word's frequency in the corpora that aligned with the rater's English dialect. These results provide evidence that people are sensitive to the statistics of their specific language environment, and importantly suggest that creators of lexical-semantic norm databases should consider how the cultural, historical, or sociopolitical context of raters might influence the nature of their ratings.

Comparing child word associations to adult associative norms: Evidence for child-specific associations with a strong priming effect in 3-year-olds

Investigating how infants first establish relationships between words is a necessary step towards understanding how an interconnected network of semantic relationships develops in the adult lexical-semantic system. Stimuli selection for these child studies is critical since words must be both familiar and highly imageable. However, there has been a reliance on adult word association norms to inform stimuli selection in English infant studies to date, as no resource currently exists for child-specific word associations. We present three experiments that explore the strength of word–word relationships in 3-year-olds. Experiment 1 collected children’s word associations (WA) (N = 150; female = 84, L1 = British English) and compared them to adult associative norms (Moss & Older, 1996; Nelson et al., 2004 (Behavior Research Methods, Instruments, & Computers, 36(3), 402–407)). Experiment 2 replicated WAs from Experiment 1 in an online adaptation of the task (N = 24: 13 female, L1 = British English). Both experiments indicated a high proportion of child-specific WAs not represented in adult norms (Moss & Older, 1996; Nelson et al., 2004 (Behavior Research Methods, Instruments, & Computers, 36(3), 402–407)). Experiment 3 tested noun–noun WAs from these responses in an online semantic priming study (N = 40: 19 female, L1 = British English) and found that association type modulated priming (F(2.57, 100.1) = 13.13, p <. 0001, generalized η2 = .19). This research presents a resource of child-specific imageable noun–noun word pair stimuli suitable for testing young children in word recognition and semantic priming studies.

Behavioral Studies of Zebrafish Reveal a New Perspective on the Reproductive Toxicity of Micro- and Nanoplastics.

The escalating prevalence of microplastics and nanoplastics in aquatic environments is a major challenge affecting the behavior and reproductive health of aquatic organisms while posing potential risks to human health and ecosystems. This review focuses on the neurobehavioral changes and reproductive toxicity of MNPs in zebrafish and their relationships. At the same time, the neurobehavioral changes caused by MNPs were studied, and the synergistic effects of the interaction of these pollutants with other environmental contaminants were explored. In addition, zebrafish, as a model organism, provide valuable insights into the subtle but important effects of MNPs on reproductive behavior, which is critical for understanding reproductive success, suggesting that behavioral changes can serve as an early biomarker of reproductive toxicity. In addition, based on classical endocrine disruptor models and behavioral research methods, the current status of the research on the reproductive toxicity of MNPs in zebrafish was reviewed, which further indicated that the behavioral parameters of zebrafish can be used as an effective and rapid tool to evaluate the reproductive toxicity of MNPs. However, behavioral methods for rapidly assessing the toxicity of MNPs are still an area of exploration. To address limitations and challenges in the current scope of research, this review outlines future research directions with the aim of improving our understanding of the environmental and health impacts of MNPs. This work aims to inform targeted environmental policies and advance public health strategies to address the growing challenge of MNPs pollution.

Minimizing observer bias in animal behavior studies revisited: Improvement, but a long way to go

AbstractFor decades, texts on methods in animal behavior research have stressed the need for observers of behavior to work to minimize potential unconscious biases in their coding of data. Two major ways of minimizing these biases are to carry out data coding blind to the key comparisons being made in the study and to have high inter‐observer reliability. Over 10 years ago, Burghardt et al. (2012, Ethology, 118, 511) reviewed five major journals in the field of animal behavior and coded randomly selected articles from five decadal volumes (1970 to 2010). That earlier article found poor rates of reporting these two common methods for minimizing potential biases. Here, we carried out similar coding for the 2020 volumes from those same five journals. We found that rates of reporting have increased in all five journals – some substantially. However, rates of reporting still lag behind the journal Infancy, which publishes research on human infant development and relies on many of the same behavioral observation and coding methods used by animal behavior researchers. Given increased calls for transparency and reproducibility in many different fields of scientific study, we argue that we – researchers, reviewers, and editors – can and need to do better at making sure we are actively conducting research in ways to minimize potential observer biases.

Rethinking orthographic neighbor in Chinese two-character word recognition: Insights from a megastudy.

The influence of orthographic neighbors on visual word recognition is well established in alphabetic scripts. To determine the universality of this effect across writing systems, researchers have been keen on exploring its presence and nature in Chinese word recognition. Given that Chinese is logographic, it necessitates a different definition for orthographic neighbors from the ones used in alphabetic scripts. One popular approach is to consider words that share characters as orthographic neighbors. Adopting this definition, a facilitative effect has been observed for characters that can create more words. However, as characters are also morphemes in Chinese, the facilitation found might actually come from a larger morphological family size. This possibility was tested in the present study by analyzing data from the Chinese Lexicon Project (CLP; Tse et al., Behavior Research Methods, 49, 1503-1519, 2017, Behavior Research Methods, 49, 1503-1519, 2022), a megastudy of two-character word recognition in traditional Chinese. If the effects of character-sharing are indeed morphological in nature, the facilitation should be smaller for ambiguous characters because the words formed are distributed over several morphological families. The results of the analyses were consistent with this hypothesis, revealing interactions between the number of words formed by a character and the number of meanings of the character. The implications of these findings were discussed in the context of definitions of orthographic neighbors and theories of word recognition in Chinese.

Feats: A database of semantic features for early produced noun concepts.

Semantic feature production norms have several desirable characteristics that have supported models of representation and processing in adults. However, several key challenges have limited the use of semantic feature norms in studies of early language acquisition. First, existing norms provide uneven and inconsistent coverage of early-acquired concepts that are typically produced and assessed in children under the age of three, which is a time of tremendous growth of early vocabulary skills. Second, it is difficult to assess the degree to which young children may be familiar with normed features derived from these adult-generated datasets. Third, it has been difficult to adopt standard methods to generate semantic network models of early noun learning. Here, we introduce Feats-a tool that was designed to make headway on these challenges by providing a database, the Language Learning and Meaning Acquisition (LLaMA) lab Noun Norms that extends a widely used set of feature norms McRae et al. Behavior Research Methods 37, 547-559, (2005) to include full coverage of noun concepts on a commonly used early vocabulary assessment. Feats includes several tools to facilitate exploration of features comprising early-acquired nouns, assess the developmental appropriateness of individual features using toddler-accessibility norms, and extract semantic network statistics for individual vocabulary profiles. We provide a tutorial overview of Feats. We additionally validate our approach by presenting an analysis of an overlapping set of concepts collected across prior and new data collection methods. Furthermore, using network graph analyses, we show that the extended set of norms provides novel, reliable results given their enhanced coverage.

Concreteness ratings for 36,000 Estonian words.

We present a collection of concreteness ratings for 35,979 words in Estonian. The data were collected via a web application from 2278 native Estonian speakers. Human ratings of concreteness have not been collected for Estonian beforehand. We compare our results to Aedmaa etal. (2018), who assigned concreteness ratings to 240,000 Estonian words by means of machine learning. We show that while these two datasets show reasonable correlation (R = 0.71), there are considerable differences in the distribution of the ratings, which we discuss in this paper. Furthermore, the results also raise questions about the importance of the type of scale used for collecting ratings. While most other datasets have been compiled based on questionnaires entailing five- or seven-point Likert scales, we used a continuous 0-10 scale. Comparing our rating distribution to those of other studies, we found that it is most similar to the distribution in Lahl et al. (Behavior Research Methods, 41(1), 13-19, 2009), who also used a 0-10 scale. Concreteness ratings for Estonian words are available at OSF .

Students’ Use and Perceptions of a Due Date Extension Policy

Students may sometimes benefit from due date extensions due to significant extenuating circumstances. These circumstances, though, may not be supported by documentation as required by university policies. Given this, I implemented a flexible due date policy called the Pause Button (PB) in a 16-week behavioral sciences research methods course. Students could use the PB to extend two due dates up to seven days each if they did not qualify for other university-approved extensions. Twenty-three students completed a survey about their use of the PB and perceptions of its helpfulness and benefits. Overall, students perceived the PB as very helpful, improving their ability to learn and complete greater quality coursework. Moreover, they perceived that it allowed them to better manage their academic workload, personal responsibilities, and stress. One of the PB’s perceived benefits, though, was dependent upon the assessment to which it was applied. This paper also suggests concerns for instructors who are considering implementation of a PB policy. It also contains recommendations for its use.

Cloze probability, predictability ratings, and computational estimates for 205 English sentences, aligned with existing EEG and reading time data

We release a database of cloze probability values, predictability ratings, and computational estimates for a sample of 205 English sentences (1726 words), aligned with previously released word-by-word reading time data (both self-paced reading and eye-movement records; Frank et al., Behavior Research Methods, 45(4), 1182–1190. 2013) and EEG responses (Frank et al., Brain and Language, 140, 1–11. 2015). Our analyses show that predictability ratings are the best predictors of the EEG signal (N400, P600, LAN) self-paced reading times, and eye movement patterns, when spillover effects are taken into account. The computational estimates are particularly effective at explaining variance in the eye-tracking data without spillover. Cloze probability estimates have decent overall psychometric accuracy and are the best predictors of early fixation patterns (first fixation duration). Our results indicate that the choice of the best measurement of word predictability in context critically depends on the processing index being considered.

A Bibliometric Analysis on Power Analysis Studies

The primary purpose of this study was to establish a theoretical framework for studies on power analysis conducted in the fields of education, psychology, and statistics for researchers. Therefore, the bibliometric characteristics of publications related to power analysis in the Web of Science database were analyzed using the Biblioshiny interface in the R programming language. The study identified influential studies on power analysis in education, psychology, and statistics. It also determined which concepts were associated with power analysis over the years and the authors and countries that contributed to the advancement of research regarding this concept. This research was conducted based on 515 studies that were included following specific criteria. It was found that the studies published between 1970 and 2023 were obtained from 183 sources, with a total of 1246 authors. There were 98 single-authored studies, and the number of co-authors per study was 2.88 on average. According to Bradford’s Law, Behavior Research Methods, Psychological Methods, and Multivariate Behavioral Research were the most productive journals concerning power analysis, taking up a larger proportion within the core sources compared to other journals. These journals were among the top three in terms of the number of publications, h-index, total number of citations, and publication rankings. These journals were followed by Structural Equation Modeling-A Multidisciplinary Journal, Frontiers in Psychology, and Educational and Psychological Measurement. An examination of studies on power analysis in education, psychology, and statistics according to Lotka's Law indicated that the relevant literature is insufficient and needs further development.

Parents' Emotional Responses to Behavior Analysis Terms: A Comparative Analysis.

Behavior analysts are concerned with developing strong client-therapist relationships. One challenge to the development of such relationships may be a reliance on technical language that stakeholders find unpleasant. Previous research suggests that some behavior analysis terms evoke negative emotional responses. However, most relevant research was conducted with individuals from the general public and not individuals with a history of interaction with behavior analysts. The current study evaluated how parents of individuals with disabilities, who accessed behavior analytic services for their child, rated their emotional responses to 40 behavior analysis terms. We found that half of behavior analysis terms were rated as less pleasant than the majority of English words by parents. Furthermore, word emotion ratings by our stakeholder sample corresponded closely to norms obtained from the general public (Warriner et al. Behavior Research Methods, 45(4), 1191-1207, 2013). Our findings suggest that, while learning history may mediate some emotional responses to words, published word emotion data could be a useful guide to how stakeholders may respond to behavior analysis terminology. A need remains for additional studies examining word emotion responses that may be unique to particular sub-categories of stakeholders and evaluating how emotional responses impact the development of effective relationships.