Within-person Consistency Research Articles

Careless responding detection revisited: Accuracy of direct and indirect measures.

To screen for careless responding, researchers have a choice between several direct measures (i.e., bogus items, requiring the respondent to choose a specific answer) and indirect measures (i.e., unobtrusive post hoc indices). Given the dearth of research in the area, we examined how well direct and indirect indices perform relative to each other. In five experimental studies, we investigated whether the detection rates of the measures are affected by contextual factors: severity of the careless response pattern, type of item keying, and type of item presentation. We fully controlled the information environment by experimentally inducing careless response sets under a variety of contextual conditions. In Studies 1 and 2, participants rated the personality of an actor that presented himself in a 5-min-long videotaped speech. In Studies 3, 4, and 5, participants had to rate their own personality across two measurements. With the exception of maximum longstring, intra-individual response variability, and individual contribution to model misfit, all examined indirect indices performed better than chance in most of the examined conditions. Moreover, indirect indices had detection rates as good as and, in many cases, better than the detection rates of direct measures. We therefore encourage researchers to use indirect indices, especially within-person consistency indices, instead of direct measures.

Within-Person Consistency of Acute Treatment Success With Ubrogepant: Results From a Long-term Safety Study (P2-2.003)

Within-Person Consistency of Acute Treatment Success With Ubrogepant: Results From a Long-term Safety Study (P2-2.003)

Exploring satisfaction with travel time profiles towards understanding intrinsic utilities of travel time

Although commuting is often viewed as a burden, transportation research has recently investigated satisfaction with commuting or commute travel times, with a goal to understand intrinsic utilities of travel time: when and for whom travel is productive or beneficial, beyond reaching a destination. However, few studies have examined perceived satisfaction with a range of travel times, and those that have only report aggregate trends. Thus, there is a need to illuminate and explain heterogeneity in travel time satisfaction profiles. By exploring individual-level responses to questions about satisfaction with hypothetical commute travel times among 588 commuters in Portland, Oregon, this study makes three contributions. First, through clustering, we identify eight different satisfaction with travel time profiles. Only one (the smallest group) has the same non-monotonic shape found in past research. Others are distinguished by different tolerable commute times: when (compared to an average/threshold commute time of 30 min) satisfaction levels drop. On this basis, we further categorize profiles into three groups: Monotonic low-tolerance (ML), Monotonic high-tolerance (MH), and Non-monotonic (NM) commuters. Second, through multinomial and ordered logit models, we determine factors associated with having a specific satisfaction profile or tolerable commute time. Significant differences are found for commute travel times (actual or ideal), mode choice (cycling), measures of intrinsic utility (satisfaction with actual commute times and the teleportation test), and a few personal characteristics (self-employment). Third, through visual comparisons, we find within-person consistency in responses to all of these questions about the intrinsic utility of travel time, supporting their use in past and future research.

Insufficient Effort Responding as a Potential Confound between Survey Measures and Objective Tests

Following research that demonstrates insufficient effort responding (IER) may confound survey measures and inflate observed correlations (Huang, Liu, & Bowling, 2015c), a question emerges as to whether and when IER can act as a confound between objective tests and surveys. Using data (N = 243) originally designed to examine training and transfer, study 1 demonstrates that (a) IER is negatively related to performance on tests, and (b) IER’s influence on surveys depends on the sample means of these measures. As a result, IER could inflate a test’s association with other tests and surveys. Study 2 investigates the impact of two parameters—within-person consistency of IER and percentage of IER cases in the sample—by randomly replacing bootstrapped attentive responses (10,000 bootstrapped samples of 200 cases identified from study 1). When predicting the confounding effects of IER, within-person consistency has positive linear and quadratic effects, percentage of IER cases has a positive linear effect, and consistency and percentage have a positive interactive effect. Research and practical implications for the design and evaluation of surveys and tests are discussed.

Subjective Confidence in the Response to Personality Questions: Some Insight Into the Construction of People's Responses to Test Items.

Drawing on research on subjective confidence, we examined how the confidence and speed in responding to personality items track the consistency and variability in the response to the same items over repeated administrations. Participants (N = 57) responded to 132 personality items with a true/false response format. The items were presented five times over the course of two sessions. Consistent with the Self-Consistency Model, the confidence and speed with which an item was endorsed at its first presentation predicted the likelihood of repeating that response across the subsequent presentations of the item, thus tracking test-retest reliability. Confidence and speed also predicted the likelihood that others will make the same response, thus tracking inter-person consensus. However, confidence and speed varied more strongly with within-person consistency than with inter-person consensus, suggesting some reliance on idiosyncratic cues in response formation. These results mirror, in part, findings obtained in other domains such as general knowledge, social attitudes, and personal preferences, suggesting some similarity in the decision processes underlying the response to binary items: responses to personality items are not retrieved ready-made from memory but constructed at the time of testing, based on the sampling of a small number of cues from a larger population of cues associated with the item’s content. Because confidence is based on the consistency with which the cues support a response, it is prognostic of within-person consistency and cross-person consensus. Theoretical and methodological implications are discussed.

Wisdom, bias, and balance: Toward a process-sensitive measurement of wisdom-related cognition.

[Correction Notice: An Erratum for this article was reported in Vol 115(6) of Journal of Personality and Social Psychology (see record 2018-63189-002). In the article, the original supplemental material has been revised to include a clarifying note to the Tests of model fit over larger sample (Samples C-G) section and post-peer review analyses added to the Post-peer review Factor Analytic Tests section. All versions of this article have been corrected.] Philosophers and behavioral scientists refer to wisdom as unbiased reasoning that guides one toward a balance of interests and promotes a good life. However, major instruments developed to test wisdom appear biased, and it is unclear whether they capture balance-related tendencies. We examined whether shifting from global, de-contextualized reports to state-level reports about concrete situations provides a less biased method to assess wise reasoning (e.g., intellectual humility, recognition of uncertainty and change, consideration of the broader context at hand and perspectives of others, integration of these perspectives or compromise), which may be aligned with the notion of balancing interests. Results of a large-scale psychometric investigation (N = 4,463) revealed that the novel Situated WIse Reasoning Scale (SWIS) is reliable and appears independent of psychological biases (attribution bias, bias blind spot, self-deception, and impression management), whereas global wisdom reports are subject to such biases. Moreover, SWIS scores were positively related to indices of living well (e.g., adaptive emotion regulation, mindfulness), and balancing of cooperative and self-protective interests, goals (influence-vs.-adjustment), and causal inferences about conflict (attribution to the self-vs.-other party). In contrast, global wisdom reports were unrelated or negatively related to balance-related measures. Notably, people showed modest within-person consistency in wise reasoning across situations or over time, suggesting that a single-shot measurement may be insufficient for whole understanding of trait-level wisdom. We discuss theoretical and practical implications for research on wisdom, judgment and decision making, well-being, and prosociality. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

"Wisdom, bias, and balance: Toward a process-sensitive measurement of wisdom-related cognition": Correction to Brienza et al. (2017).

Reports an error in "Wisdom, bias, and balance: Toward a process-sensitive measurement of wisdom-related cognition" by Justin P. Brienza, Franki Y. H. Kung, Henri C. Santos, D. Ramona Bobocel and Igor Grossmann (Journal of Personality and Social Psychology, Advanced Online Publication, Sep 21, 2017, np). In the article, the original supplemental material has been revised to include a clarifying note to the Tests of model fit over larger sample (Samples C-G) section and post-peer review analyses added to the Post-peer review Factor Analytic Tests section. All versions of this article have been corrected. (The following abstract of the original article appeared in record 2017-42043-001.) Philosophers and behavioral scientists refer to wisdom as unbiased reasoning that guides one toward a balance of interests and promotes a good life. However, major instruments developed to test wisdom appear biased, and it is unclear whether they capture balance-related tendencies. We examined whether shifting from global, de-contextualized reports to state-level reports about concrete situations provides a less biased method to assess wise reasoning (e.g., intellectual humility, recognition of uncertainty and change, consideration of the broader context at hand and perspectives of others, integration of these perspectives or compromise), which may be aligned with the notion of balancing interests. Results of a large-scale psychometric investigation (N = 4,463) revealed that the novel Situated WIse Reasoning Scale (SWIS) is reliable and appears independent of psychological biases (attribution bias, bias blind spot, self-deception, and impression management), whereas global wisdom reports are subject to such biases. Moreover, SWIS scores were positively related to indices of living well (e.g., adaptive emotion regulation, mindfulness), and balancing of cooperative and self-protective interests, goals (influence-vs.-adjustment), and causal inferences about conflict (attribution to the self-vs.-other party). In contrast, global wisdom reports were unrelated or negatively related to balance-related measures. Notably, people showed modest within-person consistency in wise reasoning across situations or over time, suggesting that a single-shot measurement may be insufficient for whole understanding of trait-level wisdom. We discuss theoretical and practical implications for research on wisdom, judgment and decision making, well-being, and prosociality. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Outcomes of early NIH-funded investigators: Experience of the National Institute of Allergy and Infectious Diseases.

Survival of junior scientists in academic biomedical research is difficult in today’s highly competitive funding climate. National Institute of Health (NIH) data on first-time R01 grantees indicate the rate at which early investigators drop out from a NIH-supported research career is most rapid 4 to 5 years from the first R01 award. The factors associated with a high risk of dropping out, and whether these factors impact all junior investigators equally, are unclear. We identified a cohort of 1,496 investigators who received their first R01-equivalent (R01-e) awards from the National Institute of Allergy and Infectious Diseases between 2003 and 2010, and studied all their subsequent NIH grant applications through 2016. Ultimately, 57% of the cohort were successful in obtaining new R01-e funding, despite highly competitive conditions. Among those investigators who failed to compete successfully for new funding (43%), the average time to dropping out was 5 years. Investigators who successfully obtained new grants showed remarkable within-person consistency across multiple grant submission behaviors, including submitting more applications per year, more renewal applications, and more applications to multiple NIH Institutes. Funded investigators appeared to have two advantages over their unfunded peers at the outset: they had better scores on their first R01-e grants and they demonstrated an early ability to write applications that would be scored, not triaged. The cohort rapidly segregated into two very different groups on the basis of PI consistency in the quality and frequency of applications submitted after their first R01-e award. Lastly, we identified a number of specific demographic factors, intitutional characteristics, and grant submission behaviors that were associated with successful outcomes, and assessed their predictive value and relative importance for the likelihood of obtaining additional NIH funding.

Distinct Methods, Biased Nonetheless: Insufficient Effort Responding as a Potential Confound

Following research that demonstrates insufficient effort responding (IER) may confound scores on survey measures (Huang, Liu, & Bowling, 2015), a question emerges as to whether and when IER can act as a confounding variable between objective tests (e.g., general cognitive ability) and survey measures (e.g., core self-evaluations). The present paper reports two studies that address this overall question. Study 1 (N = 243) demonstrated that (a) IER was negatively related to performance on objective tests, and (b) IER was related to survey measures depending on the means of these measures. As a result, IER could inflate an objective test’s associations with other object tests and survey measures. Using simulations, Study 2 examined the impact of two IER parameters – within-person consistency of IER and proportion of IER in the sample. When predicting the confounding effects of IER, within-person consistency had positive linear and quadratic effects, proportion of IER cases had a positive linear effect, and consistency and proportion had a positive interactive effect. In contrast, when predicting the attenuating effects of IER, consistency and proportion each had linear effect while their interactive effect strengthened each other.

Personality Consistency and Situational Influences on Behavior

This article examines within-person consistency in personality expression across situations as an individual difference variable that is distinct from the typically studied personality trait level. The focus of the study is the manner in which personality consistency (a conceptualization of personality strength) influences the choice and interpretation of situations and, ultimately, the enactment of organizational citizenship behavior. We conducted an experience sampling study of 167 employees over 10 workdays. At each survey, participants reported their conscientiousness, agreeableness, situation perceptions, and organizational citizenship behavior. Results demonstrated that even after controlling for the linear and quadratic effects of personality trait level (and several other variables): (1) personality consistency increased within-person consistency in organizational citizenship behavior across situations and (2) this relationship was partially mediated by perceived consistency of situational strength and trait-relevant situational content. More broadly, the findings show that individual differences in personality are not restricted solely to the personality trait level. Rather, within-person consistency in personality expression across situations is itself an important individual difference: one that possesses appreciable behavioral consequences in the workplace and one that, consequently, is deserving of considerable future research.

Illness Representations of Multiple Chronic Conditions and Self-Management Behaviors in Older Adults: A Pilot Study.

This article explores the intraindividual variability in illness representations of people with multiple chronic conditions and examines how representations of hypertension and arthritis are associated with self-management. Intraclass correlations determined the proportion of within-person variability in illness representations including Timeline, Consequences, Personal Control, and Timeline-Cyclical for 25 adults aged 64 and older. Within-person consistency across illnesses was present for Timeline and Timeline-Cyclical, but variability across illnesses in Personal Control and Consequences. Correlations revealed associations of diet, exercise, and sleep with illness representations of people with arthritis and hypertension. Representations of hypertension (Personal Control, Timeline-Cyclical, and Consequences) were associated with adherence to a reduced fat diet, walking, and total sleep time. Representations of arthritis were not associated with health behaviors. Findings demonstrate that clinical practice must consider the illness representations patients have about each of their chronic illnesses to begin to sustain positive self-management behaviors.

Comparing Average Level and Within-person Consistency in Clinical Disability Using AVP-825 vs. Sumatriptan Tablets: Applying Location Scale Mixed-effects Models (P2.166)

Comparing Average Level and Within-person Consistency in Clinical Disability Using AVP-825 vs. Sumatriptan Tablets: Applying Location Scale Mixed-effects Models (P2.166)

Personality Strength and Situational Influences on Within-Person Performance Variation

This article empirically examines the process by which personality strength (defined as the within-person consistency in personality trait expression across situations) influences job performance-i...

Justice Judgments: Individual Self-Insight and Between- and Within-Person Consistency

We use the method of policy capturing to address three open-ended questions regarding how people judge the fairness of events. First, do people differ in how they judge whether a situation is fair or unfair; second, are fairness judgments stable within-person; and, third, how much insight do people have into how they make fairness judgments? To investigate these questions, we used the method of policy capturing and a representative design that samples situations as well as participants. Forty-nine employees rated the global fairness of 56 performance appraisals sampled from their own organization (N = 2,744 situations), and regression methods were used to infer their judgment policy from their choices. We found that people differed greatly in how they judged fairness but used quite consistent policies across similar situations. Participants also provided self-reports of their judgment policies, and comparisons of these self-reports with actual policies revealed limited levels of self-insight.

Comparison of Average Weekly Pain Using Recalled Paper and Momentary Assessment Electronic Diary Reports in Children With Arthritis

The current study investigated the construct validity of a multidimensional pain diary for youth with juvenile idiopathic arthritis and also compared participants' responses on electronic and retrospective diary measures. The purpose of the latter part of this study was to compare absolute agreement, between-person and within-person consistency and judged change in weekly pain between these 2 methods of assessing pain. A total of 70 adolescents with juvenile idiopathic arthritis completed both weekly recalled and momentary reports of pain over a 2-week period and assessed their change in pain over the 2-week period using a 5-point global change in pain scale. The Pearson correlations and intraclass correlation coefficients were computed to demonstrate 3 different ways of comparing the measures on both between-person and within-person basis. Momentary ratings of pain episodes were consistently greater than weekly ratings of recalled pain. Moderate to strong consistency and agreement correlations were computed for between-person momentary and recalled pain intensity. However, these correlations were much weaker when the within-person data were analyzed. The judged change in pain across weeks was significantly associated with computed change in both average momentary and recalled pain. This is one of the few studies to explore the relationship between the measurement methods of pain recall and momentary assessment in adolescents. The poor within-person correlations observed have important implications for research design and practice in pediatric pain.

The construction of categorization judgments: Using subjective confidence and response latency to test a distributed model

The classification of objects to natural categories exhibits cross-person consensus and within-person consistency, but also some degree of between-person variability and within-person instability. What is more, the variability in categorization is also not entirely random but discloses systematic patterns. In this study, we applied the Self-Consistency Model (SCM, Koriat, 2012) to category membership decisions, examining the possibility that confidence judgments and decision latency track the stable and variable components of categorization responses. The model assumes that category membership decisions are constructed on the fly depending on a small set of clues that are sampled from a commonly shared population of pertinent clues. The decision and confidence are based on the balance of evidence in favor of a positive or a negative response. The results confirmed several predictions derived from SCM. For each participant, consensual responses to items were more confident than non-consensual responses, and for each item, participants who made the consensual response tended to be more confident than those who made the nonconsensual response. The difference in confidence between consensual and nonconsensual responses increased with the proportion of participants who made the majority response for the item. A similar pattern was observed for response speed. The pattern of results obtained for cross-person consensus was replicated by the results for response consistency when the responses were classified in terms of within-person agreement across repeated presentations. These results accord with the sampling assumption of SCM, that confidence and response speed should be higher when the decision is consistent with what follows from the entire population of clues than when it deviates from it. Results also suggested that the context for classification can bias the sample of clues underlying the decision, and that confidence judgments mirror the effects of context on categorization decisions. The model and results offer a principled account of the stable and variable contributions to categorization behavior within a decision-making framework.

Confidence in one’s social beliefs: Implications for belief justification

Philosophers commonly define knowledge as justified true beliefs. A heated debate exists, however, about what makes a belief justified. In this article, we examine the question of belief justification from a psychological perspective, focusing on the subjective confidence in a belief that the person has just formed. Participants decided whether to accept or reject a proposition depicting a social belief, and indicated their confidence in their choice. The task was repeated six times, and choice latency was measured. The results were analyzed within a Self-Consistency Model (SCM) of subjective confidence. According to SCM, the decision to accept or reject a proposition is based on the on-line sampling of representations from a pool of representations associated with the proposition. Respondents behave like intuitive statisticians who infer the central tendency of a population based on a small sample. Confidence depends on the consistency with which the belief was supported across the sampled representations, and reflects the likelihood that a new sample will yield the same decision. The results supported the assumption of a commonly shared population of representations associated with each proposition. Based on this assumption, analyses of within-person consistency and cross-person consensus provided support for the model. As expected, choices that deviated from the person's own modal judgment or from the consensually held judgment took relatively longer to form and were associated with relatively lower confidence, presumably because they were based on non-representative samples. The results were discussed in relation to major epistemological theories--foundationalism, coherentism and reliabilism.

Peripheral Blood CD4 T-Cell and Plasmacytoid Dendritic Cell (pDC) Reactivity to Herpes Simplex Virus 2 and pDC Number Do Not Correlate with the Clinical or Virologic Severity of Recurrent Genital Herpes

Leukocytes participate in the immune control of herpes simplex virus (HSV). Data from HIV coinfections, germ line mutations, and case reports suggest involvement of CD4 T cells and plasmacytoid dendritic cells (pDC). We investigated the relationships between these cells and recurrent genital herpes disease severity in the general population. Circulating CD4 T-cell responses to HSV-2 were measured in specimens from 67 immunocompetent individuals with measured genital lesion and HSV shedding rates. Similarly, pDC number and functional responses to HSV-2 were analyzed in 40 persons. CD4 responses and pDC concentrations and responses ranged as much as 100-fold between persons while displaying moderate within-person consistency over time. No correlations were observed between these immune response parameters and genital HSV-2 severity. Cytomegalovirus (CMV) coinfection was not correlated with differences in HSV-2-specific CD4 T-cell responses. The CD4 T-cell response to HSV-2 was much more polyfunctional than was the response to CMV. These data suggest that other immune cell subsets with alternate phenotypes or anatomical locations may be responsible for genital herpes control in chronically infected individuals.

The Construction of Attitudinal Judgments: Evidence from Attitude Certainty and Response Latency

Some researchers regard social attitudes as enduring predispositions, whereas others argue that they are constructed on the spot on the basis of accessible information. According to the sampling model proposed, certainty in one's attitude and the latency of forming that attitude track the online construction of the attitude and provide clues to the stable and variable contributions to attitude construction. An attitudinal judgment is based on a sample of accessible representations drawn from a large base of representations that are associated with the attitude object. Respondents behave like intuitive statisticians who infer the central tendency of a population on the basis of a small sample, and their certainty reflects the likelihood that a new sample will yield the same evaluation. Results on within-person consistency and cross-person consensus provided support for the model. Judgments that deviated from the person's own modal judgment or from the consensually held judgment took relatively longer to form and were associated with relatively lower confidence, presumably because they were based on non-representative samples. The effects of social consensus were found in the absence of any overt forces toward social conformity.

Leptin and Coronary Heart Disease: Prospective Study and Systematic Review

This study sought to better determine the link between leptin and coronary heart disease (CHD).Circulating leptin is considered a risk factor for CHD but larger studies are needed.Leptin levels were measured in 550 men with fatal CHD or nonfatal myocardial infarction and in 1,184 controls nested within a prospective study of 5,661 British men and set in context with a meta-analysis.Baseline leptin correlated with body mass index (BMI), blood pressure, total cholesterol, triglyceride, and inflammatory markers; correlations persisted after BMI adjustment. The within-person consistency of leptin values over 4 years (correlation coefficient: 0.79; 95% confidence interval [CI]: 0.73 to 0.83) was higher than those of some established cardiovascular risk factors. In a comparison of individuals in the top third with those in the bottom third of baseline leptin, the age- and town-adjusted odds ratio for CHD was 1.25 (95% CI: 0.96 to 1.62), decreasing to 0.98 (95% CI: 0.72 to 1.34) after adjustment for BMI. A systematic review identified 7 prospective reports with heterogeneous findings (I(2) = 60%, 13% to 82%). The combined adjusted risk ratio across all studies was 1.44 (95% CI: 0.95 to 2.16) in a comparison of extreme thirds of leptin levels. The inconsistency between studies was partially explained by sample size, with combined estimates from studies involving >100 CHD cases (1.28, 95% CI: 0.80 to 2.04) being somewhat weaker than those from smaller studies (1.81, 95% CI: 0.76 to 4.31).Previous studies appear to have overestimated associations of leptin and CHD risk. Our results suggest a moderate association that is largely dependent on BMI.