Reaffirming construct validity: A response to methodological concerns.

This study investigated the application of the transtheoretical (stages-of-change) model in audiologic rehabilitation. More specifically, it described the University of Rhode Island Change Assessment (URICA) scores of adults with acquired hearing impairment. It reported the psychometric properties (construct, concurrent, and predictive validity) of the stages-of-change model in this population. At baseline, 153 adults with acquired hearing impairment seeking help for the first time completed the URICA as well as measures of degree of hearing impairment, self-reported hearing disability, and years since hearing impairment onset. Participants were subsequently offered intervention options: hearing aids, communication programs, and no intervention. Their intervention uptake and adherence were assessed 6 months later and their intervention outcomes were assessed 3 months after intervention completion. First, the stages-of-change construct validity was evaluated by investigating the URICA factor structure (principal component analysis), internal consistency, and correlations between stage scores. The URICA scores were reported in terms of the scores for each stage of change, composite scores, stages with highest scores, and stage clusters (cluster analysis). Second, the concurrent validity was assessed by examining associations between stages of change and degree of hearing impairment, self-reported hearing disability, and years since hearing impairment onset. Third, the predictive validity was evaluated by investigating associations between stages of change and intervention uptake, adherence, and outcomes. First, in terms of construct validity, the principal component analysis identified four instead of three stages (precontemplation, contemplation, preparation, and action) for which the internal consistency was good. Most of the sample was in the action stage. Correlations between stage scores supported the model. Cluster analysis identified four stages-of-change clusters, which the authors named active change, initiation, disengagement, and ambivalence. In terms of concurrent validity, participants who reported a more advanced stage of change had a more severe hearing impairment, reported greater hearing disability, and had a hearing impairment for a longer period of time. In terms of predictive validity, participants who reported a more advanced stage of change were more likely to take up an intervention and to report successful intervention outcomes. However, stages of change did not predict intervention adherence. The majority of the sample was in the action stage. The construct, concurrent, and predictive validity of the stages-of-change model were good. The stages-of-change model has some validity in the rehabilitation of adults with hearing impairment. The data support that change might be better represented on a continuum rather than by movement from one step to the next. Of all the measures, the precontemplation stage score had the best concurrent and predictive validity. Therefore, further research should focus on addressing the precontemplation stage with a measure suitable for clinical use.

The Credibility Revolution advances internally valid research designs intended to identify causal effects from quantitative data. The ensuing emphasis on internal validity, however, has enabled a neglect of construct and external validity. We show that ignoring construct and external validity within identification strategies undermines the Credibility Revolution’s own goal of understanding causality deductively. Without assumptions regarding construct validity, one cannot accurately label the cause or outcome. Without assumptions regarding external validity, one cannot label the conditions enabling the cause to have an effect. If any of the assumptions regarding internal, construct, and external validity are missing, the claim is not deductively supported. The critical role of theoretical and substantive knowledge in deductive causal inference is illuminated by making such assumptions explicit. This article critically reviews approaches to identification in causal inference while developing a framework called causal specification. Causal specification augments existing identification strategies to enable and justify deductive, generalized claims about causes and effects. In the process, we review a variety of developments in the philosophy of science and causality and interdisciplinary social science methodology.

It has been suggested that occupational therapy practitioners with a poor sense of professional identity experience more difficulties in the workplace and may be less likely to implement evidence-based, occupation-focused interventions. Poor sense of professional identity has also been linked to increased risk of burnout. Although professional identity is considered important, few measures are available and none have been rigorously evaluated. The project was established to evaluate the construct and internal validity of one such measure, the Professional Identity Questionnaire (PIQ). The PIQ includes 10 questions, each rated on a 5-point Likert scale. Rasch analysis was used to evaluate several aspects of construct and internal validity of the PIQ. These included rating scale validity, unidimensionality, statistical validity, differential item functioning, overall construct validity and person response validity. Correlations were also calculated to explore the relationship between the Professional Identity Questionnaire and a commonly used measure of burnout, the Oldenburg Burnout Inventory (OLBI). A total of 118 occupational therapists working in mental health completed the PIQ and OLBI. Initial analyses suggested that the rating scale validity and targeting of item difficulties to person abilities were not optimal. Revision of the rating scale structure improved the overall rating scale validity and item targeting. Analyses using the revised scale structure showed generally good alignment with quality control criteria established for this study. Fit statistics for one item were less than optimal and some differential item functioning was present. Correlations between PIQ and OLBI total scores were -0.53 for the original PIQ rating scale structure and -0.46 for the revised rating scale structure. Results from this study provide preliminary evidence for the construct and internal validity of the PIQ, especially when using the revised scale structure. However, further research is required before the PIQ can be recommended for widespread use.

This article presents a definition of constructs, construct validity, and wn- struct validation and traces some of the historical treatments of the terms. It is argued that construct validity and validation procedures can be particularly useful in personnel-testing contexts, where jobrelatedness is required. Several examples are presented using sophisticated modeling procedures along with other kinds of data to illustrate how evidence to show job relatedness through construct validation might be developed. Several cautions are extended re- garding the use of these statistical modeling procedures.

PurposeBowel complaints are very common among patients with colorectal cancer. However, the most used questionnaires for colorectal cancer survivors do not comprehensively comprise bowel symptoms. This study aimed to examine construct and criterion validity, as well as internal consistency, of the Chilean Version of the International Consultation on Incontinence Questionnaire Bowel Module (ICIQ-B) among people with colorectal cancer.MethodsCross-sectional, validation study performed with 106 colorectal cancer patients from Hospital del Salvador, Chile. Bowel function was assessed with the ICIQ-B. Construct validity was assessed with confirmatory factor analysis and hypothesis testing. Specific items of a quality-of-life questionnaire (EORTC QLQ-CR29) were used to correlate with similar ICIQ-B items for criterion validity. For internal consistency, Cronbach’s alpha was computed.ResultsFor construct validity, the confirmatory factor analysis showed that the three factors model did not fit our data. Meanwhile, hypothesis testing favored the construct validity of the instrument, considering that rectal cancer patients showed worse bowel pattern (p = 0.001), bowel control (p = 0.001) and quality of life (p < 0.001) scores compared to colon cancer patients. In addition, those patients assessed before surgery also presented worse scores bowel control (p = 0.023) and quality of life (p = 0.009) compared to post-surgical patients. Regarding criterion validity, the ICIQ-B items showed a significant correlation with similar QLQ-CR29 items. The internal reliability of the instrument was good (Cronbach’s α = 0.909).ConclusionConsidering that this questionnaire appraises bowel function in more depth, it is recommended for use in clinical practice and research with colorectal cancer patients.

In a previous study, we developed the Health Information Technology Usability Evaluation Scale (Health-ITUES), which is designed to support customization at the item level. Such customization matches the specific tasks/expectations of a health IT system while retaining comparability at the construct level, and provides evidence of its factorial validity and internal consistency reliability through exploratory factor analysis. In this study, we advanced the development of Health-ITUES to examine its construct validity and predictive validity. The health IT system studied was a web-based communication system that supported nurse staffing and scheduling. Using Health-ITUES, we conducted a cross-sectional study to evaluate users' perception toward the web-based communication system after system implementation. We examined Health-ITUES's construct validity through first and second order confirmatory factor analysis (CFA), and its predictive validity via structural equation modeling (SEM). The sample comprised 541 staff nurses in two healthcare organizations. The CFA (n=165) showed that a general usability factor accounted for 78.1%, 93.4%, 51.0%, and 39.9% of the explained variance in 'Quality of Work Life', 'Perceived Usefulness', 'Perceived Ease of Use', and 'User Control', respectively. The SEM (n=541) supported the predictive validity of Health-ITUES, explaining 64% of the variance in intention for system use. The results of CFA and SEM provide additional evidence for the construct and predictive validity of Health-ITUES. The customizability of Health-ITUES has the potential to support comparisons at the construct level, while allowing variation at the item level. We also illustrate application of Health-ITUES across stages of system development.

OBJECTIVE To develop and validate a simulation model for laparoscopic ovariectomy in standing horses. DESIGN Prospective cohort study. SAMPLE 15 third-year veterinary students and 4 equine surgeons with experience in laparoscopy. PROCEDURES A simulation model that mimicked laparoscopic ovariectomy in standing horses was developed. Face validity of the model was determined with a questionnaire completed by the equine surgeons. Construct validity was determined by comparing performance scores (based on time to completion and accuracy completing various operative tasks) for simulated laparoscopic ovariectomy performed in the model for the students with scores for the equine surgeons. Concurrent validity was assessed by comparing performance scores with scores obtained with the validated McGill Inanimate System for Training and Evaluation of Laparoscopic Skills (MISTELS). RESULTS Questionnaire responses indicated that the simulation model replicated the operative experience to a high degree (face validity). Performance scores for simulated laparoscopic ovariectomy performed in the model were significantly different between the students and the equine surgeons (construct validity). Performance scores for the simulation model were significantly correlated with scores for the MISTELS (concurrent validity). CONCLUSIONS AND CLINICAL RELEVANCE Results suggested that the simulation model had face, construct, and concurrent validity, suggesting that it may be useful when training students to perform laparoscopic ovariectomy in standing horses.

No single cognitive screen adequately captures the cognitive domains needed for inpatient occupational therapy treatment planning. To assess the construct validity of the Gaylord Occupational Therapy Cognitive (GOT-Cog©) screen, a novel comprehensive cognitive screen that evaluates functional cognition. Randomized crossover controlled study design using the St. Louis University Mental Status (SLUMS) exam as a comparator. Long-term acute-care hospital. Participants were inpatients admitted to Gaylord Hospital who were ages 18 yr or older, prescribed occupational therapy services, with no documented history of dementia, Alzheimer's, or preexisting intellectual disability and no present aphasia. During participants' initial occupational therapy evaluation, either the SLUMS or GOT-Cog were randomly delivered; the screen that was not delivered on admission was delivered 22 to 26 hr later by the same or a different clinician. GOT-Cog and SLUMS total scores and individual item and domain scores. Ninety-eight participants yielded sufficient data for analysis. Total GOT-Cog and SLUMS scores positively correlated (p < .0001). All shared domains between GOT-Cog and SLUMS were significantly correlated (p ≤ .0155); similarly, all unique domains showed significant correlations with both GOT-Cog and SLUMS total scores (p ≤ .0194). No ordering effects were observed (p ≥ .8081). Despite having 11 more items, GOT-Cog took only 6 min longer to complete (10 vs. 16 min; p < .0001). Both demonstrated adequate internal consistency. The GOT-Cog has overall strong construct and criterion validity. Going forward, we will evaluate the rater reliability and responsiveness of the GOT-Cog. Plain-Language Summary: Occupational therapists evaluate clients' cognitive strengths and limitations in relation to activities of daily living and instrumental activities of daily living. Occupational therapists use this evaluation to help clients identify strategies to adapt to their specific environments, support their independence, and improve their ability to perform tasks. No cognitive screen currently exists that adequately evaluates a person's cognitive domains as part of treatment planning for inpatient occupational therapy. This study reviewed the construct validity, criterion validity, and internal consistency of the Gaylord Occupational Therapy Cognitive screen (GOT-Cog), a new comprehensive cognitive screen. The GOT-Cog was used with inpatients at a long-term acute-care hospital as part of their initial occupational therapy evaluation. The study found that the GOT-Cog has overall strong construct and criterion validity. Future studies will evaluate the interrater and intrarater reliability and responsiveness of this new cognitive screen.

Introduction Theory question papers form an important part of assessment in medical education. As per the Competency-Based Medical Education (CBME) guidelines 2019, questions should test higher levels of cognition. This pilot study analyzes 60 question papers from different universities in Gujarat for their construct and content validity. The aim was to analyze the quality of physiology question papers from various medical universities in Gujarat to gain insights into assessment quality and its alignment with the CBME guidelines. The objectives were twofold: to evaluate the "construct validity" and "content validity" of these physiology theory question papers over the past three years according to the CBME standards. Methods An observational study using a cross-sectional records-based approach was carried out, evaluating 60 summative exam question papers in physiology from eight different universities of Gujarat for their construct and content validity. Using Bloom's taxonomy, the learning level of the cognitive domain for the questions asked was assessed. The findings compared and displayed a sample of papers. Results A total of 1842 questions were analyzed from the 60 question papers of eight different universities of the Gujarat state. The study found that the questions asked for different levels of cognition in Bloom's taxonomy, i.e., remember, understand, apply, analyze, evaluate, and create, were 560 (30.40%), 434 (23.26%), 222 (12.05%), 118 (6.41%), 94 (5.10%), and 0.00%, respectively. A total of 414 (22.48%) questions did not have any verb, so they did not fit into any level of Bloom's taxonomy. The majority of questions (1773, 96.25%) were asked from the core competencies, while a small percentage (69, 3.75%) of questions were asked from the non-core competencies of physiology. Conclusion The majority of questions in the summative question papers in physiology were of level "remember" and "understand" as per Bloom's taxonomy. Of the questions, 26% did not have any verb. There is a need to incorporate more questions testing higher levels of cognition and to use blueprints by universities. Faculty training is also necessary to bring about course correction.

In Nigeria, the severe shortage of registered nurses (RNs) and Registered midwives (RMs) have made the Nigerian government carefully propose a policy to prolong the health personnel working lives. Conversely, little is known about the predictors that influence Intention and actual preparation to extend work life among the selected professionals. Therefore, this pilot study's objective was to evaluate a small sample data to establish the reliability and validity of the constructs' measuring instruments under study. This study recruited respondents 40 years or older by sending recruitment e-mails to them, which only 50 questionnaires were usable. The validity and reliability of the instruments were analyzed using SPSS v24 and smartPLS v2.0.The study result shows that the entire adapted constructs were reliable and valid and appropriate for the upcoming main study. Precisely, composite reliability (CR) indicated values of 0.90, 0.89, 0.89, 091, and 0.92 for the constructs of attitude, Intention, perceived behavioral control (PBC), preparation, and subjective norms, respectively, which are all above the threshold value of 0.70.Similarly, the average variance extracted (AVE) and discriminant validity were used to determine its validity. The AVE of the constructs of attitude, intention, PBC, preparation, and subjective norm are 0.82, 0.74, 0.63, 0.73, and 0.74, respectively, which indicated all the values are above the suggested threshold of 0.5. The discriminant validity of the entire constructs was attained because the AVE's square root for the adapted constructs was more significant than the correlation among other reflective constructs, as specified by the studies embolden values. The current study's main limitation is that some nurses and midwives could not respond to the e-mailed questionnaire sent to them; only a few respondents completed the questionnaires. Thus, monetary rewards should be offered to the respondents to cover the cost of data.

Consumer behavior research and neurology are combined in the emerging discipline of neuromarketing. Neuromarketing is considered to be one of emerging field to study how consumer's brain reacts to advertisement and other brand's message by observing brainwave, eye, and skin response. The current study examined the emerging field of constructs of neuromarketing like social, attention, technology, and emotions to examine Indian consumer's buying behavior. To study the validity and reliability of constructs of neuromarketing that examines consumer's buying behavior among Indian consumers. A sample of 191 people of different age groups was considered in the study. A random sampling technique was used for data collection. The self-designed questionnaire used for the measurement of neuromarketing constructs and consumers' buying behavior. The current study applied SPSS and AMOS software to validate the measurement model of neuromarketing. The Kaiser-Meyer-Olkin (KMO) and Bartlett's Test's value is 0.784 and this value confirmed that the sample is adequate for factor analysis. Apart from that, the five constructs of neuromarketing - Attention (A), Social (SC), Technological (T), Emotion (E), and Consumer Buying Behavior (BB) had shown the value of Cronbach's alpha to be more than 0.7. Confirmatory Factor Analysis (CFA) had shown value of average variance explained of each constructs 0.5 and composite reliability more than 0.7 which indicates excellent construct validity of constructs for model formation of neuromarketing. The study also validates measurement research model of neuromarketing on the basis of model fit index (chi-square/df = 3.397, RMSEA = 0.10, GFI = 0.92, and CFI = 0.87). The present study had shown good validity and reliability of constructs of neuromarketing and also proved that marketers can apply these constructs to examine behavior pattern of consumers.

MP73-19 URETEROSCOPY SIMULATION - A SYSTEMATIC REVIEW OF CURRENT TRAINING MODALITIES

Virtual reality simulators used in the education of orthopaedic residents often lack realistic haptic feedback. To solve this, the (Practice Arthroscopic Surgical Skills for Perfect Operative Real-life Treatment) PASSPORT simulator was developed, which was subjected to fundamental changes: improved realism and user interface. The purpose was to demonstrate its face and construct validity. Thirty-one participants were divided into three groups having different levels of arthroscopic experience. Participants answered questions regarding general information and the outer appearance of the simulator for face validity. Construct validity was assessed with one standardized navigation task, which was timed. Face validity, educational value and user-friendliness were determined with two representative exercises and by asking participants to fill out the questionnaire. A value of 7 or greater was considered sufficient. Construct validity was demonstrated between experts and novices. Median task time for the fifth trial was 55s (range 17-139s) for the novices, 33s (range 17-59s) for the intermediates, and 26s (range 14-52s) for the experts. Median task times of three trials were not significantly different between the novices and intermediates, and none of the trials between intermediates and experts. Face validity, educational value and user-friendliness were perceived as sufficient (median >7). The presence of realistic tactile feedback was considered the biggest asset of the simulator. Proper preparation for arthroscopic operations will increase the quality of real-life surgery and patients' safety. The PASSPORT simulator can assist in achieving this, as it showed construct and face validity, and its physical nature offered adequate haptic feedback during training. This indicates that PASSPORT has potential to evolve as a valuable training modality.

Validation of the Advanced Scope Trainer for Flexible Ureterorenoscopy Training

Development and Validation of a Novel Robotic Procedure Specific Simulation Platform: Partial Nephrectomy