Instrument Relevance Research Articles

Development and Validation of a Self-Assessment Instrument for Measuring TPACK Ability of Scientific-Based Chemistry Teachers

This research aims to establish the validity and reliability of a novel self-assessment instrument designed to quantify the Technological Pedagogical Content Knowledge (TPACK) abilities of chemistry teachers implementing a science-based approach. The study employs a Research and Development (RD) framework, utilizing the ADDIE (Analysis, Design, Development, Implementation, and Evaluation) model. The development process involved two subject matter experts for initial validity testing, followed by a panel of 18 expert reviewers for further validity assessment, and a reliability testing phase involving 50 active chemistry teachers. Initial validity assessment yielded Aiken's V coefficient values of 0.92 and 0.96 for the two experts, demonstrating high agreement on the instrument's content validity. Further validation by the expert panel resulted in an average Aiken's V coefficient of 0.94, indicating excellent consensus on the instrument's relevance and applicability. Reliability testing exhibited a satisfactory level of consistency, with Aiken's V coefficient values ranging from 0.52 to 0.64 for the seven TPACK domains (PK, CK, TK, TCK, TPK, PCK, TPACK). Consequently, the developed self-assessment instrument exhibits both robust validity and reliability in evaluating TPACK abilities of science-based chemistry teachers. The findings recommend the instrument's utility in facilitating better understanding of TPACK abilities in this context, contributing to the body of research on pedagogical tool development and supporting professional development efforts among scientific-based chemistry educators.

Development and content validation of the medication literacy test for older adults (TELUMI)

ObjectivesMedication literacy comprises the skills needed to access, understand, and act on medication information. Low medication literacy is frequent among older adults, making them more susceptible to negative health outcomes. The aim of this study was to develop and evaluate the content validity of an instrument to measure medication literacy in this population. MethodMultistage methodological study. Items were generated according to a conceptual model and a literature review. The instrument was submitted to a panel of experts to assess item's relevance, clarity, and theoretical classification. Subsequently, the instrument was applied to the target population to assess its comprehensibility. ResultsContent Validity Index results indicated excellent agreement among experts on the instrument's relevance (0.98) and clarity (0.91). Overall agreement between experts regarding the theoretical subdimensions was moderate [Fleiss’ kappa=0.57 (p = 0.00)]. Items were reported to be fully understood by most representatives of the target population. The final version of the instrument had 33 items divided into eight scenarios of medication use. ConclusionThe instrument seems to have adequate content validity and good suitability for the target population. Therefore, it can help health professionals to identify older adults with low medication literacy, aiming to improve their medication use skills.

Development and testing of an instrument (HeSoEduCo) for health and social care educators' competence in professional education

BackgroundHealth and social care education is highly important for preparing future professionals for their future roles in sustainable health and social care. However, previous studies have emphasized that health and social care educators' competence is complex and poorly defined. Thus, there is a clear need for a psychometrically validated instrument to enable clarification and assessment of the required skills. ObjectiveTo develop and psychometrically validate an instrument (the HeSoEduCo) for assessing health and social care educators' competence in higher and professional education. DesignCross-sectional study. MethodsA HeSoEduCo instrument, with items inviting 1–4 Likert scale responses, was developed, based on one systematic review and one qualitative study, then validated in terms of face, content and construct validity and internal consistency (Cronbach's alpha values). All health and social care educators based in all 21 universities of applied sciences in Finland and seven vocational colleges were subsequently invited to participate in a large-scale application of the instrument in fall 2018. In total, responses of 390 of these educators are analyzed here.Results: The face and content validity of 71 newly developed items were assessed by experts in two evaluation rounds. The final content validity showed high scores for the instrument's relevance and clarity. Confirmatory factor analysis (to test construct validity) yielded eight factors (43-items remaining), defining the following competence areas of educators: evidence-based practice, digital collaborative learning, student-centered pedagogy, collaboration & societal, leadership & management, cultural & linguistic diversity, mentoring student into professional competence development and subject & curriculum. Cronbach's alpha values for the factors ranged from 0.70 to 0.89. ConclusionThe instrument can be used to obtain self-evaluations of educators' competence when assessing their general competence levels and help human resources departments and managers to identify suitable continuous education programs for their staff.

End user nonmalicious, counterproductive computer security behaviors: concept, development, and validation of an instrument

Employees' engagement in nonmalicious, counterproductive computer security behaviors (CCSB) poses a threat to organizations' information systems (IS) resources and assets. In order to understand CCSB, there is a need to propose theoretical foundations to research the phenomenon and to offer useful tools to help organizations assess such behaviors in their particular contexts. Relevant instruments that systematically measure or assess workers' participation in CCSB remain underdeveloped. This study proposes an instrument to assess engagement in CCSB. Confirmatory factor analysis confirmed three subscales of CCSB, that is, “careless use of IS resources,” “procrastinating carrying out required IS actions,” and “improper use of IS resources.” The instrument's relevance to research and practice is discussed and directions for future research are outlined.

A qualitative examination of the content validity of the EQ-5D-5L in patients with type 2 diabetes.

BackgroundThe EQ-5D is frequently used to derive utilities for patients with type 2 diabetes (T2D). Despite widely available quantitative psychometric data on the EQ-5D, little is known about content validity in this population. Thus, the purpose of this qualitative study was to examine content validity of the EQ-5D in patients with T2D.MethodsPatients with T2D in the UK completed concept elicitation interviews, followed by administration of the EQ-5D-5L and cognitive interviewing focused on the instrument’s relevance, clarity, and comprehensiveness.ResultsA total of 25 participants completed interviews (52.0 % male; mean age = 53.5 years). Approximately half (52 %) reported that the EQ-5D-5L was relevant to their experience with T2D. When asked if each individual item was relevant to their experience with T2D, responses varied widely (24.0 % said the self-care item was relevant; 68.0 % said the anxiety/depression item was relevant). Participants frequently said items were not relevant to themselves, but could be relevant to patients with more severe diabetes. Most participants (92.0 %) reported that T2D and/or its treatment/monitoring requirements had an impact on their quality of life that was not captured by the EQ-5D-5L. Common missing concepts included food awareness/restriction (n = 13, 52.0 %); activities (n = 11, 44.0 %); emotional functioning other than depression/anxiety (n = 8, 32.0 %); and social/relationship functioning (n = 8, 32.0 %).ConclusionsThe results highlight strengths and potential limitations of the EQ-5D-5L, including missing content that could be important for some patients with T2D. Suggestions for addressing limitations are provided.

Validation and Cross-Cultural Pilot Testing of Compliance with Standard Precautions Scale: Self-Administered Instrument for Clinical Nurses

To perform detailed psychometric testing of the compliance with standard precautions scale (CSPS) in measuring compliance with standard precautions of clinical nurses and to conduct cross-cultural pilot testing and assess the relevance of the CSPS on an international platform. A cross-sectional and correlational design with repeated measures. Nursing students from a local registered nurse training university, nurses from different hospitals in Hong Kong, and experts in an international conference. The psychometric properties of the CSPS were evaluated via internal consistency, 2-week and 3-month test-retest reliability, concurrent validation, and construct validation. The cross-cultural pilot testing and relevance check was examined by experts on infection control from various developed and developing regions. Among 453 participants, 193 were nursing students, 165 were enrolled nurses, and 95 were registered nurses. The results showed that the CSPS had satisfactory reliability (Cronbach α = 0.73; intraclass correlation coefficient, 0.79 for 2-week test-retest and 0.74 for 3-month test-retest) and validity (optimum correlation with criterion measure; r = 0.76, P < .001; satisfactory results on known-group method and hypothesis testing). A total of 19 experts from 16 countries assured that most of the CSPS findings were relevant and globally applicable. The CSPS demonstrated satisfactory results on the basis of the standard international criteria on psychometric testing, which ascertained the reliability and validity of this instrument in measuring the compliance of clinical nurses with standard precautions. The cross-cultural pilot testing further reinforced the instrument's relevance and applicability in most developed and developing regions.

Cross-Cultural Validation of the Bem Sex Role Inventory

Although the Bem Sex Role Inventory (BSRI) is one of the most popular and widely used measurements of psychological masculinity and femininity, its validity in cross-cultural settings remains unverified. To determine the instrument's relevance to the Asian milieu, the 60 BSRI items (20 masculine, 20 feminine, and 20 neutral) and 44 Asian-originated items were administered to 237 South Indian and 210 Malaysian university students, who were asked to rate the desirability of each characteristic for either males or females. Data for each item were analyzed by t-test in accordance with the Bem criteria for both Indian and Malaysian samples. Results revealed that only a small portion of the 104 items were differentiated as masculine or feminine and that there was considerable variation in the overall evaluations of neutral items. In this context, it was concluded that the BSRI is not a valid measurement of psychological androgyny in South India and Malaysia due to cross-cultural differences in M/ F constructs.

An analysis of the external validity and applied relevance of three rating formats

Three rating instruments (Behaviorally Anchored Rating Scales, Numerically Anchored Rating Scales, and Trait Ratings) were compared to an external criterion of police patrolman job performance. The external criterion was five vignettes of hypothetical patrolmen's job behavior based on an inclusive job analysis. Each of the five vignettes was rated by 28 first-level supervisors using each of the three instruments. Rater perceptions of each instrument's relevance for various organizational purposes were also gathered. Analysis of the three rating instruments centered on their respective criterion sensitivity. Results indicated that the three comparison instruments were approximately equal in terms of their resistance to the errors of leniency and central tendency. None of the three instruments was shown to be resistant to halo error. The NARS instrument was preferred for applied usage. Explanations for the results center on scale ambiguity, rater opportunities to observe behavior, and the nature of rater perceptions of performance.