Establish Validity Evidence Research Articles

An Evaluation of Items and Dimensional Structure of a Scale to Measure Teachers’ Professional Commitment in Thailand

The purpose of this study was to examine the items and the dimensionality of a scale to measure professional commitment of teachers in Thailand. Data were collected by in-depth interview and questionnaire. The content analysis was conducted with the qualitative data for the purpose of item writing. Differential item functioning (DIF), exploratory factor analysis (EFA), confirmatory factor analysis (CFA), and bi-factor CFA model analysis were conducted to establish validity evidence. As a result of EFA and CFA, the study showed that a 3-dimensional scale can effectively measure professional commitment of teachers, which was achieved by the 18-item measure. A polytomuos item response theory model was also fitted to estimate item parameters and to examine the test information function.

Validity of a cross-specialty test in basic laparoscopic techniques (TABLT)

BackgroundBox trainer systems have been developed that include advanced skills such as suturing. There is still a need for a portable, cheap training and testing system for basic laparoscopic techniques that can be used across different specialties before performing supervised surgery on patients. The aim of this study was to establish validity evidence for the Training and Assessment of Basic Laparoscopic Techniques (TABLT) test, a tablet‐based training system.MethodsLaparoscopic surgeons and trainees were recruited from departments of general surgery, gynaecology and urology. Participants included novice, intermediate and experienced surgeons. All participants performed the TABLT test. Performance scores were calculated based on time taken and errors made. Evidence of validity was explored using a contemporary framework of validity.ResultsSome 60 individuals participated. The TABLT was shown to be reliable, with an intraclass correlation coefficient of 0·99 (P < 0·001). ANOVA showed a difference between the groups with different level of experience (P < 0·001). The Bonferroni correction was used to confirm this finding. A Pearson's r value of 0·73 (P < 0·001) signified a good positive correlation between the level of laparoscopic experience and performance score. A reasonable pass–fail standard was established using contrasting groups methods.Conclusion TABLT can be used for the assessment of basic laparoscopic skills and can help novice surgical trainees in different specialties gain basic laparoscopic competencies.

Linking simulation-based educational assessments and patient-related outcomes: a systematic review and meta-analysis.

To examine the evidence supporting the use of simulation-based assessments as surrogates for patient-related outcomes assessed in the workplace. The authors systematically searched MEDLINE, EMBASE, Scopus, and key journals through February 26, 2013. They included original studies that assessed health professionals and trainees using simulation and then linked those scores with patient-related outcomes assessed in the workplace. Two reviewers independently extracted information on participants, tasks, validity evidence, study quality, patient-related and simulation-based outcomes, and magnitude of correlation. All correlations were pooled using random-effects meta-analysis. Of 11,628 potentially relevant articles, the 33 included studies enrolled 1,203 participants, including postgraduate physicians (n = 24 studies), practicing physicians (n = 8), medical students (n = 6), dentists (n = 2), and nurses (n = 1). The pooled correlation for provider behaviors was 0.51 (95% confidence interval [CI], 0.38 to 0.62; n = 27 studies); for time behaviors, 0.44 (95% CI, 0.15 to 0.66; n = 7); and for patient outcomes, 0.24 (95% CI, -0.02 to 0.47; n = 5). Most reported validity evidence was favorable, though studies often included only correlational evidence. Validity evidence of internal structure (n = 13 studies), content (n = 12), response process (n = 2), and consequences (n = 1) were reported less often. Three tools showed large pooled correlations and favorable (albeit incomplete) validity evidence. Simulation-based assessments often correlate positively with patient-related outcomes. Although these surrogates are imperfect, tools with established validity evidence may replace workplace-based assessments for evaluating select procedural skills.

Using Performance Assessments to Determine Competence in Clinical Athletic Training Education: How Valid Are Our Assessments?

ContextValidity arguments can be used to provide evidence that instructors are drawing accurate conclusions from the results of students' clinical performance assessments (PAs). Little research has been conducted in athletic training education to determine if the evidence supports the use of current PAs. Measurement theories designed to provide this evidence can be confusing and unfamiliar to athletic training educators.ObjectiveThe purpose of this article is to present contemporary concepts of validity and suggest approaches athletic training educators can use to offer evidence to support the best assessment methods.BackgroundEducators often use PAs to determine a student's competence for professional practice. Competence is a complex concept that is difficult to define clearly, thus making assessments of competent performance difficult as well. Most methods of PA used in athletic training education can be classified into 2 general approaches: behavioral and holistic. Athletic training educators, in an attempt to develop effective, appropriate, and user-friendly PAs to evaluate students, may be measuring skill but not truly measuring competence.DescriptionModern validity concepts focus on the interpretations and meanings of assessment scores, not just on the characteristics of the test itself. Using an updated concept of validity can guide the development of competence PAs to determine if educational outcomes are being met. A framework for developing a validity argument is presented.ConclusionsValidity can be used to provide a simple, but rational, defense of what clinical educators do. Knowing the process of establishing validity evidence will help educators revise PAs and educational standards to further promote the profession.

Board 315 - Research Abstract Simulation-Based Educational Assessments as Surrogates for Patient-Related Outcomes

Introduction/Background Simulation-based assessments could serve as surrogates for workplace based assessments in medical education, decreasing the potential for patient harm and conserving resources by demonstrating learner competence in a safe training environment. While over reliance on surrogates should be avoided, once the necessary links are established educators and researchers can use the surrogate simulation-based measures during day-to-day operations and initial investigations while reserving use of patient-related outcomes for selected situations. Our objective in conducting this review was to synthesize studies evaluating technology enhanced simulation-based outcomes (defined as skills) as surrogates for patient-related outcomes (defined as provider behaviors and patient effects) in healthcare education. Methods We systematically searched key databases through February 26, 2013. Included studies involved assessment of health professionals using technology enhanced simulation and reported associations of these scores with scores from assessments during real patient care (e.g., workplace-based assessments). We distinguished patient-related outcomes of process behaviors (e.g., instructor ratings of performance), time behaviors (e.g., procedure time) and effects on patients (e.g., complications). Working independently and in duplicate, we extracted data on participants, topics, validity evidence, study quality and magnitude of correlation. We pooled correlations using random effects meta-analysis. Results We identified 33 studies with 1,203 participants including postgraduate physicians, practicing physicians, medical students, dentists and nurses. Topics included laparoscopic surgery, anesthesiology and bedside procedures. Pooled correlations with simulation-based outcomes were 0.51 (95% CI, 0.38-0.62; N=27 studies) for process behaviors, 0.44 (95% CI, 0.15-0.66; N=7) for time behaviors and 0.24 (95% CI, -0.02 to 0.47; N=5) for patient effects. Most reported validity evidence was favorable, though incomplete. Three tools (OSATS, GOALS, and FLS), each evaluated in multiple studies, demonstrated large pooled correlations and favorable validity evidence. Conclusion Simulation-based assessments typically correlate positively with patient-related outcomes. While surrogates are imperfect, some tools with established validity evidence may replace workplace based assessment for select procedural skills. Our findings have implications for improving research practices when establishing simulation-based outcomes as surrogates in competency-based curricula (e.g., ACGME milestones). Disclosures None.

Board 358 - Research AbstractVirtual-Reality Simulator Training is More Efficient than Apprenticeship Training in the First Part of the Learning Curve

Introduction/Background Endosonography has proven superior to traditional Methods for obtaining tissue samples from the mediastinum and has now become the method of choice for staging lung cancer.1-3 However, the technique is highly operator dependent and focus on training and certification is essential before Results from expert centers can be generalized. Virtual-reality (VR) simulation are superior to no training but studies should instead compare VR training to traditional training.4 Outcomes should be measured on real patients (transfer studies) using assessment tools with established validity evidence from similar settings. Several procedures must be assessed by multiple raters in a blinded fashion to ensure reliability and reduce the risk of bias.5 Credible standards are necessary for summative assessment. The aims of this study were to determine whether simulator training could replace practicing on patients. Secondly, to explore a reliable assessment method and set a credible pass/fail standard. Methods A twelve item EndoBronchial UltraSound Assessment Tool (EBUSAT) was created by experts in endosonography and assessment. Each item was scored from 0 to 4 points giving a maximum score of 48 points. An international volunteer sample of 16 respiratory physicians without EBUS experience and a convenience sample of two EBUS experts were included in the study. The respiratory physicians attended an eight hour theoretical course and participated in two EBUS procedures on patients. Then they were randomized to a standardized, individual four hour training program on a virtual-reality EBUS simulator, administered by the same operator in both countries vs. a half-day, standard, hands-on training on patients supervised by one of two international EBUS experts. Each physician performed three consecutive EBUS-TBNA procedures. The endoscopic and the ultrasound videos were recorded and assessed blindly and independently by three EBUS experts using the EBUSAT. Ten consecutive procedures by EBUS experts were assessed simultaneously to gather validity evidence. Procedures performed by simulator-trained novices, apprenticeship-trained novices and endosonography experts respectively were compared using independent samples t-tests. Reliability was explored using Generalizability Theory (a two-facet design with participants crossed with procedures crossed with raters). The contrasting group’s method was used to set a credible pass/fail-standard (secondary objective). Results The simulator group performed significantly better than the standard training group; 24.2 points vs. 20.1 points, p=0.006. The EBUSAT scores were very reliable; generalizability coefficient=0.85. An acceptable Generalizability Coefficient for high-stakes assessment of 0.8 could be achieved by using three raters, assessing two procedures and two raters assessing three procedures, or one rater assessing six procedures. The experts scored significantly higher than the novices indicating that the EBUSAT possess construct validity; 35.3 points, p<0.001. A credible pass/fail standard could be established = 28.2 points. 4% of the procedures performed by the standard training group passed vs. 17% in the simulation group. Only one of the procedures performed by the experts failed to meet standards (90% passed). Conclusion EBUS training on a virtual-reality simulator is more efficient than standard apprenticeship training in the initial part of the learning curve but subsequent supervised training on patients are necessary to ensure basic competency. The EBUSAT can be used to assess performances in an unbiased, reliable and valid way. A credible standard set by an accepted method make it possible to perform high-stakes certification using the EBUSAT provided that the number of raters and procedures assessed ensures sufficient reliability. Focus on patient safety and competency based education makes virtual-reality training and assessment of competence very relevant issues in the implementation of endosonography.

Fundamentals of Endoscopic Surgery cognitive examination: development and validity evidence

Flexible endoscopy is an integral part of surgical care. Exposure to endoscopic procedures varies greatly in surgical training. The Society of American Gastrointestinal and Endoscopic Surgeons has developed the Fundamentals of Endoscopic Surgery (FES), which serves to teach and assess the fundamental knowledge and skills required to practice flexible endoscopy of the gastrointestinal tract. This report describes the validity evidence in the development of the FES cognitive examination. Core areas in the practice of gastrointestinal endoscopy were identified through facilitated expert focus groups to establish validity evidence for the test content. Test items then were developed based on the content areas. Prospective enrollment of participants at various levels of training and experience was used for beta testing. Two FES cognitive test versions then were developed based on beta testing data. The Angoff and contrasting group methods were used to determine the passing score. Validity evidence was established through correlation of experience level with examination score. A total of 220 test items were developed in accordance with the defined test blueprint and formulated into two versions of 120 questions each. The versions were administered randomly to 363 participants. The correlation between test scores and training level was high (r = 0.69), with similar results noted for contrasting groups based on endoscopic rotation and endoscopic procedural experience. Items then were selected for two test forms of 75 items each, and a passing score was established. The FES cognitive examination is the first test with validity evidence to assess the basic knowledge needed to perform flexible endoscopy. Combined with the hands-on skills examination, this assessment tool is a key component for FES certification.

Setting Standards for English Foreign Language Assessment: Methodology, Validation, and a Degree of Arbitrariness

Cut‐scores were set by expert judges on assessments of reading and listening comprehension of English as a foreign language (EFL), using the bookmark standard‐setting method to differentiate proficiency levels defined by the Common European Framework of Reference (CEFR). Assessments contained stratified item samples drawn from extensive item pools, calibrated using Rasch models on the basis of examinee responses of a German nationwide assessment of secondary school language performance. The results suggest significant effects of item sampling strategies for the bookmark method on cut‐score recommendations, as well as significant cut‐score judgment revision over cut‐score placement rounds. Results are discussed within a framework of establishing validity evidence supporting cut‐score recommendations using the widely employed bookmark method.

Casting the First Stone of Validity Standards: A Less Critical Perspective of the MSCEIT

This comment responds to Maul’s (2012) article evaluating the validity evidence and argument for the Mayer–Salovey–Caruso Emotional Intelligence Test (MSCEIT) as a measure of emotional intelligence (EI). We suggest that Maul’s standards for establishing validity evidence are unrealistically high, and may not be met by other established psychometric tests. As an example, we show that evidence for the validity of Raven’s Progressive Matrices (RPM) is of a similar standard to the MSCEIT.

Development and Psychometric Evaluation of Scales that Assess Stigma Associated With Illicit Drug Users

This study established validity evidence for scales that assess perceived public stigma and stigmatization of illicit drug use. These concepts were measured with respect to five commonly used drugs: marijuana, powder cocaine, ecstasy, and nonmedical use of opioids and amphetamine. Data were collected from a diverse sample of 1,048 emerging adults in New York City in 2009. Exploratory and confirmatory factor analyses suggested two distinct factors, which were inversely related to exposure to users and recent use of each drug. These measures demonstrated good criterion, construct, and incremental validity and effectiveness in analyzing predictors of use. Study limitations were discussed.

Validation of a Talking Pedometer for Adults with Visual Impairment

Alterations in gait mechanics and mobility aid (MA) use have been observed in persons with visual impairment (VI) in response to environmental changes, yet the influence of these modifications on the accuracy of an adaptive pedometer has not been documented. The purpose of this study was to establish validity evidence for the Centrios talking pedometer relative to environmental familiarity and MA use in adults with VI. Thirteen adults with VI (age = 38 ± 14 yr) completed two walking trials over an unfamiliar, quarter-mile course while wearing a Centrios talking pedometer at the right and left sides of the hip. Walking speed, pedometer-determined steps, and actual steps were recorded during the first session, reflecting walking in an "unfamiliar environment." After a series of additional walks over the same course, outcome measures were reassessed during a second trial, reflecting walking in a "familiar environment." Absolute percent error (APE) scores were calculated between actual and pedometer-determined steps. Paired t-tests were used to assess differences in APE relative to mounting position across environmental settings. During unfamiliar trials, the pedometer accurately reported steps when mounted at the hip opposite the user's MA (APE = 2.1%) but was significantly less accurate when mounted at the hip on the user's MA side (APE = 11.1%). In the familiar setting, the pedometer accurately reported steps when mounted at the left hip and the hip opposite the user's MA (APE <3%). APE values did not differ across environmental conditions (P > 0.05). In unfamiliar and familiar walking conditions, the Centrios pedometer accurately monitors step-based activity in adults with VI when mounted at the hip opposite the user's MA.

Validation Is Like Motor Oil: Synthetic Is Better

Although synthetic validation has long been suggested as a practical and defensible approach to establishing validity evidence, synthetic validation techniques are infrequently used and not well understood by the practitioners and researchers they could most benefit. Therefore, we describe the assumptions, origins, and methods for establishing validity evidence of the two primary types of synthetic validation techniques: (a) job component validity and (b) job requirements matrix. We then present the case for synthetic validation as the best approach for many situations and address the potential limitations of synthetic validation. We conclude by proposing the development of a comprehensive database to build prediction equations for use in synthetic validation of jobs across the U.S. economy and reviewing potential obstacles to the creation of such a database. We maintain that synthetic validation is a practically useful methodology that has great potential to advance the science and practice of industrial and organizational psychology.

Validation Is Like Motor Oil: Synthetic Is Better

Although synthetic validation has long been suggested as a practical and defensible approach to establishing validity evidence, synthetic validation techniques are infrequently used and not well understood by the practitioners and researchers they could most benefit. Therefore, we describe the assumptions, origins, and methods for establishing validity evidence of the two primary types of synthetic validation techniques: (a) job component validity and (b) job requirements matrix. We then present the case for synthetic validation as the best approach for many situations and address the potential limitations of synthetic validation. We conclude by proposing the development of a comprehensive database to build prediction equations for use in synthetic validation of jobs across the U.S. economy and reviewing potential obstacles to the creation of such a database. We maintain that synthetic validation is a practically useful methodology that has great potential to advance the science and practice of industrial and organizational psychology.

Establishing validity evidence for the Sport Multidimensional Perfectionism Scale-2 in intercollegiate sport

Objectives The purpose of the two studies presented in this paper is to establish validity evidence for the Sport Multidimensional Perfectionism Scale-2 (Sport-MPS-2: Gotwals & Dunn, 2009). Studies 1 and 2 are respectively focused on the production of structurally-related and external validity evidence. Methods In Study 1, 263 male and female intercollegiate volleyball players ( M age = 19.98 years) completed the Sport-MPS-2. In Study 2, 181 male intercollegiate ice hockey players ( M age = 22.16 years) responded to the Sport-MPS-2, the Multidimensional Perfectionism Scale ( Frost, Marten, Lahart, & Rosenblate, 1990), and a modified Sport Anxiety Scale ( Smith, Smoll, & Schutz, 1990). Results Exploratory factor analysis results in Study 1 revealed supportive structurally-related validity evidence for the six factors of the Sport-MPS-2. Study 2 generated supportive external validity evidence by producing anticipated moderate to large positive correlations between conceptually analogous Sport-MPS-2 and Frost-MPS subscales (all ps < .001), using canonical correlation to demonstrate that the Sport-MPS-2 was able to represent theory-based relationships between perfectionism and competitive trait anxiety (CTA), and indicating that the Sport-MPS-2—but not the Frost-MPS—was capable of identifying an anticipated link between healthy perfectionism and CTA. Conclusions Results provide support for the continued use of the Sport-MPS-2 as a measure of multidimensional perfectionism in sport.

Influence of Environment and Mobility Aid on Talking Pedometer Accuracy in Adults with Visual Impairment

BACKGROUND: Given the efficacy of pedometer-based programs to improve physical activity and health status in various clinical sub-populations, implementation of similar programs for persons with vision loss should be considered. Due to alterations in the gait patterns of persons with vision loss, factors such as environmental familiarity and mobility aids may influence results of pedometer validation trials. PURPOSE: The purpose of this study was to establish validity evidence for the Centrios talking pedometer relative to walking environment and mobility aid use in adults with vision loss. METHODS: Fifteen adults with legal blindness (10 females; age = 38 ± 14 years; BMI = 26.5 ± 4.2 kg·m2) completed two walking sessions over an unfamiliar, quarter-mile walking course while wearing a pedometer at the right and left hip. Walking speed, pedometer-determined step counts, and actual step counts were recorded during the first session, which represented walking in an unfamiliar environment. Following the completion of additional walking trials over the same route, outcome measures were reassessed during a second walking session over the course, which reflected walking in a "familiar environment." To obtain validity evidence for the talking pedometer, absolute percent error (APE) was calculated between actual and pedometer-determined steps. Paired t-tests were used to assess differences in APE relative to mounting position (right hip vs. left hip; mobility aid side vs. non-mobility aid side) across unfamiliar and familiar walking trials. RESULTS: The Centrios pedometer accurately reported step counts during unfamiliar (average speed = 3.2 mph) and familiar (average speed = 3.7 mph) walking trials when mounted at the hip opposite the user's mobility aid (APE = 3.1% and 2.4%, respectively), but was less accurate on the mobility aid side during unfamiliar and familiar walking (APE = 9.2% and 6.3%, respectively). Paired t-tests revealed no significant differences (p >.05) in APE relative to mounting position and familiarity with the environment. CONCLUSIONS: During walking in familiar and unfamiliar settings, the Centrios talking pedometer provides an accurate step count in adults with visual impairment when mounted at the hip opposite of the user's mobility aid.

How Valid Are Standard Self-Assessment Scales for International Medical Graduates?

To the Editor: In a recent article in Academic Medicine, Gozu et al1 concluded that international medical graduates (IMGs) are more likely to be less fatigued and have higher self-esteem and personal growth than U.S. medical graduates. They state that the self-assessment scales utilized in the study “have established validity evidence”: the Iowa Fatigue Scale, Perceived Stress Scale (PSS), Rosenberg Self-Esteem Scale (RSES), and Personal Growth Scale. However, we maintain that the cross-cultural validity of these scales for the extremely heterogeneous IMG group is questionable. Hence the data require cautious interpretation. First, use of the Likert scale can generate variability of responses in different cultural groups.2 Chosen numbers on the scale may not accurately reflect the degree of the individual's psychometric constructs. Individuals in some cultures may exhibit a negative-item bias, and individuals in others tend to avoid extreme options.3 Additionally, the threshold for reporting any psychological status publicly may also vary by the learned experiences of persons within individual cultures. Second, the context of an IMG's acculturation is essential in examining his or her psychological conditions. The questions in the PSS4 are so broad that they can mask some stresses provoked by unique challenges IMGs face during residency. Without stimulating a recall of each specific situation, subconscious and emotionally charged stresses may not surface. Third, the type and heterogeneity of the populations previously examined for the scales should be equivalent to the population the authors investigated. The existing studies with the RSES sampled relatively homogeneous high school and college students,3,5 not culturally diverse medical graduates. Culture influences cognitive processes that determine scores on the scales and can lead to variability of responses. When assessing IMGs, the multilayered cultures and contexts that they have lived through should not be ignored. Keiichiro Narumoto, MD Faculty development fellow, Department of Family Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania; [email protected]. Stephen A. Wilson, MD, MPH Faculty development fellowship program director and faculty member, Department of Family Medicine, University of Pittsburgh, UPMC St. Margaret Family Medicine Residency, Pittsburgh, Pennsylvania.

RESPONSE

Dear Editor-in-Chief: In their recent letter, Jehn et al. (6) questioned our conclusions regarding measurement validity and reliability of the Omron HJ-151 and HJ-720ITC pedometers (5) and implied that our conclusions could not be extended to low-intensity ambulatory activity or for chronically ill populations. We agree that researchers cannot make generalizations across populations or conditions from a single piece of validity evidence. As a measurement tool is validated, its accuracy (or lack thereof) is specific to the population and parameters for which the device was originally assessed (7). Therefore, the use of the tool cannot be generalized to situations in which the environmental conditions, walking speeds, mounting positions, and populations are different from the context of the validation process. Our study was conducted among college students with a normal gait pattern. Considering that validity is context specific, the results of our study should not be generalized to other populations or conditions. Similarly, the concept of context specificity also extends to the physical activity monitors themselves. For this reason, although Jehn et al. (6) provide evidence that spring-levered pedometers may not be accurate at slow speeds, we caution the authors in comparing the validity of physical activity monitors for recording ambulatory activity at slow speeds among monitors with vastly different internal mechanisms. Moreover, previous authors have shown the superiority of piezoelectric pedometers when compared with spring-levered models (2,4) and that piezoelectric pedometers are fairly accurate at speeds as slow as 54 m·min−1 (2,3). In our study of college students, walking speeds were selected objectively from the compendium of physical activities (1) to illustrate walking at 2.0 mph ("slow"), 3.0 mph ("moderate"), and 4.0 mph ("very brisk"). The scripted protocol was used to allow natural variations in our participants' perceptions of "slow," "moderate," and "very brisk" speeds to be observed, enabling the pedometer to be assessed across a wide range of speeds (from 1.5 to 5.9 mph). In our study, the average speeds corresponding to "slow" and "moderate" were even slower than those elicited by older individuals (age = 71.9 ± 5.7 yr) at "slow" (3.1 mph) and "normal" (3.5 mph) walking speeds (4). Both models recorded steps with less than 3% absolute error, illustrating a high degree of accuracy among the tested intensities. Moreover, the incorporation of self-selected walking trials enabled validity and reliability evidence to be established for both pedometers at each participant's freely chosen walking speed, providing a "real-life" representation of the accuracy of these devices. In regard to the concept of context specificity, perhaps Shepard (8) described the process of establishing validity evidence best, as "being a never-ending process" (1993, p. 407). We agree with the authors that the Omron pedometers may not be appropriate for clinical populations with abnormal gait characteristics. We encourage such research to be performed because the unique possibility of the Omron pedometers to be mounted at various body positions may make these pedometers more attractive to special populations. Minsoo Kang Elizabeth A. Holbrook Tiago V. Barreira Department of Health and Human Performance Middle Tennessee State University Murfreesboro, TN

Validation of the Actiheart Monitor for the Measurement of Physical Activity

The Actiheart monitor uniquely allows simultaneous measurement of heart rate and movement counts. The purpose of this study was to establish validity evidence for the Actiheart monitor under laboratory and free-living conditions. A total of 34 college students (17 males and 17 females, age = 21.8 ± 3.6 years) participated in the study. In the laboratory environment, the participants completed three, 5 min bouts of treadmill walking and/or running at speeds of 3.2, 6.4, and 9.6 km/h. Outside the laboratory, participants were asked to perform free-living physical activity for 30 min. For validation, energy expenditure, movement counts, and heart rate measurements from the Actiheart monitor were compared with an AEI Moxus Metabolic Cart, Actigraph accelerometer, Polar heart rate monitor (HRM), and electrocardiogram (ECG), respectively. The Actiheart underestimated energy expenditure only at the highest workload in the laboratory environment compared with the metabolic cart (p = .009). Actiheart heart rate (HR) was similar to the HR measured by ECG at all workloads. Under free-living conditions, the Actiheart energy expenditure was highly correlated (r = .81) with the Actigraph energy expenditure with no significant differences (t(33) =.26; p = .80). Actiheart heart rate was also highly correlated with HR from the Polar HRM (r = .93), however, there was an overestimation of HR by the Actiheart monitor (t(33) = 3.00; p = .005) under free-living conditions. The Actiheart monitor appears to accurately measure physical activity under free-living conditions and at low and moderate intensities in the laboratory environment.

Construct Validation of Physical Activity Surveys in Culturally Diverse Older Adults

The purpose of this study was to establish validity evidence of four physical activity (PA) questionnaires in culturally diverse older adults by comparing self-report PA with performance-based physical function. Participants were 54 older adults who completed the Continuous Scale Physical Functional Performance 10-item Test (CS-PFP10), Physical Activity Scale for the Elderly (PASE), CHAMPS Physical Activity Questionnaire for Older Adults, Yale Physical Activity Survey (YPAS), and modified Baecke questionnaire. The total PASE score, three outcome scores for the CHAMPS, and three summary indices for the YPAS were significantly correlated with total CS-PFP10 score. The modified Baecke exhibited no correlations with CS-PFP10 scores. The PASE, CHAMPS, and YPAS appear to be the most valid PA self-report questionnaires for culturally diverse older adults.

Validation of the Actiheart Monitor in the Laboratory Setting

The recently released Actiheart monitor allows simultaneous measurement of heart rate (HR) and movement counts and thus, may provide a more precise estimate of energy expenditure (EE). PURPOSE: The purpose of this study was to establish validity evidence of the Actiheart regarding EE and HR in a laboratory setting. METHODS: A total of 34 college students (17 males and females, respectively) participated in the study. The participants completed three, 5-min bouts of treadmill walking and/or running at speeds of 2, 4, and 6 mph. Prior to walking, resting HR was measured following 10 min of supine rest. Concurrent evidence of validity was obtained by comparing EE and HR from the Actiheart with those from a metabolic cart (MC) and EKG, respectively. Data were analyzed using Pearson correlations and paired t-tests, and the Bonferroni technique was used to adjust the alpha level. RESULTS: Descriptive statistics are presented in the table below. The Actiheart EE was highly correlated with the MC EE (r = .81, .66, and .77 at 2, 4, and 6 mph, respectively). The Actiheart, however, tended to underestimate EE compared to the MC at all three levels (t = −17.22, −11.86, and −8.50; p < .017). The Actiheart HR was highly correlated to the EKG HR (r = .99, .98, and .88). There were no significant differences in HR between the Actiheart and the EKG at all speeds (t = .14, 1.64, and 1.96; p = .89, .11 and .06).TableCONCLUSIONS: The Actiheart accurately measured HR at all speeds. While fairly well correlated to the MC, the Actiheart consistently underestimated EE at all speed levels in the laboratory environment. Hence, some adjustments to the Actiheart EE estimation formula may be necessary to better refect EE.