Literacy Assessment Instrument Research Articles

Using Think-Aloud Protocols to Uncover Misconceptions and Improve Developmental Math Instruction: An Exploratory Study

Deficiencies in education continue to escalate around the world. The focus on outcomes assessment has narrowed instructional research and curriculum evaluation to standardized testing in certain subject areas. A prototype for a quantitative literacy assessment instrument was developed with the goal of diagnosing student misconceptions of basic mathematics content and changing instructional practices to undo the misconceptions by applying cognitive psychological theory. Two hundred thirty-eight basic math high school students and 209 remedial community college students in New Jersey and New York were administered the instrument, which had been based on coded data from think-aloud protocols. The instrument asked students to answer 20 basic mathematics items and, in addition, to evaluate four possible solution strategies. For each item, frequencies of selected solution strategies and the association between strategy selection and performance on the 20-question math test are presented as a means for improving instruction. Follow-up research is proposed for determining whether undoing the student misconceptions first before teaching material on a new unit of instruction may yield more positive student outcomes.

Cognitive Interviewing as a Method for Revising the Nutrition Literacy Assessment Instrument

Cognitive Interviewing as a Method for Revising the Nutrition Literacy Assessment Instrument

Measuring Nutrition Literacy in Breast Cancer Patients: Development of a Novel Instrument.

No nutrition literacy instruments have been tested in breast cancer survivors, yet nutrition is a critical lifestyle factor for optimizing weight and improving quality of life in breast cancer survival. Our objectives were to adapt our Nutrition Literacy Assessment Instrument for breast cancer populations and to pilot test its validity and reliability. We modified the instrument based on review by content experts in cancer and nutrition and cognitive interviews with 18 cancer survivors. The modified instrument (Nutrition Literacy Assessment Instrument for Breast Cancer, NLit-BCa) was pilot-tested with 17 high-risk women and 55 breast cancer survivors. We conducted the NLit-BCa on two separate occasions 4weeks apart and assessed reliability by confirmatory factor analysis. Construct validity was evaluated by comparing results of the NLit-BCa to a Healthy Eating Index score derived from two separate 24-h dietary recalls. Content validity of the NLit-BCa was acceptable (0.93). Entire reliability for three instrument domains was substantial (>0.80), while remaining domains demonstrated fair or moderate reliability. Significant relationships were found between five of the six domains of nutrition literacy and diet quality (P < 0.05). The NLit-BCa is content valid and demonstrates promising reliability and construct validity related to diet quality, through a larger sample size, and removal of non-discriminating items is needed to confirm these findings. Thus, the NLit-BCa demonstrates potential for comprehensively measuring nutrition literacy in breast cancer populations.

A Critique of the Short Test of Functional Health Literacy in Adults.

Effective healthcare relies on the ability to communicate with patients. Ninety-eight million Americans are estimated to have limited health literacy that can impair their ability to read and interpret health-related education and information. Low health literacy is associated with higher mortality and 30-day hospital readmissions. Clinical nurse specialists and other advanced practice nurses must be able to evaluate and select a health literacy assessment instrument that is both reliable and produces valid data for the populations they serve. To assist with this important decision-making process, the psychometric properties of the Short Test of Functional Health Literacy in Adults are critiqued in the following article.

A Pilot Study to Explore the Correlation Between Parental Nutrition Literacy, BMI, and Child Healthy Eating Index-2010

Improving diet quality and weight status of children and parents is important for promoting future health. The purpose of this study was to investigate parental nutrition literacy (PNL) in relation to child Healthy Eating Index-2010 (HEI) and parental body mass index (BMI). Forty-six parents of children aged 4-6 years consented to this cross sectional study. PNL was assessed using the Nutrition Literacy Assessment Instrument for Parents (NLAI-P) that contains 42 questions relating to 5 domains of nutrition literacy. Child HEI was determined from the average of two 24-hour dietary recalls. BMI was calculated with standing height and weight. Parental NLAI-P was scored as a combined percent correct. The relationship among parental BMI, parental NLAI-P score, child BMI percentile for age and child HEI were measured by Spearman correlations and linear regression. Average NLAI-P score was 83.6% (range=64.2-97.6; SD=8.7). Average child HEI was 53.5 (range=31-86; SD=14.1). Child HEI was related to parental NLAI-P score (r=0.324, p=0.03). Parental BMI, not child BMI percentile for age, had a significant inverse relationship with parental NLAI-P score (r=-0.456, p=0.001). There was a linear relationship between parental BMI and NLAI-P score (R2=0.157; p=0.004). For every 1% increase in parental NLAI-P score, BMI decreased by 0.26. This is the first investigation into measured PNL in relation to weight status and child diet quality. Results suggest that lower PNL may be a barrier for parents to build a healthful diet for their child. Larger population studies are needed to confirm this relationship.

Development of functional oral health literacy assessment instruments: Application of literacy and cognitive theories

This paper describes the development of a new literacy assessment instrument, the Hong Kong Oral Health Literacy Assessment Task for Paediatric Dentistry (HKOHLAT-P). Its relationship to literacy theory is analyzed to establish content and face validity. Implications for construct validity are examined by analyzing cognitive demand to determine how "comprehension" is measured. Key influences from literacy assessment were identified to analyze item development. Cognitive demand was analyzed using an established taxonomy. The HKOHLAT-P focuses on the functional domain of health literacy assessment. Items had strong content and face validity reflecting established principles from modern literacy theory. Inclusion of new text types signified relevant developments in the area of new literacies. Analysis of cognitive demand indicated that this instrument assesses the "comprehension" domain, specifically the areas of factual and procedural knowledge, with some assessment of conceptual knowledge. Metacognitive knowledge was not assessed. Comprehension tasks assessing patient health literacy predominantly examine functional health literacy at the lower levels of comprehension. Item development is influenced by the fields of situated and authentic literacy. Inclusion of content regarding multiliteracies is suggested for further research. Development of functional health literacy assessment instruments requires careful consideration of the clinical context in determining construct validity.

Establishing Content Validity for the Nutrition Literacy Assessment Instrument

IntroductionIdentification of low levels of health literacy is important for effective communication between providers and clients. Assessment instruments for general health literacy are inadequate for use in nutrition education encounters because they do not identify nutrition literacy. The primary objective of this 2-part study was to assess content validity for the Nutrition Literacy Assessment Instrument (NLAI).MethodsThis study included a 35-item online survey of registered dietitians (134 of whom answered all questions) and a pilot study in which 5 registered dietitians used the NLAI among 26 clients during nutrition education consultations. To assess agreement with the NLAI by survey participants, we used the following scale: “necessary” (70% agreement), “adequate” (80% agreement), or “good” (90% agreement); comments were analyzed by using content analysis. For the pilot, we made comparisons between subjective assessments, the Rapid Estimate of Adult Literacy in Medicine (REALM), and the NLAI. Registered dietitians also completed a postpilot–study survey.ResultsFor the online survey, we found good agreement (average, 89.7%) for including each section of the NLAI. All sections accomplished their purpose (average, 81.5%). For the pilot, REALM and NLAI correlation (r = 0.38) was not significant; the subjective assessment of clients by dietitians and NLAI lacked agreement 44% of the time, and registered dietitians provided instruction on deficient knowledge and skills identified by the NLAI 90% of the time.ConclusionThe NLAI is a content-valid measure of nutrition literacy. Additional validation of the NLAI is important because an objective instrument is needed for identifying nutrition literacy, a construct that appears to be different from health literacy.

Assessment of orthopaedic literacy in an urban emergency department

Background: Health literacy is considered the single best predictor of an individual’s health status. However, it often is difficult to assess. As such, our group developed the Literacy in Musculoskeletal Problems (LiMP) questionnaire, a musculoskeletal-specific literacy assessment tool. We hypothesized that the degree of inadequate musculoskeletal health literacy would be higher than the rate of inadequate general healthy literacy, as assessed through the Newest Vital Sign (NVS). Methods: Each of the 65 study participants completed a demographic questionnaire, the NVS and the LiMP survey. The NVS and LiMP scores were assessed using contingency table analysis. Categorical outcome variables as a function of demographic parameters also were compared using χ2 tests. Results: More subjects were found to have inadequate musculoskeletal literacy (60%) as compared with general health literacy (48%). Contingency table analysis showed that at LiMP scores of less than 6, the sensitivity and specificity (0.74 and 0.59, respectively) were optimized for predicting limited literacy, as defined by an NVS score of 3 or less. A significantly higher rate of adequate musculoskeletal literacy was observed in Caucasian patients and those who had previously seen a physician for a musculoskeletal complaint. Conclusions: The LiMP’s sensitivity and reliability is in line with prior work on disease and specialty-specific literacy, making it a valid and reliable musculoskeletal literacy assessment instrument. It is only with the ability to identify those lacking the skills crucial to making informed decisions regarding their musculoskeletal health that we can accurately target education campaigns, an approach that will ultimately enhance physician-patient interactions and improve clinical outcomes.

21세기 과학적 소양 평가기준 개발 및 교과서 내용 분석에의 적용

본 연구는 21세기 과학적 소양을 탐색하기 위한 평가 기준을 개발하고, 이를 교과서 내용 분석에 적용하였다. 21세기 과학적 소양의 하위 영역으로 과학적 사고습관(habits of mind), 과학에서의 가치관(character and value), 과학 본성의 이해(science as human endeavor)를 설정하고, 각 하위 영역에 대한 평가 요소와 평가 기준을 개발하였다. 개발된 평가 기준은 21세기 과학적 소양이 교육 상황에서 실제 실현되고 있는가를 확인할 수 있는 지표이다. 개발된 평가 도구를 사용하여 2009 개정 교육과정에 기반을 둔 과학 교과서 내용에서 21세기 과학적 소양의 3개 영역에 관련된 내용이 포함되어 있음을 확인하였다. 이 중 과학적 사고 습관에 관련된 내용이 가장 높은 비율을 차지하였으며 과학에서의 가치관과 과학본성의 이해에 관련된 내용은 상대적으로 부족한 것을 알 수 있었다. 개발된 21세기 과학적 소양 평가 기준은 교과서 내용 분석 외에도 교실 수업 평가, 교사 평가, 교수-학습 자료의 평가에 활용하여 현장 과학 수업 실행에 대한 실질적인 이해를 가져올 수 있을 것으로 기대된다. We developed <TEX>$21^{st}$</TEX> century scientific literacy assessment instrument and applied it to explore the contents of seven Korean science textbooks. The <TEX>$21^{st}$</TEX> century scientific literacy assessment instrument involved three dimensions (habits of mind, character and values, science as human endeavor). Each dimension consists of three sub-dimensions. Five science education experts assessed the content of textbook using criteria. We discussed issues in which the examiners responses did not match and reached an agreement on initial disagreement. As a result, we found that most Korean textbook contained contents on habits of mind, especially, communication, collaboration, and information management. We also found that most materials lacked information about character and values and science as human endeavor. Based on the result, we suggest that researchers and science educators need to consider all dimensions of the <TEX>$21^{st}$</TEX> century scientific literacy when they develop curriculum and teaching materials. In addition, the rubric for <TEX>$21^{st}$</TEX> century scientific literacy can be adopted as an assessment tool for examining curriculum, teaching materials.

Establishing Content Validity for the Nutrition Literacy Assessment Instrument

Introduction Identification of low levels of health literacy is important for effective communication between providers and clients. Assessment instruments for general health literacy are inadequate for use in nutrition education encounters because they do not identify nutrition literacy. The primary objective of this 2-part study was to assess content validity for the Nutrition Literacy Assessment Instrument (NLAI).

A Practical First Step to Integrating Genetics Into the Curriculum

We conducted a systematic literature review to identify tools that could be used to measure faculty and undergraduate nursing students' genetic literacy, and we assessed the utility of one of those tools, the Genetic Literacy Assessment Instrument (GLAI), with faculty and students at a college of nursing in a large public university. No significant differences were noted between faculty and students in terms of their overall scores on the GLAI. On average, faculty answered 76% of the questions correctly and students answered 73% of the questions correctly. Both groups scored highest in the genetics and society domain (93% and 85% correct answers for faculty and students, respectively) and lowest in the evolution domain (52% correct for both groups). The GLAI may be used to evaluate curricula for strengths and deficiencies, as well as to identify students and faculty in need of additional instruction.

A national evaluation of the impact of the secondary social and emotional aspects of learning (SEAL) programme

In recent years, the English education system has reflected a worldwide interest in social and emotional learning (SEL), as evidenced by the national launch of the secondary social and emotional aspects of learning (SEAL) programme in 2007. SEAL is a whole-school approach designed to positively influence a range of pupil outcomes, including increased social and emotional skills, better behaviour and reduced mental health difficulties. The aim of the current study was to examine the impact of SEAL on such outcomes. The study utilised a quantitative, quasi-experimental design with a sample of 22 schools (approximately 2360 pupils) implementing the SEAL programme, and 19 ‘matched comparison’ schools (approximately 1991 pupils), selected on the basis of similar school-level characteristics. A cohort of pupils in these schools completed annual self-rated assessments of their social and emotional skills (using the Emotional Literacy Assessment and Intervention instrument), mental health difficulties and pro-social behaviour (using the Strengths and Difficulties Questionnaire) over a two-year period. After controlling for a range of school- and pupil-level characteristics, analysis using multi-level modelling indicated marginal, non-significant effects of the SEAL programme on pupils’ social and emotional skills and mental health difficulties, and no significant effect on their pro-social behaviour. The study findings are discussed in relation to existing evidence about the effectiveness of the SEAL programme and the broader SEL evidence base.

Pre-service science teachers’ genetic literacy level and attitudes towards genetics

The aim of this study is to determine pre-service science teachers’ genetic literacy level and their attitude towards genetics. The Genetics Literacy Assessment Instrument (GLAI; Bowling et al., 2008) and the Genetics Attitude Scale (see Acra, 2003) were used to collect relevant data. Pre-service science teachers held quite favorable attitude towards genetics (M=3.75) while they had a relatively low level of genetic literacy (M=15.39). Their knowledge levels with respect to dimensions of the GLAI varied. For example, they were found to be more knowledgeable in the transmission of genes and followed by the nature of genetic material dimensions compared to gene expression, gene regulation and evolution dimensions. Their knowledge level in the genetics and society dimension was relatively low, indicating limited understanding of the interactions between genetics and society.

Quantitative Literacy at Michigan State University, 2: Connection to Financial Literacy

The lack of capability of making financial decisions has been recently described for the adult United States population. A concerted effort to increase awareness of this crisis, to improve education in quantitative and financial literacy, and to simplify financial decision-making processes is critical to the solution. This paper describes a study that was undertaken to explore the relationship between quantitative literacy and financial literacy for entering college freshmen. In summer 2010, incoming freshmen to Michigan State University were assessed. Well-tested financial literacy items and validated quantitative literacy assessment instruments were administered to 531 subjects. Logistic regression models were used to assess the relationship between level of financial literacy and independent variables including quantitative literacy score, ACT mathematics score, and demographic variables including gender. The study establishes a strong positive association between quantitative literacy and financial literacy on top of the effects of the other independent variables. Adding one percent to the performance on a quantitative literacy assessment changes the odds for being at the highest level of financial literacy by a factor estimated to be 1.05. Gender is found to have a large, statistically significant effect as well with being female changing the odds by a factor estimated to be 0.49.

Information to fight the flab:

The purpose of the paper is to examine information use and information literacy in the context of weight management. It reports on a two-year study funded by the Department of Health known informally as the Net.Weight Study. Net.Weight examined the potential for increased, innovative and effective uses of information and communication technologies (ICTs) to support the self management of weight. The research was conducted in the city of Brighton &amp; Hove by an inter-disciplinary team from the University of Brighton. The paper gives a brief overview of the various methods used in the study as a whole but discusses one strand, the user survey, in more detail. The survey gathered data on people’s information and ICT use around weight management. The design of the survey questionnaire required the adaptation of existing literacy assessment instruments and this process is described in this paper. The findings show that people use a wide range of information sources for information and support around weight management. The most useful sources are slimming groups, food packaging, friends and family, magazines, TV and health books, thus representing a variety of media, formal and informal, and including human sources. The internet was reported to be a useful source for around half the survey respondents and is most often used for information about diet and exercise. A majority of respondents described themselves as active information seekers and confident about their information skills. They are less confident about internet information than information generally and even less confident about using the internet to support weight management activities. The concept of literacies, particularly around information and health, provide a framework for examining the Net.Weight findings. The findings are discussed in terms of their implications for health information policy and for those interested in applying information literacy theory to health. The role of healthcare practitioners in weight management information is addressed, as is the need for targeted rather than generic health information. It is suggested that the work done in the education sector to increase awareness of information literacy and improve skills could provide a useful model of good practice in a health context. However, the evidence provided by the Net.Weight study suggests that for such an approach to be relevant it needs to reflect the complexity of health information processes in everyday lives.

The Medical Library Association Guide to Health Literacy

The Medical Library Association Guide to Health Literacy

Reconciling DIBELS and OSELA: What Every Childhood Educator Should Know

Since the No Child Left Behind Act of 2001, teachers and parents have witnessed a series of heated debates on the most basic issues of literacy assessment—what to assess and how to assess it. In particular, the controversy rages over DIBELS and OSELA, two popular early literacy assessment instruments. The purpose of this article is to address the early literacy assessment issues from a balanced literacy perspective. In particular, the authors will: 1) introduce DI-BELS and OSELA assessments; 2) inspect DIBELS and OSELA from educational philosophy, educational psychology, and literacy development theory bases; 3) examine the DIBELS and OSELA debate from balanced literacy perspectives; and 4) discuss implications and make recommendations for reconciling the DIBELS and OSELA debate.

Genetic Literacy of Undergraduate Non–Science Majors and the Impact of Introductory Biology and Genetics Courses

ABSTRACT With the advancement of genetic information and technologies, there is an increasing need for a genetically literate public. This study looks critically at student learning and at the current instruction of genetics in introductory non–science major biology and genetics courses at the undergraduate level. A new diagnostic tool, the Genetic Literacy Assessment Instrument, was administered pre- and postcourse to more than 300 students in six introductory nonmajor courses that emphasize genetics to varying degrees. Current data from students in these courses show a precourse average score of 43 percent correct zn the inventory. Postcourse scores increased only modestly, to an average of 49 percent. In this article, we discuss the impact of teaching methods and course content on scores, as well as student learning in the different content areas of genetics. The results suggest that further studies in genetics education are needed to better understand the effect of teaching methods on achieving geneti...

How Long Does It Take to Assess Literacy Skills in Clinical Practice?

Health literacy screening is often not performed in clinical settings. One possible reason is the concern about the time involved in performing such assessments. Our objective was to measure the time required to administer the Newest Vital Sign (NVS) literacy assessment instrument to English-speaking primary care patients. The NVS was administered to 78 consecutive English-speaking patients in an outpatient primary care clinic. The length of time to complete the NVS was timed with a stopwatch. The average time to complete the NVS was 2.9 minutes (95% confidence limit, 2.6-3.1 min). The NVS is a health literacy screening tool of sufficient brevity to be considered for use in primary care practices.

Teaching and Assessing Information Literacy in a Geography Program

In the technological world of the Twenty-first Century, students must be information literate. They must have the skills to access, evaluate, and utilize information needed in their undergraduate experience and in their future endeavors. It is important for Geography majors to acquire these skills as part of their undergraduate education. At one institution of higher learning, information literacy learning is embedded in the Geography curriculum. An online instrument to assess information literacy skills is used to evaluate seniors. In this article, goals for information literacy, the creation of the information literacy assessment instrument and the results of assessment testing are discussed.