Quality Rating And Improvement Systems Research Articles

Protocol for a mixed-methods investigation of quality improvement in early childhood education and care in Australia

International recognition of the early years as a crucial foundational period has led to the design and implementation of quality rating and improvement systems (QRIS) that define, communicate, and monitor the components of quality in early childhood education and care (ECEC). The aim of these policies is to achieve effective quality assurance and improvement through a system-oriented approach to assessment and evaluation. Informed by ecological systems theory, this paper outlines a three-phase, mixed-methods design for researching a national sample of child care centres that showed overall improvement on the Australian National Quality Standard (NQS) assessment and rating (A&R) criteria. The study samples are drawn from a national dataset of centre-based child care services with two or more A&R rounds and an initial rating of Working Towards NQS (N = 1,935). The results of this study will provide insights into the macro-, exo-, meso- and micro-systems level factors and strategies that support quality in ECEC services.

Macro-structural predictors of Australian family day care quality.

This study explores the predictive power of macro-structural characteristics on quality rating and improvement system (QRIS) outcomes of Family Day Care (FDC) services in Australia. The dataset consisted of 441 FDC National Quality Standard (NQS) ratings from all Australian states and territories, with overall ratings of Exceeding NQS, Meeting NQS, Working Towards NQS, or Significant Improvement Required. Multinomial logistic regressions confirmed that management type, community socioeconomic status (SES), level of urbanization, and government jurisdiction explained 6.9 to 19.3% of the variation in QRIS outcomes. Results indicated that lower FDC NQS ratings were more likely for (1) private for-profit vs. not-for-profit; (2) low-SES vs. high-SES area; and (3) regional or remote area vs. metropolitan. State/territory jurisdiction also influenced NQS ratings. These findings imply the need for policy attention to inequalities in FDC quality associated with systemic and organizational differences. Greater effort is needed to promote equality and equity in FDC services.

Testing theoretical explanations for heterogeneity in associations between a state quality rating and improvement system and prekindergarten children’s academic performance

The present study tests two complementary developmental theories regarding the fit between children’s skills and their environments within the context of a state Quality Rating and Improvement System (QRIS). Specifically, we examine whether high-quality prekindergarten disproportionately benefits children based on their entering ability levels. The sample includes 684 children (48% female; Mage = 57.56 months) residing in a midwestern state who were eligible to receive child care subsidies and participated in an evaluation of state-funded prekindergarten during the 2015-2016 school year (cohort 1), 2016-2017 school year (cohort 2), or 2018-2019 school year (cohort 3). We implemented multilevel regression models with children nested in their respective classrooms and schools to investigate whether children in higher-rated prekindergarten programs on the QRIS experienced greater growth in math, literacy, and vocabulary from fall to spring if they entered with stronger executive function skills (i.e., foundational skills hypothesis), or if they entered with weaker skills in the same respective domain as the outcome (i.e., compensatory skills hypothesis) relative to children who attended programs that were lower-rated or unrated on the QRIS. Contrary to both hypotheses, only one significant interaction emerged indicating that children who attended prekindergarten programs rated as low-quality on the QRIS were more likely to have higher spring literacy performance if they began the year with stronger executive function skills. Implications for research and policy are discussed.

Structures and systems influencing quality improvement in Australian early childhood education and care centres.

This study assessed the impact of structural characteristics on quality rating and improvement systems (QRIS) outcomes in an Australian national study. Data from the Australian Children's Education and Care Quality Authority (ACECQA) repository of National Quality Standard (NQS) ratings were used to identify long day care services that had improved from Working Towards NQS to Meeting or Exceeding NQS or had no change over two assessments. QRIS outcomes were examined for state/territory jurisdiction, urban-rural location, community socio-economic status, type and size of provider organisation, centre size and stability of centre owner/provider using multinomial logistic regression analyses. Controlling for jurisdiction, results showed that improvement to Meeting NQS was more likely for not-for-profit versus for-profit providers and for large multi-site provider organisations versus small, stand-alone providers. Improvement to Exceeding NQS was also associated with not-for-profit and larger provider organisations, as well as larger versus smaller centres, and centres that had stable ownership.

Promotion of early childhood development and mental health in quality rating and improvement systems for early care and education: A review of state quality indicators

In the United States (U.S.), quality rating and improvement systems (QRIS) are used by many states to incentivize quality in ECE and may be a viable lever for promoting early childhood development and mental health on a population level. We conducted a qualitative review of publicly available data on state QRIS indicators to better understand how states incorporate evidence-informed early childhood development and mental health promotion standards in QRIS. We systematically compared QRIS indicators for 41 U.S. states with child development and mental health promotion quality standards from Caring for Our Children National Health and Safety Performance Standards; Guidelines for Early Care and Education Programs, 3rd Edition, as of March/April 2020. Of those, 39 states included at least one indicator consistent with child development or mental health promotion standards, including practices that can lead to early detection of developmental delays such as developmental monitoring, activities or curriculum addressing developmental domains, and regular communication and resource-sharing with parents/guardians. Opportunities exist within states for incorporating more specific guidance within indicators, such as use of childcare health consultants and advocates, validated screening tools, parent/guardian participation or input in developmental monitoring and screening, and staff training on family engagement. We found that in most states QRIS indicators offer guidance for ECE systems to support and monitor early development and foster mental health, with opportunities to enhance guidance. Findings point to QRIS as a viable opportunity for promotion of early childhood development and mental health standards in ECE systems.

The effects of a high quality state-run preschool program as rated by a Quality Rating and Improvement System on children's school readiness

The goal of this study was to evaluate the effects of attending a state-run preschool program that is rated as “higher quality” on a state's Quality Rating and Improvement System (QRIS) for children's outcomes in prekindergarten and kindergarten. It was expected that children who attended higher QRIS-rated programs would demonstrate greater gains in school readiness skills relative to children who received similar subsidies but attended lower-rated programs in the prekindergarten year. However, because there is little to base a hypothesis on with regard to longitudinal associations, we had no a priori hypotheses regarding potential effects into the fall and spring of kindergarten. Participants included 2 cohorts (N = 516; 50% female, Mage = 4.80 years). Data were collected on school readiness outcomes in the fall and spring of prekindergarten and kindergarten. Results suggested that participating in the higher-rated program was related to gains in language during the prekindergarten year that persisted into kindergarten. Findings indicated that children who participated in the program experienced greater gains in their math, literacy, and school readiness in kindergarten. However, the majority of the effects were generally small, albeit positive.

Do Early Care and Education Programs Improve When Enrolled in Quality Rating and Improvement Systems? Longitudinal Evidence from One System

ABSTRACT Quality rating and improvement systems (QRISs) are designed to assess, improve, and communicate the quality of early care and education programs. Using administrative data, we investigated whether providers that participated in a county-level QRIS saw increases to their global quality (tier) rating, domain-specific quality ratings, and scores on an observation measure of classroom quality. Research Findings: On average, participating providers increased their overall tier ratings; this pattern was particularly true for providers with lower initial tier ratings. We found more consistent evidence of increases on indicators of structural quality components (e.g., teacher and director qualifications), and limited evidence of improvement on process quality indicators (e.g., measure of teacher–child interaction). This finding suggests structural quality improvements may be driving global tier rating changes. In addition, providers saw average increases on a measure of classroom quality, with larger increases seen in domains that measure classroom organization and emotional support. Practice and Policy: We discuss implications for how QRISs can promote quality improvement for participating providers and suggest avenues for future research.

Improving Preschool Accountability Systems: Bringing Individual Children’s Experiences Back to Child Policy

Preschool accountability systems, including Quality Rating and Improvement Systems (QRIS), seek to assess, monitor, and improve children’s outcomes across the early care and education (ECE) landscape. QRIS have a number of strengths, especially by focusing on classroom-level quality inputs for all children collectively across multiple domains that are well aligned with developmental science. This article considers how to build on the QRIS framework by highlighting children’s individual experiences within classrooms as a key indicator of quality in addition to the more traditional classroom-level measures. The article first provides the theoretical rationale and empirical evidence for focusing on children’s individual experiences based on new insights from developmental science. The article then illustrates key factors that relate to variation in children’s experiences, including child temperament, gender, age, and race/ethnicity. The article concludes by considering opportunities for innovation to better measure individual children’s experiences in QRIS.

An intensive professional development in family child care: a promising approach

ABSTRACT The quality of early care and education affects a wide range of children’s experiences and developmental outcomes. Family Child Care (FCC) is no exception, and attention to increasing the quality of Family Child Care is needed. The purpose of the current study was to examine an experimental case of intensive Professional Development (PD) in FCC programmes focusing on domain-specific quality (language and literacy) and global quality. Forty-nine FCC providers in two states participated in the study. Thirty-eight FCC providers received intensive PD support, including five three-hour training workshops on early language and literacy, weekly onsite technical assistance, and monthly Community of Practice meetings. Eleven FCC providers served as a comparison group. The Child/Home Environmental Language and Literacy Observation (CHELLO) and Family Child Care Environmental Rating Scale (FCCERS-R) were used to measure the quality of FCC. Results showed significant improvement in the high-intensity PD group on both measures. The study presents a promising model of PD that may be an appropriate approach to embed within Quality Rating and Improvement Systems (QRIS) in early child care and education.

Association of the Quality Rating and Improvement System, Texas Rising Star, on Physical Activity and Screen Time Policies and Practices in Texas Child Care Centers

Quality rating and improvement systems (QRIS) are systems approaches to assist states in providing high quality early childhood education. Texas Rising Star (TRS), a voluntary QRIS, exceeds state licensing standards and meets some obesity prevention guidelines. This study examines differences in physical activity, screen time, and outdoor policies and practices by QRIS certification. Cross-sectional. Online. After exclusion criteria, respondents were 431 Texas childcare centers. 2016 survey of policies and Go NAPSACC best practices. Chi-square and t-tests indicated differences in 1) practices and 2) policies by QRIS status. TRS-certified centers reported more policies for physical activity (M = 4.57 ± 3.07 vs. 3.61 ± 2.95, p = 0.009) and screen time (M = 1.91 ± 1.84 vs. 1.28 ± 1.56, p < 0.001) than non-certified centers. TRS-certified centers reported significantly higher frequencies for 7 of 14 physical activity practices, however no significant differences for screen time practices were found. Additionally, TRS-certified centers reported more outdoor practices, including more classrooms/storage (p < 0.001) and vegetable gardens (p = 0.025). TRS-certified centers reported more physical activity policies and practices, more screen time policies, and more outdoor practices. TRS certification was not associated with screen time practices. QRIS can be a practical way to insert obesity prevention in early care and education. Using items from a widely used survey enables comparisons, however future research is needed in larger-scale studies. Some COVID-19 implications are discussed.

Systemwide Quality Improvement in Early Childhood Education: Evidence From Louisiana

Despite substantial federal, state, and local investments in improving early care and education (ECE), we know little about whether ECE program quality has improved over time. The lack of data tracking the quality of publicly funded ECE programs at scale creates a substantial evidence gap for policymakers attempting to weigh the returns on, and future of, quality improvement policies. Data from Quality Rating and Improvement Systems (QRIS) provide a promising opportunity to address this problem. Using 4 years of data from a mandatory, statewide QRIS covering subsidized child care, Head Start, and state prekindergarten, we document systemwide quality and improvement trends over a period of targeted investment in quality improvement statewide. We find improvements in quality overall, across sectors and communities over this period. Results also reveal differential growth across sectors such that quality gaps diminished. This study highlights the potential of statewide ECE data for informing quality improvement efforts.

A mixed-methods study of Maryland’s monetary incentives to improve the quality of child care centers

This study used a sequential explanatory equal-status mixed-method design to investigate whether Maryland’s child care tiered reimbursement system incentivized child care centers to be rated at least 3 (and receive an incentive payment) on Maryland’s 5-level Quality Rating and Improvement System (QRIS). The first stage of research consisted of multilevel logistic regressions to determine the association between centers’ reliance on child care subsidy payments and whether the center had a rating of 3 or higher. Reliance on subsidy payments was defined as the percentage of licensed slots filled by children receiving a child care subsidy. State administrative data on 1003 centers that received a subsidy payment in January 2018 were combined with demographic data from the U.S. Census. The second stage of research consisted of 14 interviews with center directors to understand how they made decisions about which QRIS rating to attain and how tiered reimbursements factored into their decisions. Results from the quantitative research showed that a greater subsidy density was associated with a greater likelihood of a center being rated 3 and receiving an incentive payment. However, results from the qualitative research showed that few center directors reported that tiered payments factored into their decision on what QRIS rating to reach and no directors were singularly motivated by the incentives. Rather, directors reported being intrinsically motivated to improve QRIS ratings or motivated by technical assistance providers. Additionally, directors who did not attain a rating of 3 experienced capacity challenges. Policy implications are discussed.

Who participates in quality rating and improvement systems?

Even with rapid and widespread expansion of states’ quality rating and improvement systems (QRIS)—tiered frameworks that assess, communicate, and improve early childhood education (ECE) quality—there exists no population-level information regarding which providers choose to participate in these primarily voluntary systems. We use a nationally representative survey of ECE centers to examine how the characteristics of ECE centers and the communities in which they are located predict participation in QRIS to understand the scope of QRIS policy implementation and the extent to which QRIS may be equity enhancing. We find that approximately one-third of centers nationwide participated in QRIS in 2012. Selection model results reveal that participation is more likely among centers that blend multiple funding sources and who are NAEYC accredited, and in communities with high poverty rates. However, QRIS participation is less likely in communities with relatively higher proportions of Black residents. Findings raise questions about how QRISs can equitably engage programs in all communities.

Exploring the Relationship Between Quality Rating and Improvement System Supports and Global Quality in Family Child Care

A Quality Rating and Improvement System (QRIS) is one widespread method to provide a framework for quality improvement in early care and education. However, there is little research examining the relationship between QRIS supports and quality changes in family child care programs. This study aims to examine the associations between supports offered through Delaware’s QRIS and global quality changes in family child care programs. Delaware’s QRIS supports included financial incentives and on-site technical assistance. Data were extracted from the administrative database on 139 family child care programs participating in the QRIS. Repeated measures multivariate analysis of covariance was used to examine the effects of time and Delaware’s QRIS supports. Results showed significant relationships between QRIS supports and the improvement of global quality in family child care programs as measured by the Family Child Care Environment Rating Scale-Revised. Specifically, the amount of on-site technical assistance was significantly associated with changes in the Interaction and Program Structure subscale scores. Additionally, grants received through the QRIS were significantly associated with quality improvements in the home environment. Considering that the amount of technical assistance alone was effective in improving the global quality of family child care programs, professional development strategies that aim to articulate and enhance the technical assistance that is provided to programs would be useful. Further, various types of financial incentives are encouraged to assist family child care programs in improving the structural quality of their environments.

Simplifying quality rating systems in early childhood education

High-quality preschool experiences can promote children’s short- and long-term development, yet many children in the United States still lack access to high-quality care. Many states have turned to Quality Rating and Improvement Systems (QRIS) in an attempt to solve this problem. Unfortunately, recent empirical work has shown that most QRIS ratings are not linked to either program quality or children’s learning gains, leading to calls for reform. In particular, in recent years there have been calls to simplify QRIS and focus accountability on a smaller set of quality measures that are linked to children’s development. This exploratory study probes the potential of such an approach, comparing how a single measure of quality compares to a set of commonly-used QRIS indicators (e.g. teacher education, an observational measure of teacher-child interactions, classroom size/ratio, opportunities for family involvement, etc.). Our findings—that a QRIS system based on an observational measure of the quality of teacher-child interactions alone is equally effective at predicting children’s learning gains as a more complex set of indicators—align with recent calls for second generation QRIS that use fewer, but more powerful indicators. Implications for research and policy are discussed.

Early Childhood Education and Child Care in Arizona : Is Availability Alone Sufficient?

With many states increasingly adopting Quality Rating and Improvement Systems (QRIS) to rate their early childhood education (ECE) and child care programs, researchers question the use of these systems. Specifically, they are trying to understand the value of information provided by QRIS ratings and the implications QRIS ratings have on the quality of and access to ECE and child care programs for families. In this study, we attempt to understand the value of QRIS ratings when they are provided for families at the household level. To do so we take a close look at the relationship between availability of programs rated by the Quality Improvement and Rating System (QIRS) in Arizona and demographics of the communities they serve, and compare the utilization of the programs in communities with varying demographic compositions. While we find that more high-quality ECE and child care programs are available for children that are Hispanic, Black, and eligible for free and reduced-price lunch, we also find that families underutilized those programs. We argue that this underutilization might be due to a variety of barriers that the families are experiencing and believe that efforts should be directed to work with families and assist them in understanding their enrollment options.

Influence of quality credentialing programs on teacher characteristics in center-based early care and education settings

Considerable evidence relates the quality of early childhood education (ECE) to children’s developmental outcomes, with teacher qualifications often cited as a critical ingredient for high-quality care. As a result, quality credentialing programs—including Quality Rating and Improvement Systems (QRIS) and the National Association for the Education of Young Children (NAEYC) Early Learning Program—are increasingly prominent vehicles for improving program quality. This study combines a randomized resume audit study with administrative data on provider participation in QRIS (N = 5607) and NAEYC (N = 10,553) to examine whether these quality credentialing programs influence provider behavior during the teacher hiring process. While NAEYC-accredited providers are strongly attracted to job applicants with ECE work experience, education, and other professional credentials, QRIS participants are not.

Policy levers to promote cultural competence in early childhood programs in the USA: recommendations from system specialists

The purpose of the study was to investigate ways in which early childhood systems that rate and license child care programs, known as Quality Rating and Improvement Systems (QRISs), might incorporate requirements related to cultural competence into the system, including incentives, supports, and rewards for programs that foster cultural competence of the early care and education (ECE) workforce. The study employed a focus group design to consider the research problem from the vantage point of key informants such as coaches and other system staff who routinely support ECE programs to meet system requirements and to engage in quality improvement work. Three focus groups were convened, with a total of N = 28 participants. The meetings were audio-recorded, transcribed, and analyzed using the constant comparative method. Emergent themes were noted and grouped into four categories: QRIS requirements; system alignment; ECE progrm administrator training; and ECE teacher training and technical assistance. Focus group members advocated for incorporation of system-level requirements related to cultural competence as well as program supports to ensure that ECE providers could succeed in meeting new requirements. Policy recommendations to support cultural competence development in the ECE workforce and to embed cultural competence in ECE systems are discussed.

The Effects of Accountability Incentives in Early Childhood Education

AbstractIn an effort to enhance the quality of early childhood education (ECE) at scale, nearly all U.S. states have recently adopted Quality Rating and Improvement Systems (QRIS). These accountability systems give providers and parents information about program quality and create both reputational and financial incentives for program improvement. However, we know little about whether these accountability reforms operate as theorized. This study provides the first empirical evidence on this question using data from North Carolina, a state with a mature QRIS. Using a regression discontinuity design, we examine how assignment to a lower quality rating influenced subsequent outcomes of ECE programs. We find that programs responded to a lower quality rating with comparative performance gains, including improvement on a multi‐faceted measure of classroom quality. Programs assigned to a lower star rating also experienced enrollment declines, which is consistent with the hypothesis that parents responded to information about program quality by selectively enrolling away from programs with lower ratings. These effects were concentrated among programs that faced higher levels of competition from nearby providers.

Self-Report QRIS: Challenges with Validation

ABSTRACTResearch Findings: The current study looks at the validity of a voluntary self-report Quality Rating and Improvement System (QRIS) and the characteristics of participating childcare centers. The self-reported quality indicators are compared to external ratings of quality Early Childhood Environment Rating Scale-Revised ([ECERS-R]) and correlated with variables such as size of center and number of state subsidy clients. ECERS-R scores were unrelated to capacity but significantly lower for centers with a large percentage of state-supported clients. Regarding self-reported quality, centers frequently under-reported their quality and what was claimed was not always externally validated, suggesting a self-report QRIS may not be an accurate assessment of quality. Additionally, no significant differences in quality were found between centers participating and those not participating in the self-report QRIS. Practice or Policy: Self-reported childcare quality was not accurate in this study. Although providers over-reported some quality, they frequently under-reported quality, by claiming fewer indicators than external validators found. When centers are unmotivated to participate in a voluntary, self-report QRIS, when items reported are the easiest to report, and when existing quality indicators are unreported, a self-reported QRIS cannot validly reflect quality. Because providers over-reported and under-reported quality criteria, it is doubtful the system truly incentivizes desired quality changes.