Student-level Data Research Articles

Assessing library contributions to university outcomes: the need for individual student level data

PurposeThe purpose of this paper is to suggest move beyond examining the impact of the library from the library's perspective but rather to collaborate with university‐wide assessment efforts to determine the impact of the library from the student's perspective, the instructor's perspective, or the researcher's perspective.Design/methodology/approachA limited number of studies have been conducted in an attempt to determine the impact of the use of the library's physical or electronic resources, reference services, and the library as place. Considerably more studies have been prepared that have examined library instruction programs and information literacy programs although the results paint a very mixed picture. A majority of the evaluation efforts for information literacy programs have used opinion surveys, skills testing, and observed actual behavior using rubrics and other methods. A consistent criticism of such studies has been the small sample size and the lack of replication of such studies. Clearly an approach that collaborates with the university's assessment efforts and allows the library to determine the correlation levels between use of a library collection or service and a desired university outcome will be welcome by all academic libraries.FindingsThe suggested approach is to combine library usage data with university data. In the best of worlds the library data would be combined with both indirect measures of learning (such as student persistence, graduation rates) as well as direct measures of student learning such as the Collegiate Learning Assessment, Collegiate Assessment of Academic Proficiency, and the Measure of Academic Proficiency and Progress. If university libraries were to embark on a coordinated program of data analysis then the results would have greater credibility and impact within the university.Practical implicationsThe paper provides a set of recommendations for combining library data (where the identification of each user is preserved) with institutional performance and demographic data for each user (after which the unique identification information can be eliminated so as to preserve the privacy and confidentiality rights of the user). The resulting large data set can be then used to explore possible relationships between use of library services and important outcomes – student success, student retention, graduation rates, and so forth that is necessary in order to demonstrate the value of the academic library.Social implicationsOnce libraries have a better understanding of the impacts of specific library services in the lives of their students, faculty and researchers, libraries can enhance services that produce positive results and drop services that have marginal or no value. The end result will be that academic libraries will better serve the needs of their customers.Originality/valueThis paper offers a new broad perspective that will encourage libraries to undertake a collaborative effort with others on campus in order to better assess the impact of the academic library.

Twice considered: Charter schools and student achievement in Utah

A relatively small state, Utah presents an interesting case to study charter schools given its friendly policy environment and its significant growth in charter school enrollment. Based on longitudinal student-level data from 2004 to 2009, this paper utilizes two approaches to evaluate the Utah charter school effectiveness: (a) hierarchical linear growth models with matched sample, and (b) general methods of moments with student-fixed effects regressions. Both methods yield consistent results that charter schools on average perform slightly worse as compared to traditional public schools, a result that is primarily affected by the low effectiveness and high student mobility of newly opened charter schools. Interestingly, when charter schools gain more experience they become as effective as traditional public schools, and in some cases more effective than traditional public schools. This research has implications for local and state charter school policies, particularly policies that avoid “start-up” costs associated with new charter schools.

Student and school characteristics associated with use of nicotine replacement therapy: A multilevel analysis among Canadian youth

BackgroundResearch indicates that it is common for youths to use nicotine replacement therapy (NRT) despite limited empirical evidence for its effectiveness within this population. Since very little is known about the characteristics associated with NRT use by youth, the current study examined the association between ever and current use of NRT as a function of student characteristics and the characteristics of the schools they attend. MethodsThis study used nationally representative student-level data from 29,296 grade 9 to 12 students who participated in the 2008–2009 Canadian Youth Smoking Survey (YSS). School-level data on the built environment surrounding schools were provided by DMTI-Spatial, and data on school location were provided by the Canadian Census. Two multilevel logistic regression models were used to predict ever use of NRT and current use of NRT as a function of student and school characteristics among current smokers. ResultsOverall, 21.1% of youth smokers in Canada had ever used NRT and 5.1% were currently using NRT. Odds of ever and current NRT use were highest among daily smokers and boys, while youths who had made multiple quit attempts or participated in a quit and win contest were more likely to be ever NRT users. Attending a school located within an urban area increased youths' odds of ever and current NRT use, whereas higher density of pharmacies surrounding a school was inversely associated with current NRT use. ConclusionsCharacteristics of students and the schools they attend were associated with the likelihood of youth smokers using NRT. Significant between-school differences in NRT use exist, however further research is needed to identify which school characteristics account for these differences and understand how youth are accessing NRT.

Developing a Master Data Sharing Agreement: Seeking Student-Level Evidence to Support a Collaborative Community Effort in Education

· A private foundation, a public school system, and a state university joined forces to address a difficult, long-standing challenge: closing the academic achievement gap between urban and suburban students. · All parties agreed that sharing of longitudinal, student-level data was required to drive and evaluate multiple efforts to close the gap, but significant technical, regulatory, and political obstacles stood in the way. · The parties worked through multiple challenges and forged a Master Data Sharing Agreement (MDSA) that will facilitate both daily intelligence for program staff and powerful post-hoc research capacity. · This MDSA text has been released online for your use under the Creative Commons license (Community Research Institute, 2011a). · Reaching the agreement required a shared vision, definitive research, genuine trust, true alignment, dogged patience, ample investment, iterative development, selfless collaboration, careful coordination, and fidelity to a common language: data.

Mobilizing for Change: A Case Study of a Campus and Community Coalition to Reduce High-Risk Drinking

Campus and community coalitions include a partnership between campus leaders and community stakeholders and can effectively address the environment that may promote high-risk drinking. Despite evidence suggesting that coalitions may be effective vehicles for producing sustainable changes in college drinking, few campuses work within such a structure. The purpose of this study was to examine the effectiveness of a campus and community coalition to implement environmental changes and thereby reduce high-risk drinking and associated consequences. This study utilized a case study method to tell the story of a campus and community coalition (CCC) implemented on a large university campus in the Northeast. The study employed multiple methods including archival document review, review of campus and community level data (i.e. alcohol-related arrests and sanctions) and analysis of student level data. The case study discusses the strategies employed, the environmental changes that occurred and the impact these changes have had on student drinking and consequences. Since implementing the campus and community coalition, the campus has seen an increase in enforcement by campus and local police, changes in community by-laws, and significant reductions in student drinking and consequences. The data provide evidence that a comprehensive approach to reducing high-risk drinking can have an impact on the campus and community environment, which in turn impacts student drinking and associated consequences. The CCC utilized a strategic and comprehensive approach to substance abuse prevention, allowing all participants to have a shared understanding of the challenges and best practices. Implications for research and practice are also discussed.

School competition and students’ entrepreneurial intentions: international evidence using historical Catholic roots of private schooling

School choice research mostly focuses on academic outcomes. Policymakers increasingly view entrepreneurial traits as a non-cognitive outcome important for economic growth. We use international PISA-2006 student-level data to estimate the effect of private-school competition on students’ entrepreneurial intentions, measured by their occupational desire to manage a small enterprise. We exploit Catholic-Church resistance to state schooling in the 19th century as a natural experiment to obtain exogenous variation in current private-school shares. Our instrumental-variable results suggest that a 10%-point higher private-school share raises students’ entrepreneurial intentions by 0.3–0.5% points (11–18% of the international mean) even after controlling for current Catholic shares, students’ academic skills, and parents’ entrepreneurial occupation.

Developing and applying a socio-ecological model to the promotion of healthy eating in the school.

To develop and apply a socio-ecological model (SEM) for healthy eating in school students, to better understand the association between factors at different levels of the SEM and pupils' dietary choices. Student-level data, collected through anonymised questionnaires, included reported dietary choices and correlates to these; data on school approaches to food were collected through postal surveys. We used multilevel analysis to study the association of each level of the SEM on student dietary choice while controlling for factors found at other levels. Data were collected from secondary schools in Wales that were a part of the 2005/2006 Health Behaviour in School-aged Children (HBSC) study. The final sample for analysis included data from 6693 students aged 11-16 years and 289 teachers from sixty-four secondary schools in Wales. Student interpersonal factors, an individual's social environment, had a greater association with the dietary choices students made for lunch than student intrapersonal characteristics, those that reside within the person, which were found to have a greater association with the dietary choices made outside school. School organisational factors, such as rules and policies, had a greater association with whether students ate unhealthy foods, whereas the community nature of the school had a greater association with the choosing of healthy foods. Using the SEM and multilevel analysis allowed us to study how factors were associated with the choice of different foods at different times of the day by students. Interventions can use an SEM to target specific correlates and change health outcomes in the school.

Measuring the effect of charter schools on public school student achievement in an urban environment: Evidence from New York City

This paper uses student level data from New York City to study the relationship between a public school losing enrollment to charter school competitors and the academic achievement of students who remain enrolled in it. Geographic measures most often used to study the effect of school choice policies on public school student achievement are not well suited for densely populated urban environments. I adopt a direct approach and measure charter school exposure as the percentage of a public school's students who exited for a charter school at the end of the previous year. Depending on model specification, I find evidence that students in schools losing more students to charter schools either are unaffected by the competitive pressures of the choice option or benefit mildly in both math and English.

The Battle River Project: school division implementation of the health-promoting schools approach

The Battle River Project (BRP) is a school division-level intervention in rural Alberta, Canada, built upon the health-promoting schools approach to health promotion. Using self-reported school and student-level data from administrators and students, the central aim of the BRP is to examine: 'How can the school environment and health behaviours (healthy eating, physical activity and mental wellness) of children and youth be improved when a health-promoting schools model, the Ever Active Schools program, is implemented with school division support?' Evidence used to inform school level changes included students' demographic, behavioral, and psychosocial variables linked to school environment data, comprised of school demographics and administrator-assessed quality of policies, facilities, and programs related to physical activity. Each participating school and the division were provided with a tailored report of their schools' results to reflect, plan and implement for positive health behavior change. The main lesson learned was that sharing school-specific evidence can operate as a catalyst for embedding health promoting policy and practices within the school and division culture.

Do School Incentives and Accountability Measures Improve Skills in the Middle East and North Africa? The Cases of Jordan and Tunisia

There is general agreement that skill-enhancing school reforms in the Middle East and North Africa (MENA) region are necessary for economic, political and social reasons. Using student-level data from Jordan and Tunisia, this study assesses the relationship between skills and the following school incentive and accountability measures: pedagogical autonomy, school competition, freedom to hire and fire teachers, publicly posting data, and parent involvement in school affairs. Quantile regression analyses of mathematics, science, and reading skills of 15-year-old students suggest that students in schools with incentive and accountability measures do not have higher skills than students in school without the measures; this suggests that schools with incentive and accountability measures are no more efficient than other schools that have not adopted the measures. In terms of equity, the reforms are not associated with higher skills for the less skilled; a notable exception is parent involvement in Tunisia, which is associated with higher science and reading skills among low-skilled students. The main policy implication is that school incentive- and accountability-based reform should not be pursued until researchers have identified the effective design properties of each incentive and accountability measure.

Imposing monotonicity on outputs in parametric distance function estimations

The technology set involved in the estimation of a multi-output production frontier theoretically implies monotonicity on outputs. This is because an efficient firm cannot reduce the vector of outputs holding the vector of inputs fixed while it still belongs to the frontier. In empirical studies dealing with the estimation of parametric distance functions, this hypothesis is often violated by observations with far from average characteristics. To overcome this limitation, we propose a new approach for allowing the easy imposition of monotonicity on outputs in this context. This methodology is tested in the educational sector using Spanish student level data from the Programme for International Student Assessment (PISA) database. The results indicate that a nonnegligible 8.33% of the production units break the monotonicity assumption. Furthermore, although there is no statistically significant difference in efficiency distribution by school ownership, our methodology helps to detect a slight worse mathematical performance for students attending public schools.

The Sorting Effect of Charter Schools on Student Composition in Traditional Public Schools

This article investigates how Michigan’s charter school policy influences the composition of students by race and socioeconomic status in urban traditional public schools. Using 2 years of student-level data in Michigan’s urban elementary and middle schools, the dynamic student transfers between charter schools and TPSs are analyzed through a series of hierarchical generalized linear models. The two-way transfer analysis shows that the student sorting under the charter school program tends to intensify the isolation of disadvantaged students in less effective urban schools serving a high concentration of similarly disadvantaged students. The findings imply that a challenge for the state policy makers is to help disadvantaged students who are left behind in the most disadvantaged schools, without significantly reducing the benefits to students who take advantage of school choice.

Impact of Family vs. School Factors on Cross-National Disparities in Academic Achievement: Evidence from the 2006 PISA Survey

This paper investigates whether international differences in math and science achievement, and specifically the lower US scores, can be explained by school programmatic, institutional, and resource differences after controlling for family SES factors. Using 2006 PISA student-level data for the 10 largest developed OECD countries, the results show that, while family SES has a strong impact on students’ achievement, it does not explain the US achievement gap with other developed OECD countries. In contrast, a substantial number of school variables not only have significant impacts on math and science achievement, but they contribute more to these gaps than SES differences. Of particular importance for policy purposes are the lower amounts of time devoted to studying math and science in the US, as well as student-centered pedagogical techniques which are emphasized in the US but not in such high-scoring countries such as Korea.

Relationship between teacher knowledge of concepts and connections, teaching practice, and student achievement in middle grades mathematics

The mixed method sequential nested study examines whether and how the cognitive type of teachers’ content knowledge is associated with student achievement, and correlated with teaching practice. In the context of this study, the cognitive type refers to the kind of teacher content knowledge and thinking processes required to accomplish a task successfully, in terms of knowledge of facts and procedures (Type 1), knowledge of concepts and connections (Type 2), and/or knowledge of models and generalizations (Type 3). A sample of 102 middle school mathematics teachers (grades 6–8) was assigned to the study from a number of school districts in an urban area in the Southwestern US. Teachers were tested using the Teacher Content Knowledge Survey. Student level data of about 2,400 middle grades students’ standardized test passing rates including percentage of students meeting the state standards by objectives were collected. The type of teachers’ content knowledge was assessed and tested for association with student achievement on the state-mandated standardized test using multivariate methods including tests for variance and independence. The nested research consisted of three phases. Substudy-1 focused on quantitative analysis of the association between cognitive type of teacher content knowledge and student achievement. Substudy-2 aimed at the correlation between cognitive type of teacher content knowledge and teaching practice. Finally, substudy-3 was a case study on examining middle grades mathematics teachers’ knowledge and understanding of fraction division.

A Multilevel Examination of School and Student Characteristics Associated With Physical Education Class Enrollment Among High School Students

Schools can be an efficient venue for promoting physical activity (PA) among adolescents. Physical education (PE) requires investigation because it is a variable associated with adolescent PA levels and its existence in schools represents a significant opportunity for strategies to combat declining PA levels among this population. This article examines the between-school variability in student rates of PE enrollment among a large sample of high schools in Ontario, Canada, and identifies the school- and student-level characteristics associated with PE enrollment. This cross-sectional study utilized self-reported school- and student-level data from administrators and students at 73 high schools. Students' enrollment in PE, demographic, behavioral, and psychosocial variables was linked to school environment data comprising of school demographics and administrator assessed quality of policies, facilities, and programs related to PA. Analysis involved multilevel modeling. The mean rate of PE enrollment among the 73 high schools was 62.4%, with rates by school ranging from 28.9% to 81.1%. When student demographics, behavioral, and psychosocial factors were controlled for, there was still a school effect for student PE enrollment. The school effect was explained by the provision of daily PE and school median household income. This is the first study to examine the extent to which PE enrollment varies between schools and to identify school factors associated with school variability in rates of PE enrollment. Although most variation in PE enrollment lies between students within schools, there is sufficient between-school variation to be of interest to practitioners and policy makers.

Who succeeds in STEM studies? An analysis of Binghamton University undergraduate students

Using student level data, the characteristics of STEM and Non-STEM students are examined for attributes associated with academic success. We use fixed effects models to analyze the variables’ role in attaining graduation and college GPA and find preparation and ability, as evidenced by Advanced Placement course work, mathematical ability, gender, ethnicity, high school GPA and college experience are all statistically significant indicators of success. These attributes may confer a comparative advantage to STEM students. The engineers have statistically significant differing response elasticities than the non-engineers, and show evidence of persistence that may arise from learning-by-doing. A successful engineering STEM major at Binghamton has good mathematics preparation, and disproportionately is of Asian ethnicity. Women are few in numbers as engineers. Other STEM fields see less emphasis on mathematics preparation, but more emphasis on the presence of AP course work. Women have the same presence in these other STEM fields as in the whole university.

The Retention Index of the Community College Survey of Student Engagement (CCSSE): How Meaningful Is It?

Institutional and student level data from the Florida Community Colleges were examined to explore the Retention Index of the Community College Survey of Student Engagement (CCSSE). A simple correlation and regression analysis was conducted to determine if the CCSSE Retention Index could be used to predict the institutional retention rate calculated for the state accountability measure. Student level data were used with a nested ANOVA to examine if there were mean differences in the CCSSE Retention Index scores of males and females of different racial groups when accounting for the potential influence of the institution attended. Mean differences were examined since differences in retention rates of these groups have been documented.

Do School Incentives and Accountability Measures Improve Skills in the Middle East and North Africa? The Cases of Jordan and Tunisia

There is general agreement that skill-enhancing school reforms in the Middle East and North Africa (MENA) region are necessary for economic, political and social reasons. Using student-level data from Jordan and Tunisia, this study assesses the relationship between skills and the following school incentive and accountability measures: pedagogical autonomy, school competition, freedom to hire and fire teachers, publicly posting data, and parental involvement in school affairs. Quantile regression analyses of mathematics, science, and reading skills of 15-year old students suggest that students in schools with incentive and accountability measures do not have higher skills than students in school without the measures; this suggests that schools with incentive and accountability measures are no more efficient than other schools which have not adopted the measures. In terms of equity, the measures are not associated with higher skills for the less-skilled; a notable exception is parental involvement in Tunisia, which is associated with higher science and reading skills among low-skilled students. The main policy implication is that school incentive- and accountability-based reform should not be pursued until researchers have identified the effective design properties of each incentive and accountability measure.

Ranking the Schools: How Quality Information Affects School Choice in the Netherlands

This paper analyzes whether information on high school quality published by a national newspaper affects school choice in the Netherlands. For this purpose, we use both school level and individual student level data. First, we study the causal effect of quality scores on the influx of new high school students using a longitudinal school dataset. We find that negative (positive) school quality scores decrease (increase) the number of students choosing a school after the year of publication. The positive effects are particularly large for the academic school track. An academic school track receiving the most positive score sees its inflow of students rise by 15 to 20 students. Second, we study individual school choice behaviour to address the relative importance of the quality scores, as well as potential differences in the quality response between socio-economic groups. Although the probability of attending a school is affected by its quality score, it is mainly driven by the travelling distance. Students are only willing to travel about 200 meters more in order to attend a well-performing rather than an average school. In contrast to equity concerns that are often raised, we cannot find differences in information responses between socio-economic groups.

School Competition and Students' Entrepreneurial Intentions: International Evidence Using Historical Catholic Roots of Private Schooling

School choice research mostly focuses on academic outcomes. Policymakers increasingly view entrepreneurial traits as a non-cognitive outcome important for economic growth. We use international PISA-2006 student-level data to estimate the effect of private-school competition on students' entrepreneurial intentions. We exploit Catholic-Church resistance to state schooling in 19th century as a natural experiment to obtain exogenous variation in current private-school shares. Our instrumental-variable results suggest that a 10 percentage-point higher private-school share raises students' entrepreneurial intentions by 0.3-0.5 percentage points (11-18 percent of the international mean) even after controlling for current Catholic shares, students' academic skills, and parents' entrepreneurial occupation.