Program-level Data Research Articles

The news hour: Welfare estimation in the market for local television news

We estimate the welfare consequences of local news broadcasting decisions in advertiser-funded television, a central question in media regulation. We model programming decisions as the outcome of a discrete game played by rival stations competing for advertising revenue (which depends on viewing) by choosing lineups of local news and entertainment broadcasts. Using program-level data on television viewing and advertising prices, we find modest under-provision of local news relative to the level that maximizes television viewing. Counterfactual simulations indicate an average deficit of 7.4 broadcasts per market, or 12.8% of local station broadcasts during the evening news hours. Most of this shortfall is in the 7:30 timeslot leading into prime time. We distinguish two sources of inefficiency: losses due to advertiser valuation of local news and entertainment viewers, and losses from classic business stealing. Losses from competition represent about one third of the estimated shortfall, suggesting gains to cooperation among competing stations.

Institution, Major, and Firm-Specific Premia: Evidence from Administrative Data

We examine how a student’s field of degree and institution attended contribute to the labor market outcomes of young graduates. Administrative panel data that combines student transcripts with matched employer-employee records allow us to provide the first decomposition of premia into individual and firm-specific components. We find that both major and institutional premia are more strongly related to the firm-specific component of wages than the individual-specific component of wages. On average, a student’s major is a more important predictor of future wages than the selectivity of the institution attended, but major premia (and their relative ranking) can differ substantially across institutions, suggesting the importance of program-level data for prospective students and their parents.

Surviving and thriving: How changes in teaching modalities influenced student satisfaction before, during and after COVID-19

This paper leverages analytics methods to investigate the impact of changes in teaching modalities shaped by the COVID-19 pandemic on undergraduate students’ satisfaction within a Spanish brick-and-mortar higher education institution. Unlike research that has focused on faculty- or programme-level data, this study offers a comprehensive institutional perspective by analysing large-scale data (N = 83,532) gathered from satisfaction surveys across all undergraduate courses in eight faculties from 2018 to 2021. The longitudinal analysis revealed significant changes (p < 0.05) in satisfaction indicators, particularly overall satisfaction and perceived workload. During the emergency remote teaching period, there was a significant decrease in satisfaction and high levels of variability across courses. However, a year after emergency remote teaching, with increased implementations of technology-supported online and mixed teaching modalities, satisfaction measures not only recovered but exceeded pre-COVID levels in the aforementioned indicators when the teaching modality was fully co-located. The variability of answers also reached historical lows, reflecting more uniform student experiences. These findings highlight the resilience of educators and the current higher education system and suggest a capacity to learn and improve from disruptive pedagogical changes. The study also provides insights into how data analytics can help monitor and inform the evolution of teaching practices. Implications for practice or policy Higher education institution administrators should improve the understanding of the effects derived from changes in their teaching and learning models, for example, in teaching modalities and related technology support. Student satisfaction data analytics offer useful indicators to study the impact of those effects. Higher education institutions should provide support for educators to ensure minimal deviations from expected averages of educational quality indicators regardless of the educators’ capacity to adapt to changes in the teaching models.

Impact of Integrated Vascular Surgery Residency on General Surgery Resident and Vascular Fellow Operative Volume: A National Analysis.

The impact of integrated vascular surgery (VS) residency (0 + 5) programs on general surgery (GS) resident and VS fellow (5 + 2) operative volume has not been investigated on a national scale. Accreditation Council for Graduate Medical Education (ACGME) case logs were reviewed for GS resident, VS resident, and VS fellow operative volume from 2001-2021. Integrated VS resident data was available from 2012-2021, corresponding with the introduction of the 0 + 5 paradigm. Trends in operative volume were evaluated via linear regression analysis. The national cohort of chief GS resident graduates increased from 1005 to 1357 per year. Total operative volume also increased from 932 to 1039 cases (+7.4 cases/yr, R2 = .80, P < .0001) among GS residents. Major vascular cases decreased among GS residents from 138 to 101 cases (-2.4 cases/yr, R2 = .58, P < .0001) with a decrease in proportion of chief-level vascular cases from 30.4% to 11.9% (-1.0%/yr, R2 = .92, P < .0001). Palliative procedures (amputations and hemodialysis access) comprised a significant proportion of GS cases (median 44.7%). Concurrently, integrated VS graduates increased from 11 to 37 per year, with an increase in major vascular case volume from 506 to 658 cases (+18.4 cases/yr, R2 = .63, P = .01). Total VS fellow major case volume also increased from 369 to 444 cases (+3.5 cases/yr, R2 = .73, P < .0001). The introduction of the 0 + 5 intgrated VS residency paradigm has correlated with a significant decrease in GS operative experience in major vascular procedures on a national level. Traditional VS fellow case volume does not appear to be impacted by 0 + 5 integrated residents. Further analysis with program-level data may help to explain the causative relationship of these findings.

Study protocol for a multi-level cross-sectional study on the equitable reach and implementation of coordinated specialty care for early psychosis

BackgroundApproximately 115,000 young adults will experience their first episode of psychosis (FEP) each year in the USA. Coordinated specialty care (CSC) for early psychosis is an evidence-based early intervention model that has demonstrated effectiveness by improving quality of life and reducing psychiatric symptoms for many individuals. Over the last decade, there has significant increase in the implementation of CSC programs throughout the USA. However, prior research has revealed difficulties among individuals and their family members accessing CSC. Research has also shown that CSC programs often report the limited reach of their program to underserved populations and communities (e.g., ethnoracial minorities, rural and low socioeconomic neighborhoods). Dissemination and implementation research focused on the equitable reach and implementation of CSC is needed to address disparities at the individual level.MethodsThe proposed study will create a novel integrative multi-level geospatial database of CSC programs implemented throughout the USA that will include program-level data (e.g., geocoded location, capacity, setting, role availability), provider-level data (race, ethnicity, professional credentials), and neighborhood-level census data (e.g., residential segregation, ethnic density, area deprivation, rural-urban continua, public transit time). This database will be used to characterize variations in CSC programs by geographical location and examine the overall reach CSC programs to specific communities. The quantitative data will be combined with qualitative data from state administrators, providers, and service users that will inform the development of dissemination tools, such as an interactive dashboard, that can aid decision making.DiscussionFindings from this study will highlight the impact of outer contextual determinants on implementation and reach of mental health services, and will serve to inform the future implementation of CSC programs with a primary focus on equity.

Building an empirically grounded conceptual frame for business networking as a mechanism for pro-environmental business growth

Whilst there is a growing body of evidence relating to the environmental capabilities, eco innovation, sustainable supply chains and green skills of individual SMEs (small and medium-sized enterprises), much less attention has been paid to networked approaches to pro-environmental and sustainable SME support. The empirical studies that do exist mostly take a firm-centric approach. This study takes a network centric view to describe hitherto under-explored features of network functionality based on case analysis of a single business support network in the East Midlands of the UK. The resulting conceptual framework offers an analytical tool that can be used to study network functionality in the service of pro-environmental SME development more widely. Empirical data for this case analysis of a regional ‘low carbon’ business network is drawn from interviews with 25 stakeholders, analysis of programme level data and documentation, ‘insider’ observation of network activity and evidence from both independent and internal evaluations. The study charts key aspects of network provision and its impact upon SMEs and wider stakeholders. The paper then presents a conceptual model of pro-environmental business networking that identifies key internal features of network activity as well as various external drivers of multi-stakeholder engagement in the shift to a more sustainable local economy.

Screening and Brief Intervention With Low-Income Youth in Community-Based Settings

PurposeWe described screening, brief intervention, and referral to treatment (SBIRT) results and assessed whether SBIRT is associated with positive changes in substance use, risky use, and educational/employment outcomes for youth in community-based settings that are not healthcare focused. MethodsYouthBuild USA serves youth of ages 16–24 who are neither in school nor employed. In an SBIRT intervention, youth completed substance use surveys and Alcohol Use Disorders Identification Test and Drug Abuse Screening Test screenings at entry and program completion. Staff reported on services provided in response to screening scores. Regression models compared changes in youth screening results and substance use from intake to follow-up and, with aggregate program-level data, youth outcomes across programs with and without the SBIRT intervention. ResultsYouth significantly reduced Alcohol Use Disorders Identification Test (3.1 vs. 2.3, p < .001) and Drug Abuse Screening Test (1.9 vs. 1.4, p < .001) scores, positive screens (64% vs. 54%, p < .001), and need for referrals to treatment (48% vs. 37%, p < .001), indicating less risky substance use, although self-reports of substance use in the past 30 days did not decrease. Proportionately more youth in SBIRT programs attained a high school diploma or equivalent (49% vs. 42%, p = .01) and were still in educational/job placements 3 months after program completion (67% vs. 59%, p = .02), compared to youth in non-SBIRT programs. DiscussionThese findings suggest that community-based SBIRT is associated with positive outcomes–both reduced risky substance use and improved education and employment–that relate to longer-term positive development for youth. SBIRT appears to be an evidence-based approach to intervene and help youth.

Organizational characteristics of Brain Injury Clubhouse Model programs

ABSTRACT Objective We provide an overview of the Clubhouse Model and the history and development of Brain Injury Clubhouses. We describe organizational-level characteristics associated with eight Brain Injury Clubhouses to address gaps in the literature and inform future studies or program development. Methods A electronic survey, the Clubhouse Profile Questionnaire (CPQ) was tailored for Brain Injury Clubhouses. The CPQ gathers program-level data that can be used to identify active ingredients of Clubhouses, understand best practices, examine, and evaluate program characteristics. The brain injury version of the CPQ was administered to a sample of eight Clubhouses affiliated with the International Brain Injury Clubhouse Association as part of a project designed to gather data on Clubhouse program characteristics and describe sociodemographic characteristics of people served by Brain Injury Clubhouses. Results CPQ data from eight Brain Injury Clubhouses was analyzed. Brain Injury Clubhouse programs in this sample served approximately 17 members per day. There was wide variability in the size, funding and funding mechanisms, and length of operation of Brain Injury Clubhouses in this study. Conclusions Findings suggest that Brain Injury Clubhouses offer a wide range of services and supports. Additional research on the impact of Brain Injury Clubhouses is needed.

A National Study on the Relationship Between Programmatic Factors and Athletic Training Education Board of Certification Pass Rates

Context In athletic training education, the first-time Board of Certification (BOC) pass rate is a significant marker of a program's success, and the Commission on Accreditation of Athletic Training Education (CAATE) and requires programs to maintain a first-time 3-year aggregate BOC examination pass rate over 70% (Standard 11). Published research on BOC pass rates on professional master's (PM) programs is limited. Therefore, it is essential to identify modifiable factors that have a relationship with first-time pass rates. Objective The aim of this study aim was to investigate the relationships between programmatic factors and the first-time BOC pass rate for PM athletic training students while controlling for student and institutional factors. This study is necessary to fill the literature gap and identify PM programmatic factors that may be significant in predicting student success in PM athletic training programs. Design Cross-section study. Setting A multiple regression analysis of program-level data that captured student, programmatic, and institutional factors obtained from the deidentified CAATE data was conducted on 77 PM athletic training programs in the 2018–2019 reporting year. Main Outcome Measure(s) Independent variables included in the study were institutional type, admissions selectivity, cohort diversity, clinical immersion hours per week, students per core faculty member, students per lab faculty member, percent doctoral faculty, and total spending on professional development. The dependent variable was the programmatic 1-year, first-time BOC pass rate. Results A significant positive relationship was found between admissions selectivity, clinical immersion hours per week, percentage of doctoral faculty, and total amount spent on professional development and 1-year, first-time BOC program pass rates. These results suggest that increased programmatic investments into development of faculty and the evaluation of the clinical immersive experience may help programs increase their first-time BOC pass rate.

Future Directions: Study Protocol for an Effectiveness-Implementation Hybrid Evaluation of a State-based Social Housing Strategy and Three Social Housing Programs

Background: In the Australian state of New South Wales nearly 60,000 approved applicants are waiting for social housing. Future Directions for Social Housing is a response to this challenge. This collection of housing programs aims to provide more social housing, support and incentives for leaving social housing and a better social housing experience. This document presents the protocol of the evaluation of these programs and the overarching Future Directions Strategy.&#x0D; Methods/Design: The evaluation will use a Type 1 effectiveness-implementation hybrid design, with an integrated, dual focus on assessing the effectiveness of Future Directions and better understanding the context for reform implementation. Program effectiveness will be examined using quasi-experimental techniques applied to linked administrative data. The implementation context will be examined via program level data, qualitative interviews and focus groups with stakeholders and tenants. Some quantitative survey and administrative data will also be used. Findings from the implementation evaluation will be used to inform and interpret the effectiveness evaluation. Economic evaluations will also be conducted.&#x0D; Discussion: This methodology will produce a high-quality evaluation of a large, complex government program which aims to facilitate rapid translational gains, real-time adoption of effective implementation strategies and generate actionable insights for policymakers.

Gender disparities in access and retention in outpatient methadone treatment for opioid use disorder in low-income urban communities

The purpose of this study was to detect and understand gender disparities in access and retention among outpatient methadone treatment programs located in low-income urban communities in Los Angeles, California. The study collected client- and program-level data in 4 waves in 2011, 2013, 2015, and 2017 from 34 publicly funded methadone treatment programs serving 11,169 clients with opioid use disorder (OUD). The sample included 29.8% female and 70.2% male clients, where 10.6% identified as Black or African American, 41.5% as Latino, 44.2% as non-Latino white, and 3.8% as Other. We conducted two multilevel negative binomial regression models to examine direct and moderated relationships related to both access (days on the waitlist) and retention (days in treatment) while accounting for clients clustered within programs. Gender disparities existed in both access and retention where women spent more time than men waiting to enter treatment but then remained in treatment longer. Further, female clients identifying as African American, Latino, and Other were at greater risk for shorter treatment duration than those who identified as non-Latino white and men. Overall, OUD clients receiving methadone treatment in low-income neighborhoods experienced barriers to access and retention in treatment associated with mental illness, family responsibilities, and use severity. OUD clients with MediCal insurance eligibility were consistently more likely to gain access to and remain in methadone treatment. Overall, findings call for improving treatment access and retention for women with OUD who receive methadone in outpatient methadone treatment programs through comprehensive, gender-specific, and evidence-based programming.

Do Promise Programs Reduce Student Loans? Evidence from Tennessee Promise

ABSTRACT As college promise programs proliferate across the United States with noted intentions to promote access through increased affordability, it is necessary to understand the relationship between these programs and other forms of financial aid, including loans. Using federal, state, and program-level data, we leverage a natural experiment to estimate causal impacts of the nation’s first statewide program (Tennessee Promise) on students’ borrowing behaviors in a community college system. In the difference-in-differences framework, we find robust evidence to suggest Tennessee Promise reduced the percent of first-time, full-time students borrowing by 8–10 percentage points on average after implementation—an over 40% decline—and reduced the average community college cohort loan by 230-360 USD (nearly 32%). While consistent with prior work on grant aid, these findings are among the first to explore promise students’ financial outcomes. We discuss these results in relation to Tennessee Promise’s programmatic features and identify important implications for policy and research.

Faculty Staffing Patterns: Breadth and Flexibility in Professional Physical Therapy Programs

Overall trends of academic staffing in the US have indicated declines in tenure, increased use of contingent faculty, and stratification of teaching and research efforts. However, little is known about academic staffing patterns in professional programs, and little research has been done using program-level data. The purposes of this study were to identify faculty staffing patterns, determine if patterns change over time, and identify predictors of staffing patterns in US-based physical therapy programs. Yearly program-level accreditation data from 2008 to 2017 were analyzed. A finite mixture model analysis identified staffing patterns. Logistic regression analyses were run to predict category membership and observe change over time. Two academic staffing categories emerged: (1) Broad staffing patterns (90.4% of programs) composed of more core faculty on tenure track and who held academic degrees and (2) Flexible programs (9.7% of observations) with higher numbers of core faculty, fewer faculty on tenure track, and fewer faculty with an academic doctoral degree. There was little change over time from one category to another. Programs were more likely to adopt a Flexible pattern when budget and faculty teaching time rose. Membership to one of two staffing models appears to be predicted by institutional reliance on professional programs for revenues. Either programs lower expenses by having tenured faculty balance teaching and research, or they rely on non-tenured faculty positions and higher enrollments.

Analysis of the pediatric surgery fellowship application process using the Thalamus™ database

BackgroundThe pediatric surgery fellowship interview process is costly and time intensive. We hypothesized that the increasing number of interviews completed by applicants and programs have become inefficient over time. MethodsWe analyzed pediatric surgery fellowship program and applicant interview data between 2018 and 2020. Cancellations, program fill time, regional analysis of programs and applicants, and program rank list data were also captured. Analyses were performed using descriptive statistics and Chi-Square analysis. ResultsOur dataset included 34, 41, and 45 programs, which represented 81%, 91%, and 97% of all programs in 2018, 2019, and 2020, respectively. The median number of interviews completed per program remained constant, while the median number of interviews per applicant increased from 9.0 in 2018 to 13.0 in 2020. For 75% of programs, a program required only 4 or less candidates to fill their position. On average, 96% of program interviews do not result in a matched candidate. ConclusionsPrograms offer interviews out of proportion to the number of positions available, and most applicants attend all interviews offered. We recommend an initial program goal of 20 interviews, which may be achieved by increased use of virtual interviews and the creation of program-level data on ideal applicant profiles.

Toolkit for assessing and addressing cognitive health in early psychosis: Evaluation of feasibility and utility in a coordinated specialty care setting.

Methods to identify and harness individual cognitive strengths while addressing relative weaknesses have the potential to complement recovery services for first-episode psychosis but systematic implementation is needed. We developed a cognitive health toolkit, trained teams from OnTrackNY, a network of coordinated specialty care (CSC) programs and examined toolkit feasibility and clinical utility during the first year of roll-out. The toolkit includes a clinician manual, assessment and decision-making tools, and a menu of cognitive health service options. Assessment uses the WRAT5-Reading subtest and a new Self-Assessment of Cognitive Functioning which, together, determine participant- and clinician-perceived cognitive health need. Program-level data were analysed for rates of assessment, identification of cognitive health needs and cognitive health service provision. Data from 18 OnTrackNY teams included 933 participants, including 310 new admissions. Across teams, 43.9% of newly admitted participants received a WRAT5-Reading and 41.3% received a self-assessment. Of all assessments completed in the study period, 50.7% were at or within 3-months of admission and 69.1% were within the first year of program participation. Cognitive health need was identified by self-report (57.6%) and clinician-report (69.9%) and led to provision of specific services, including psychoeducation, compensatory skills training and in some cases restorative cognitive training. Preliminary feasibility data are encouraging but barriers to assessment need to be identified and addressed. Rates of identified cognitive health need warrant further study of the implementation process and outcomes so that cognitive health assessment and treatment practices may ultimately be disseminated to CSC programs more broadly.

O-HEALTH-EDU: A scoping review on the reporting of oral health professional education in Europe.

The variability in oral health professional education is likely to impact on the management of oral health needs across Europe. This scoping review forms the initial part of a larger EU-funded collaborative Erasmus+project, 'O-Health-Edu'. The aim of this scoping review is to investigate how oral health professional education in Europe is reported. The PRISMA and Arksey & O'Malley methodological frameworks for scoping reviews were used to guide reviewers in answering the research question "How is oral health professional education reported in Europe?". The search strategy encompassed published literature searches, internet searches and further searching of relevant documents from educational organisations, regulators and professional bodies. Once the search strategy was developed, it was sent to key stakeholders for consultation. Sources were reviewed by two authors (JD, JF) and included in the review if they reported on oral health professional education in Europe. A total of 508 sources were retrieved from all of the searches. A total of 405 sources were excluded as they did not report on the topic of interest, leaving 103 sources that reported on oral health professional education in Europe. Handsearching the references of published sources lead to a further 41 sources being screened, of which, 15 were included. In total, 33 duplications were removed and the final number of included sources was 85. The average year of publication for the included sources was 2007, with sources most commonly published in journals dedicated to dental education. Surveys represented the most common form of reporting. From the data obtained, four broad themes of reporting were evident: dental education at a programme level, dental education at a discipline level, other oral health professional education, and postgraduate education and continuous professional development. The reporting of dental and oral health professional education in Europe is limited. Whilst there are many useful documents that provide guidelines on dental education, there is limited knowledge on how education is implemented and delivered. There is a greater need for comprehensive educationally driven programme-level data on oral health professional education across Europe.

Competing for Bachelor’s Degrees: Are Community Colleges Cutting Into the Market Share of 4-Year Institutions?

To address local workforce needs and expand access to affordable bachelor’s degrees, some states allow community colleges to offer bachelor’s degree programs. Despite concerns that community college baccalaureate (CCB) programs will duplicate efforts and cut into the market share of nearby 4-year institutions, extant literature has yet to examine the impact of CCB adoption on bachelor’s degree program enrollment and bachelor’s degree production at 4-year institutions. Using program-level data, our findings show that local CCB degree programs have a negative effect on overall bachelor’s degree enrollment and bachelor’s degree production at 4-year institutions, but this effect is concentrated primarily within for-profit 4-year institutions. This study represents the first comprehensive evaluation of the impact of CCB degree programs on neighboring 4-year institutions.

Development of a Patient-specific Search of Transplant Program Outcomes and Characteristics: Feedback From Kidney Transplant Patients.

Background.Patients face obstacles in finding a transplant program that meets their healthcare needs. Acceptance criteria and waiting times vary by region and program. The Scientific Registry of Transplant Recipients provides program-specific information, but it is unclear what patients and referring physicians need to know.Methods.We examined variability in program-specific characteristics that could influence access to transplantation. We also conducted 20 interviews and 16 focus groups with transplant candidates, recipients, and their family members. Participants were shown prototypes of a patient-specific search tool and evaluated its capacity to identify programs tailored to the needs of individual patients.Results.The distribution of recipient and donor characteristics that may impact access to transplantation, such as recipients on Medicaid, varied across programs (all with P < 0.01). Several themes emerged related to impressions of access to transplantation and the usability of patient-specific search functions. Perceptions of the prototypes and results varied, but were positive overall and support providing an individualized search of program level data. Participants revealed significant barriers to identifying and evaluating transplant programs and suggest that patient-specific search results reduce the anxiety associated with selecting a program.Conclusions.Providing patient-specific tools is valued by patients and important to maximizing access to transplant.

MP17: Evaluation of a national competency-based assessment system in emergency medicine: A CanDREAM study

Introduction: In 2018, Canadian postgraduate specialist Emergency Medicine (EM) programs began implementing a competency-based medical education (CBME) assessment system. To support improvement of this assessment program, we sought to evaluate its short-term educational outcomes nationally and within individual programs. Methods: Program-level data from the 2018 resident cohort were amalgamated and analyzed. The number of Entrustable Professional Activity (EPA) assessments (overall and for each EPA) and the timing of resident promotion through program stages was compared between programs and to the guidelines provided by the national EM specialty committee. Total EPA observations from each program were correlated with the number of EM and pediatric EM rotations. Results: Data from 15 of 17 (88.2%) EM programs containing 9,842 EPA observations from 68 of the 77 (88.3%) Canadian EM specialist residents in the 2018 cohort were analyzed. The average number of EPAs observed per resident in each program varied from 92.5 to 229.6 and correlated strongly with the number of blocks spent on EM and pediatric EM (r = 0.83, p &lt; 0.001). Relative to the guidelines outlined by the specialty committee, residents were promoted later than expected and with fewer EPA observations than suggested. Conclusion: We present a new approach to the amalgamation of national and program-level assessment data. There was demonstrable variation in both EPA-based assessment numbers and promotion timelines between programs and with national guidelines. This evaluation data will inform the revision of local programs and national guidelines and serve as a starting point for further reaching outcome evaluation. This process could be replicated by other national assessment programs.

SPREAD OF QUALITY IMPROVEMENT PROJECTS IMPLEMENTED IN A GERIATRICS WORKFORCE DEVELOPMENT PROGRAM

This project characterizes spread of quality improvement (QI) projects initiated in the U.S. Department of Veterans Affairs (VA) Geriatric Scholars Program (GSP) workforce development program. This mixed methods study analyzes a recent cross-sectional survey of GSP participants and program-level data on participant characteristics and QI project topics. We surveyed 578 scholars who had completed all program requirements to that point, and still worked for VA; 207 (35%) responded. The majority of respondents who had been in the program for at least six months (70%) reported sustainment of their QI project beyond initial implementation and nearly a third (30.4%) reported any spread beyond their own care team. QI project topics spanned many domains and percent of projects reporting spread varied across domains from 0% to 67%. A workforce development capstone activity in which participants demonstrate substantive and methodological competency can foster bottom-up QI activities in a large, diverse health care system.