Evaluation Process Research Articles

The effects of implicit ethnic expectations and burnout on teachers’ evaluations of students’ performance

ABSTRACT This study explores the effects of teachers’ implicit ethnic expectations and burnout on their evaluation of student performance. Ethnic minority students frequently encounter disadvantages in education, which may be partly attributed to these factors related to teachers. During the academic year of 2021/2022, 55 teachers from a central region of Italy took part in the study, including 20 preservice teachers. The Maslach Burnout Inventory was used to measure teacher burnout, and the Relational Responding Task was utilised to assess teacher implicit beliefs about students’ achievement. A linear mixed-effects model was used to examine the relationship between teachers’ burnout, implicit expectations, and their evaluation of students. Results showed that teachers’ burnout was negatively associated with teachers’ evaluations of ethnic majority and ethnic minority students, regardless of their performance level. Additionally, results suggest that teachers’ implicit expectations may negatively influence their evaluations of ethnic minority students, even when those are high achieving. These findings highlight the need to further investigate the role of teachers’ stereotypical implicit expectations and burnout in their evaluation process. Future interventions should explore strategies to address these biases and promote more equitable evaluation practices.

Influence of Strategy Implementation on Academic Performance in Public Secondary Schools in Kenya: Case Study of Bungoma County

Strategy management is crucial for organisational success, but the relationship between strategy management and performance remains unclear. In the context of public secondary schools in Kenya, the Ministry of Education has directed schools to develop and implement strategic plans. However, some schools face challenges in their strategy management. This study aimed to examine the impact of strategy implementation on academic performance in public secondary schools in Bungoma County, Kenya. The study population included 258 principals, 1,290 heads of departments, 258 bursars and quality assurance officers from 258 public secondary schools in the county. Data were collected through questionnaires, interviews, and document analysis. The results indicated a strong positive relationship between strategy implementation and academic performance in public secondary schools. It is concluded that schools that successfully implement strategies to improve education outcomes are more likely to achieve higher academic performance than schools that do not. Recommendations of the study are that streamlining strategy formulation, implementation, and evaluation processes in public secondary schools, enforcing policies to enhance education quality and access, and empowering schools to develop tailored strategic plans are highly encouraged. These findings can inform the Ministry of Education and other education stakeholders in improving policies and practices related to strategy implementation and academic performance in secondary schools

Snow Resource Reutilization: Design of Snow Collection and Compression Equipment Based on Functional Analysis Method

High-latitude regions of the Earth are rich in natural snow resources; however, owing to their negative impact on daily life, they have not been effectively utilized for a long time and are instead viewed as obstacles that require substantial resources for clearance. This waste of resources contradicts the principles of sustainable development. With the rapid development of the ice and snow industry, the social and economic value of snow resources is gradually becoming apparent. Therefore, to promote sustainable social and economic development, this study explores new methods for processing snow resources to achieve their recycling and high-value transformation. This study employs the functional analysis method to analyze and solve the functions of the design system, utilizing the Analytic Hierarchy Process (AHP) and fuzzy comprehensive evaluation for the objective assessment and selection of schemes. The obtained relative optimal principles are utilized to guide the parameter setting and innovative design of the scheme in terms of functional structure. After the final scheme is output, its feasibility is verified through finite element simulation. Ultimately, to address the issue of snow resource clearing and recycling, this study designs a product scheme capable of collecting and compressing snow on road surfaces, proposing the use of snow in the form of standardized compressed snow blocks for large-scale snow sculpture construction and other fields. This method significantly reduces the cost of snow sculpture production, enhances efficiency, realizes the comprehensive utilization and high-value transformation of snow resources, and provides a reference for the sustainable development of the low-altitude ice and snow tourism industry.

Ethylenediaminetetraacetic Acid-Dependent Pseudothrombocytopenia Discovered During Preoperative Evaluation: A Case Report

Pseudothrombocytopenia (PTCP) can be caused by various factors, and misdiagnosing it as true thrombocytopenia can cause significant risks to the patient. When using ethylenediaminetetraacetic acid (EDTA) as an anticoagulant in a complete blood cell (CBC) test, EDTA-dependent PTCP can occur in approximately 0.1% to 0.2% of cases. Anesthesiologists review various medical findings during the preoperative evaluation process, one of which is the blood test results. PTCP can lead to unnecessary interventions or treatments before surgery, potentially harming the patient. This study presents a case of PTCP discovered during preoperative evaluation of a patient diagnosed with pancreatic cancer. As anesthesiologists, ensuring the optimal condition of the patient before surgery is paramount, and this case highlights the importance of proper management and in-depth consideration when encountering patients suspected of having PTCP.

Effect of Fines Content on Cyclic Strength Development of Sand–Clay Mixtures under Cyclic Loading: A Process Evaluation Using Energy Methods

Effect of Fines Content on Cyclic Strength Development of Sand–Clay Mixtures under Cyclic Loading: A Process Evaluation Using Energy Methods

Clinical skills examination as part of the Swedish proficiency test of dentists educated outside of the EU/EEA.

The increase in the migration of dentists educated outside the EU/EEA calls for the sharing of information and evaluation of recognition processes within countries in the EU. In 2017, the Swedish National Board of Health and Welfare implemented the Proficiency test, a recognition process for dentists who have completed an education programme outside the EU/EEA. The Proficiency test consists of a theoretical and an integrated clinical skills examination, followed by a 6-month clinical practice. The clinical skills examination is a two-part examination that includes an OSCE and an operative test on a dental manikin. This paper presents data from proficiency tests between 2018 and 2022, and explores factors related to grade fail, that is, demographics, theoretical exam scores and language comprehension. In a cohort study, demographics and factors associated with grade fail were explored using test results from theoretical and clinical skills examinations (n = 181) from 2018 to 2022. Pearson correlation coefficient and linear regression analysis were used for studying correlations and associations between exam results. Univariable linear and logistic regression models were used for background variable associations with clinical skills exam outcomes. Higher age was a significant risk factor for failing the clinical skills examination and the OSCE. Higher scores in the theoretical exam reduced the odds of failing the OSCE but were not associated with results in the operative test or the overall results of the clinical skills examination. Regarding the OSCE there was a statistically significant difference within all professional qualifications explored between participants who passed and participants who failed the OSCE. Four years of data collection reveal that age and previous theoretical exam results influence the odds of failing the clinical examination. The study results also highlight the necessity of multiple assessment formats to assess clinical and communication skills of foreign-trained dentists.

Characterization and evaluation of the recovery process of saturated reverse osmosis membranes by chemical oxidation

Characterization and evaluation of the recovery process of saturated reverse osmosis membranes by chemical oxidation

Advanced Machine Learning and Experimental Studies of Polypropylene Based Polyesters Tribological Composite Systems for Sustainable Recycling Automation and Digitalization

Digitalization and automation are emerging solutions to the complex problems of recycling. In this research work, the experimental and Python based Archard deep learning wear rate models are introduced regarding recycling automation and composite tribological systems optimization. The optimum polyester fibers (PESF) of length of 3-3.5 mm were used for fabrication of polypropylene (PP)-PESF composite systems. The deformation, high texture, asperities, and micro-cracks were observed during scanning electron microscope and machine-learning studies. The lowest experimental value of abrasive wear of 3.0 × 10-6 mm3/Nm was observed for PP. Comparatively, higher experimental values of abrasive wear of the PP-PESF composites are found in the range of 4.35 × 10-6 to 4.7 × 10-6 mm3/Nm due to presence micro-defects on the surface of composites. The experimental values of Coefficient of friction (COF) of PP and PP-PESF are found in the range of 0.70 - 0.8 and 1.1 - 1.3, respectively. The experimental values of abrasive wear and COF are found compatible with literature. Similarly, the simulated values of abrasive wear of PP and PP-PESF composites are predicted in the range of 4.8×10-7 to 3.75×10-7 mm3/Nm, respectively. The predicted values of PP and PP-PESF composite show better resistance towards abrasive wear. The proposed experimental and simulated (in terms of Python coding, machine learning, image processing, artificial intelligence, and deep learning studies) research work can be introduced industrially for automation as well as digitalization of grinding of PES waste, processing, tribological testing, and SEM characterization evaluations.

Application of artificial intelligence in career development of women in STEM field: Counseling, network construction and recruitment

With the rapid development of science and technology, the application of artificial intelligence (AI) in STEM (science, technology, engineering and mathematics) is increasingly extensive, which provides a new idea for solving the challenges of women's career development in STEM. This paper deeply discusses the application of AI in counseling, network construction and recruitment of women's STEM career development, aiming at promoting gender equality and women's career growth. In career counseling, AI technology provides customized career advice and development path for women through deep learning and big data analysis. According to personal education background, skill level, hobbies and career goals, the system intelligently recommends suitable career directions, and updates industry trends and job market data in real time to help women make more informed career choices. The AI tutoring system also enhances women's professional self-confidence through functions such as simulated interviews and career planning exercises. In terms of network construction, AI helps women to expand their professional networks more effectively through intelligent recommendation system. Based on user behavior data and preferences, AI accurately recommends industry seminars, technical exchange activities and online communities to promote women to establish contacts with like-minded professionals. AI also assists women in managing and maintaining network relationships, ensures the continuity of network interaction through intelligent reminding functions, and identifies key nodes and influential people to enhance network quality. In the recruitment process, AI technology reduces gender bias and improves the fairness and transparency of recruitment through automatic resume screening, standardized interview process and competency-based evaluation model. AI system can analyze a large amount of data, accurately match the skills and job requirements of female candidates, and improve the employment rate of women in STEM field. At the same time, through transparent algorithm decision-making process and fairness check, the potential gender bias problem in AI algorithm is overcome. The research in this paper provides theoretical support and practical guidance for using AI technology to promote women's career development in STEM field, and helps to build a more equal and inclusive scientific and technological working environment.

Participatory developed school-based interventions targeting physical activity and sedentary behaviour among children and adolescents: A scoping review.

A significant number of children and adolescents are insufficiently physically active and have a sedentary lifestyle, which is associated with adverse health outcomes, necessitating effective interventions. Using a participatory approach, which engages youth in intervention development, may enhance intervention effectiveness, as such interventions are believed to be more tailored to the needs of the target group. This scoping review aimed to provide an overview of the results of process and effect evaluations of studies that developed school-based physical activity and/or sedentary behaviour interventions using a participatory approach among children and adolescents. Six databases (MEDLINE, Embase, Web of Science, CENTRAL (Cochrane Central Register of Controlled Trials), Scopus and SPORTDiscus) were searched. Studies were included when they evaluated the participatory process, intervention implementation or health-related intervention effects. A quality assessment was completed using two QualSyst checklists. Extracted data included characteristics of studies, interventions, and participatory processes; characteristics of participants from whom data were collected; outcomes measured and measurement methods used; and key findings. Twenty-five papers, covering 18 original studies, were included. Studies suggested that participatory processes were generally perceived positively, resulting in pupils feeling empowered, experiencing ownership and gaining certain skills. Participatory developed interventions also seemed to be generally well received, although the intervention dose delivered and intervention reach seemed to vary between studies. Finally, studies generally tended to report no effects on health. Although a participatory approach could be promising in empowering pupils and promoting intervention acceptance, apparent differences in intervention implementation and limited effectiveness highlight the need to enhance participatory processes and intervention effectiveness.

Concurrent Determination of Heat and Capacity Change of a Sessile Droplet Using a Single Measurement

Microcalorimetry, designed for the independent measurement of enthalpy and heat capacity, has been commercially available for a considerable time. However, heat-related states in samples, especially liquids, can introduce complicated phenomena and challenging measurement and data evaluation processes. Such complexity becomes apparent when observing fluctuations in thermal capacity (Cp) while measuring heat consumption (Q) during water evaporation. This paper presents a continuous heat pulse measurement (CHPM) method for concurrently analyzing Q and Cp in a single test using microcalorimetry. The sample droplet of 400 nL was directly dispensed on the microcalorimeter surface, followed by a light-emitting diode (LED) radiation generating heat to perform CHPM. We repetitively heated the microcalorimeter using heat pulses provided by LED irradiation, with their duration set to 100 ms and 10s repetition, while measuring the temperature response of the microcalorimeter sensor. A MATLAB-based simulation model was established to validate the accuracy of our Cp measurements, which show its value of 0.79% of minimum variance. Water evaporation coupled with simultaneous salt crystallization served as our study model, where the Cp values were calculated from real-time responses to heat pulses provided by LED. The experimental outcomes confirm the suitability of CHPM in extracting key thermal properties and emphasize its versatility as a diagnostic tool, providing a significant method for research and applications in the fields of physics, engineering, and beyond.

Assessing Experiential Learning Program Design and Delivery Using Institutional Mission and Professional Standards

BackgroundHigh-impact practices (HIPs), such as mentored research, Capstone projects, study abroad, and field experiences, have been shown to improve college student engagement through experience-based learning. Research indicates that these practices can have high impact when they are intentionally designed and delivered to align with institutional mission and professional standards. The administrative structure of a HIP initiative may not consider how to assess program effectiveness. Few close examples exist in the literature to guide program staff to understand whether the HIPs supported by their programs are being designed and delivered for high impact. Our study details the efforts of one unit at a large private university to evaluate the implementation of an experiential learning initiative intended to support faculty to design and deliver high-impact practices to students. PurposeThis qualitative case study responds to the call for assessment to ensure that HIPs qualify as high impact. It illustrates a standards-based approach to evaluating a program of co-curricular, experientially based high-impact practices by determining how educational design and delivery aligns with the institution’s mission and professional standards for best practice. Methodology/approachTo qualitatively evaluate how well our HIP program aligned with institutional mission and HIP standards, we collected narrative data drawn from six faculty projects, faculty interviews, and reflections from 70 participating students. We used the institutional mission and established HIP professional standards as confirmatory themes to code the narrative and observe levels of coverage. Administrative directives were mapped against the confirmatory themes to identify areas of high and low emphasis and emergent themes that fell outside specific mission and standards language were considered for meaning. Findings/conclusionsOur qualitative assessment of a HIP program revealed program strengths and areas in need of improvement. By evaluating the concentration of institutional mission and professional standard themes across the project narratives and within the way the program was structured and administered, we found strong alignment existed in the areas of intellectual growth and interaction with faculty and peers. When we found that other mission themes and standards were present at lower levels, we mapped the administrative directives to the themes and discovered that the high concentration themes occurred more frequently in the directives. This insight led us to observe that addressing all mission and professional standard themes more clearly in the design phase could serve to align administrative directives more closely. In addition to the presence of confirmative themes across the data, we found emergent themes that had meaning to study participants but fell outside of the mission and industry standards. The evaluative process taught us important lessons around how administration-to-faculty program directives can impact the design and delivery of pedagogical practice. RecommendationsEducational leaders should consistently use institutional mission and professional standard language in directives to faculty and staff; training faculty in designing HIPs to meet professional standards can benefit underserved students; and recognizing meaningful themes that emerge through program evaluation can illuminate important unanticipated outcomes.

Effects of Competency-based Education on Resident Training: A Comparative Review of Korean Neurologist’s Entrustable Professional Activities 13 on Self-evaluation and Professor's Evaluation

Since the implementation of the Resident Training Special Act in 2018, the Training Committee of the Korean Neurological Association has made significant efforts to establish a competency-based resident training program. Through various research activities, the committee has verified the effectiveness of the competency-based training program and conducted additional studies related to the evaluation of the resident training program. In this study, we aimed to analyze the evaluation process of the Korean neurologist’s entrustable professional activities (K-NEPA 13), which outlines 13 core competencies required for neurology residents, and to explore potential improvements in training guidelines and evaluation methods related to these competencies. New evaluation guidelines for the K-NEPA 13 were developed, and a study was conducted with supervising professors and residents from seven training hospitals. The residents were asked to conduct self-assessments, while the supervising professors also evaluated each resident. The evaluation results were compared by categorizing the residents into junior and senior groups to analyze the differences between the two assessments. The results of the evaluations could be classified into several distinctive groups, and each group, based on the specific content of the questions, helped identify the current status and challenges of the training program. Based on these findings, we believe that this research serves as a sufficient foundation for more in-depth investigations and studies in the future, and continuous efforts should be made to improve the competency-based resident training program.

The Utilization of Diagnostic Musculoskeletal Ultrasound in the Evaluation for Ischiofemoral Impingement: A Perspective for Rehabilitation Providers

Ischiofemoral impingement (IFI) is a significant but often underdiagnosed source of deep posterior hip pain and functional impairment, frequently associated with narrowing of the ischiofemoral space (IFS) between the ischial tuberosity and the lesser trochanter of the femur. Traditionally, diagnostic approaches for IFI have relied on clinical examination and magnetic resonance imaging (MRI). However, the advent of diagnostic musculoskeletal ultrasound (MSKUS) has transformed the evaluation process. MSKUS offers a non-invasive, cost-effective, and dynamic method for assessing the IFS and associated musculature in real time, allowing for high-resolution visualization of soft tissues, bones, and neurovascular structures. For rehabilitation providers, understanding the application, strengths, and limitations of MSKUS can enhance diagnostic accuracy, guide targeted therapeutic interventions, and potentially expedite recovery. This article reviews the anatomy of the IFS, the pathophysiology of IFI, and the use of MSKUS in diagnosing IFI, equipping rehabilitation professionals with the knowledge to integrate this valuable diagnostic tool into clinical practice.

Community mental health for migrant women in Barcelona (“Self-Care Among Women”): Protocol for a mixed-methods process evaluation of a pilot psychological intervention

IntroductionMigrant populations, especially women, are often more susceptible to poor mental health and they face healthcare access barriers. This study aims to develop and evaluate a community-based mental health intervention through a participative research process to promote mental health and prevent psychiatric disorders among migrant women in Barcelona city. The intervention is built upon two evidence-based guidelines: Group Problem Management Plus (Group PM+) and Self-Help Plus (SH+). Methods“Self-Care Among Women” is a mixed-methods process evaluation study for the development of a community-based psychological pilot intervention. A participatory approach will be employed to adapt the content agenda of the community mental health intervention. Process indicators on context, implementation and mechanisms of change will be assessed through an intersectionality lens to explore the way in which the intervention is developed and implemented. The contextual fit, acceptability and feasibility of the proposed psychological intervention will be examined through participants' and facilitators' perceptions of the intervention. Further, women's clinical mental health condition and wellbeing will be assessed before and after the intervention, through standardized and culturally-validated instruments. Ethics and disseminationEthical approval of the research protocol has been obtained by the Research Ethics Committees of Hospital Clínic, Fundació Institut Universitari per a la recerca a l’Atenció Primària de Salut Jordi Gol i Gurina (IDIAPJGol) and Parc Sanitari Sant Joan de Déu. Written informed consent will be requested from all research participants for inclusion in the study. The study results will be published in peer-reviewed journals and communicated to congresses.

Tele-collaborative outreach to rural patients with chronic pain: pragmatic effectiveness trial protocol for the CORPs study.

Despite the increased availability of evidence-based treatments for chronic pain, many patients in rural areas experience poor access to services. Patients receiving care through the VA may also need to navigate multiple systems of care. To examine the effectiveness of a remotely delivered collaborative care intervention for improving pain interference among veterans with high-impact chronic pain living in rural areas. We will conduct a four-site pragmatic effectiveness trial of remotely delivered collaborative care for high-impact chronic pain. Participants (n = 608) will be randomized to the Tele-Collaborative Outreach to Rural Patients (CORPs) intervention or to minimally enhanced usual care (MEUC). Participants randomized to CORPs will complete a biopsychosocial assessment and five follow-up sessions with a nurse care manager (NCM), who will collaborate with a consulting clinician to provide personalized recommendations and care management. CORP participants will also be invited to a virtual 6-session pain education group class. Participants randomized to MEUC will receive a one-time education session with the NCM to review available pain services. All participants will complete quarterly research assessments for one year. The primary study outcome is pain interference. This trial will oversample veterans of female birth sex and minoritized race or ethnicity to test heterogeneity of treatment effects across these patient characteristics. We will conduct an implementation process evaluation and incremental cost-effectiveness analysis. This pragmatic trial will test the real-world effectiveness of a remotely delivered collaborative care intervention for chronic pain. Study findings will inform future implementation efforts to support potential uptake of the intervention.

A self-efficacy enhancement alcohol reduction intervention for men on-remand in prison: the APPRAISE feasibility pilot RCT.

As many as 70% of remand prisoners have admitted to being under the influence of alcohol when committing the crime leading to their imprisonment. Providing support and advice regarding alcohol consumption can be effective in some groups of people. There is little evidence regarding this for men on remand in prison. To pilot the study measures and evaluation methods to assess the feasibility of conducting a future definitive multicentre, pragmatic, parallel group, randomised controlled trial. A two-arm, parallel group, individually randomised pilot study of a self-efficacy-enhancing psychosocial alcohol intervention to reduce levels of alcohol consumption for males on remand in prison and on liberation. Two purposively selected prisons in Scotland and England. Adult men on remand in prison with an Alcohol Use Disorders Identification Test score of ≥8. The APPRAISE intervention delivery comprised four steps: Step 1: 1×40-minute face-to-face session, delivered by a trained practitioner from Change Grow Live in prison. Steps 2, 3 and 4: 20-minute sessions conducted by phone, on or as close as possible to days 3, 7 and 21 post liberation. Control: assessment, screening and referral onto further alcohol support options. Recruitment and retention rates, completion of follow-ups, outcome measures at 12 months and interventions delivered. The primary outcome for the pilot study was alcohol consumed in the 28 days prior to Time Point 2, assessed using the extended Alcohol Use Disorders Identification Test-C. Of 182 men on remand approached across two study sites, 132 were randomised (90 in England; 42 in Scotland) with 46 randomised to intervention and 44 to care as usual in England and 22 randomised to intervention and 20 to care as usual in Scotland. A total of 53 in-prison interventions were delivered. One day-3 post-liberation intervention was delivered, no day-7 and one day-21. At 12 months, of 132 randomised, 18 (13%) were followed up, 53 (40%) were not liberated; 47 (36%) were uncontactable and 14 (11%) had been released but could not be located. Data completeness was 96% at baseline and 8% at 12 months. The process evaluation reported good acceptability of the intervention with investment in time, capacity and space to support implementation identified. The economic study produced guidance on how to assess costs associated with implementing the APPRAISE intervention which could be applied more broadly. No adverse events or side effects were noted. A future definitive trial would be possible, but only if follow-up mechanisms can be addressed as well as full access to recidivism and health data. Collaboration with the probation service in future could offer the opportunity to develop a robust process and system to optimise follow-up post liberation. Dedicated resources to support the intervention delivery both in and out of the prison setting are recommended. Coronavirus disease discovered in 2019 impacted recruitment and follow-up, with access to prisons restricted. We were unable to deliver the post-liberation element of the intervention. We did not include probation services or other agencies in the trial. This trial is registered as Current Controlled Trials ISRCTN36066. This award was funded by the National Institute for Health and Care Research (NIHR) Public Health Research programme (NIHR award ref: 17/44/11) and is published in full in Public Health Research; Vol. 12, No. 11. See the NIHR Funding and Awards website for further award information.

Welcome to the October 2024 Issue (Vol: 30, No: 5) and Current News of the European Journal of Therapeutics

Dear Colleagues, We are very pleased to present you with another issue of the Eur J Ther (2024; Vol: 30, Issue: 5). This issue contains very valuable articles, and we believe you will read them with great interest. Another happiness is that the Eur J Ther is progressing day by day. In this respect, we would like to share with you that Index Copernicus’ application of the Eur J Ther has also been approved (ICV 2023: 100.00) this year [1]. Finally, we would like to sincerely thank our esteemed authors who contributed to the preparation of this issue, our referees who meticulously carried out the evaluation process, our team that provides the technical infrastructure of our journal, and our esteemed readers who support us in every issue. We hope to share more beautiful developments with our regards.

Optimising the Economic Feasibility of High-Speed Maglev Systems: A Simulation-Based Approach for Variable and Parameter Analysis

This study introduces an advanced software platform and process for the quantitative national economic evaluation of high-speed maglev systems, overcoming limitations of traditional methods through parameter variation experiments and automated solution search. Utilising the adapted German standardised evaluation, this research demonstrates how integrated modelling, evaluation and optimisation software can deeply analyse the impact of various variables and parameters on economic outcomes. By employing an optimisation algorithm, the software not only determines critical evaluation parameters to ensure benefits exceed costs but also deduces optimised model variables. The macroeconomic benefit-cost ratio guides the optimal design concept, with the research finding a critical value for ensuring economic feasibility. The proposed solution achieves a 22% improvement in this ratio (1.106 vs. 0.909) compared to the existing Hefei-Wuhu route, highlighting its potential for large-scale maglev implementation. Future development directions include integration with micro-simulation systems, support for random behaviour, sensitivity analysis, data-driven machine learning and enhanced user interface design for broader applicability. The findings underscore the software’s capability to provide robust, data-driven insights for economic feasibility studies of high-speed maglev systems, presenting a significant step forward in infrastructure project evaluation.

Evaluation and Use of Natural Language Processing (NLP) Reasoning and Classification Models to Support Clinical Trial Patient Identification and Enrollment in the Community Oncology Setting

Clinical trial research in oncology relies heavily on clinical documentation within the electronic medical record (EMR) to ascertain patient eligibility in clinical trials based on inclusion and exclusion criteria. The structured data elements within the EMR serve as the primary information source for defining patient cohorts, with clinical cancer stage and performance status being two pivotal criteria determining trial eligibility. The challenge arises from the inconsistent availability of clinical stage and performance status data within the structured fields of the EMR despite their consistent presence in clinical notes. Additionally, there is a deficiency of standardization of this data that exists in the unstructured field. Hence, due to lack of structured data and standardization of said data, there are limitations in developing artificial intelligence (AI) models. To increase the comprehensiveness of clinical records, a clinical research team at a community oncology practice was consulted to identify requirements and extract essential clinical features from de-identified data. The methods outlined in this paper focused on eliminating false positives to allow future development of Large Language Models (LLM) using the outputted structured fields which resulted in an increase in patient record completeness with high accuracy. The accuracy ranged from 97.5-97.75% for the models that were developed. Out of the 60,000+ patients, the numerical staging, TNM (tumor, node, metastasis) staging, and Karnofsky performance score models added a structured field for 29.62%, 21.01%, and 40.64% patients respectively. Additionally, a semi-supervised NLP algorithm was applied on the performance status algorithm which achieved a mean absolute error (MAE) of 1.57. This work demonstrates the use case of natural language processing (NLP) in optimizing the clinical research enrollment process by providing an efficient and accurate method to detect key clinical values in unstructured patient data. Similar methodology with more advanced algorithms such as LLM can be employed to detect additional patient elements such as molecular biomarkers, imaging reports, postoperative surgical outcomes (i.e., clear margins etc.) and patient treatment outcomes using the extracted structured fields.