Evaluated Research Articles

Evaluasi Program Pembinaan Pusat Pendidikan dan Latihan Pelajar (PPLP) Provinsi Kepulauan Riau

The Student Education and Training Center (PPLP) in Riau Islands was formed based on the Ministry of Youth and Sports (KEMENPORA) program in the development of achievement sports coaching centers in the regions. Currently Riau Islands Province has 3 PPLP Sports Branches that are running, namely Pencak Silat, Taekwondo and Archery which are managed by the Riau Islands Provincial Youth and Sports Office. In its implementation there are several shortcomings such as sports facilities, facilities and infrastructure, and the achievements of athletes. The purpose of this study was to obtain a complete picture and evaluate the process of implementing the coaching program for PPLP athletes in Riau Islands Province. This research methodology uses a qualitative descriptive approach with primary and secondary data sources, and data collection methods in the form of documentation, interviews, and observations. The theory used by researchers in evaluation uses Masson & Gibbon's theory in Hayat (2018) which includes indicators: The process has run optimally, seen from the role of the management in implementing the coaching program that has been well coordinated. For formative indicators, it is less than optimal as seen from the fulfillment of athletes' facility needs which are not fully met. For summative indicators, it runs less optimally seen from the results of athletes' achievements that have not been consistent at the national level. The conclusion of this research is that the PPLP program has been running quite well in Riau Islands Province, it's just not optimal seen from additional professionals such as sports specialists, infrastructure and the results of athletes' achievements at the National level. Suggestions for improvement are that the administrators involved in the PPLP program must be able to increase their contribution, especially in terms of meeting the needs of athletes

Commentary: What young people want from psychotherapy.

In recent times there has been increasing acknowledgement of the importance of attending to the agenda of people with lived experience in psychotherapy research. In particular, young people's voices have been recognised as central to the design and development of psychotherapies that work for them. It is important to recognise the limits of professional agendas and make sure that young people's own priorities are represented in the indicators against which we measure change in research evaluations of psychotherapy. This requires an extension of evaluation research indicators from psychiatric symptomatology, to include aspects of wellbeing that matter to young people themselves. This article joins others in calling for a shift from the focus on symptom change in the evaluation of psychotherapy with youth, to acknowledge subjective indicators identified through research conducted with young people. New indicators might, for example, be centred on the degree to which young people experience increased capacity for acceptance of their emotions, a comfortable sense of identity, improved relational trust, and a stronger sense of their own agency. If psychotherapy is to be meaningful to young people, it is vital that we tailor it to young people's own needs and priorities and evaluate it against the aspects of change that matter to them.

Group decision making model for selection of performance indicators for sustainable supplier evaluation in agro-food supply chain

Sustainable food production is of vital importance to ensure food security, enhanced stability, environmental protection, and economic development for any developing nation. In an Agro- Food Supply Chain (AFSC), joint efforts of all supply chain actors inclined towards sustainability can contribute significantly towards sustainable production. Suppliers, who are intermediaries between Food Processing Firm (FPF) and sub-suppliers or farmers, must follow the FPF's sustainability practices at their own level, and they must monitor the enactment of these activities at their sub-supplier level. In this regard, the study focuses on developing a group consensus mathematical model to derive a broader set of performance indicators to evaluate a supplier's competency in maintaining, monitoring, and tracing sustainable performance in his sub-supplier network. A two-phased group consensus model is presented; it selects performance indicators using the Delphi method and it utilizes a feedback mechanism rooted in Group Analytical Hierarchy Process. This system is based on an iterative algorithm to generate weights that all decision makers accept. The proposed model is illustrated with an analysis of an Indian flour milling company. Kendall's concordance coefficient is employed to understand the degree of agreement among decision makers before and after the application of the proposed model. Increase in its value post-application validates the efficacy of the model. This approach for performance indicator selection for sustainable supplier evaluation research is unique in the field of AFSC. Results of the study indicate that the top ranking indicator is ‘Food Safety’ followed by ‘Payment to Sub-suppliers’. Key policy implications are drawn from the result findings which can be further integrated to support sustainable growth and development of suppliers and sub-suppliers in any AFSC.

The Ecological Improvement of Architectural Design by Renewable Energy Equipment

Renewable energy is an important part of green building, which will directly promote the development of a low-carbon economy and have a direct or indirect impact on the regional ecological improvement. At present, the ecological evaluation research of renewable energy green building mainly focuses on the theoretical aspect, explaining green materials, low-carbon recycling, social responsibility and other contents, which lacks practical discussion and analysis, but ignores the impact of green ecological improvement. In order to deeply study the impact of renewable energy equipment on the ecological improvement of architectural design, this paper extracts the content and indicators of green building analysis of renewable energy equipment by the factor regression analysis method of ecological footprint, standardizes the indicators, and eliminates the invalid indicators. Then, the green building of renewable energy equipment is analyzed according to the index to find out the main impact aspects. Finally, the problems are analyzed, the reasons are explored, and relevant countermeasures are proposed according to the regression results. The research results show that the renewable energy equipment has a significant impact on the ecological improvement of architectural design, and the ecological footprint model has certain practicability and reliability in the evaluation of green buildings of renewable energy.

University rankings in the context of research evaluation: A state-of-the-art review

Abstract Initially created as a marketing and benchmarking tool, global university rankings have evolved into a part of research evaluation and policy initiatives. Governments around the world, concerned about the low rankings of national universities, have launched numerous excellence initiatives in higher education. The aim of this study is to provide an overview of the literature on the use of university rankings in research evaluation and excellence initiatives. A systematic review of the literature on rankings in the context of research evaluation and excellence initiatives was performed. The majority of the review is based on English-language sources, but the review also covers literature from Russia, where the role of rankings is emphasized by their mention in the title and goal of the policy project 5top100. In addition, the review also includes gray literature. The general academic consensus is that rankings cannot be used in research assessment, though a few authors have a positive or neutral attitude. The literature reveals a lot of issues, such as technical and methodological flaws; biases of different nature, including bias towards research in evaluation of both universities and individual researchers; conflicts of interest; and risks to national identity. However, we must acknowledge the complexity of rejecting rankings.

DEVELOPMENT OF WORDWALL LEARNING EVALUATION ON CONSUMER BEHAVIOR MATERIAL

This study aims to assess the validity of the development of wordwall-based learning evaluation in business communication subjects with consumer behavior material for class X retail business at Vocational High School Tamansiswa 1 Jakarta. This study uses Research and Development (RnD) using the ADDIE development model. Thus, the analysis, design, development, and implementation stages are the first stages carried out in the research and development of this learning evaluation. The results of this study are the results of the media expert validation analysis which gives a percentage of 84% in the very valid group. In addition, 80% of the results in the valid category come from the material expert validation analysis. Furthermore, based on the validation results by the teacher, the figure was 80%, which is also in the valid category. The results of the small group trial by students showed a figure of 82% which falls into the very valid category. This percentage shows the validity and feasibility of using wordwall-based learning evaluation as a learning assessment in the classroom.

Reducing Breast Cancer Disparities with Precision Public Health: A New Strategy to Improve Prevention and Advance Health Equity in Delaware Hotspots.

While Black and White women are diagnosed with breast cancer (BC) at similar rates, Black women die from BC at a 40% higher rate. This disparity is even more pronounced for younger Black women, who die from BC at nearly twice the rate as younger White women. Black-White differences in BC mortality are largely attributable to health care and tumor biology factors. Black women face greater barriers to accessing BC screening and are twice as likely to be diagnosed with the aggressive triple-negative breast cancer (TNBC) subtype. Delaware leads the US for the incidence of late-stage BC diagnosed among younger women and TNBC. This commentary begins with a discussion of precision public health, an emerging framework that builds on and complements recent advances in precision medicine. Next, a new precision public health initiative designed to reduce BC disparities in Delaware by targeting local hotspots with prevention interventions is presented. Finally, next steps are considered for implementation, evaluation, and new research activities.

Development of a team-specific research strategy using a modified Delphi method in a regional public hospital dietetics department.

Evaluate research capacity and culture among regional hospital dietitians, develop a team specific research strategy, and build research skills of novice researchers. The Research Capacity in Context Tool was used to assess current research capacity and culture at organisational, team and individual levels. Results were analysed using descriptive statistics and content analysis of free text responses. A modified Delphi method gained consensus regarding research capacity building. Mixed method study. Dietetics department of a regional tertiary hospital (Modified Monash Category 2). All clinical dietitians currently employed within the hospital (n = 20) regardless of employment duration. Self-rated response to research capacity and culture to produce a dietetics-specific research strategy. Fifteen dietitians (75%) completed the Research Capacity in Context Tool. The overall mean score was highest at an organisational level at 7.9 (IQR 2), and lowest at team and individual levels at 4.3 (IQR 2.7) and 4.9 (IQR 3.3) respectively. Common barriers to research included time, lack of skills, knowledge and support. Using the modified Delphi method 39 statements relating to research capacity building met consensus and informed the creation of a research strategy. The results of the Research Capacity in Context Tool from this regional study reflect those reported in the literature at metropolitan sites. A dietetic-specific research strategy was developed to assist with increasing research capacity at a team and individual level in a regional setting. Evaluation of long-term outcomes post implementation will be the subject of further research.

Financial Well-being in relation to Job Motivation and Work Engagement among Nurses in Selected Private Tertiary Hospitals in Angeles City

Nurses are leaving their bedside jobs at an alarming rate because they are exhausted—mentally and physically. Aside from this, top-level nurses in private hospitals make as much as those entry level nurses in public hospitals. The objective of this study is to serve as advocacy research, influencing the official and informal policies and evaluate the relationship between financial well-being and private nurses’ quality of work life, particularly job motivation and work engagement. A descriptive evaluative, descriptive comparative and descriptive correlational research design was employed in this study. Purposive sampling was utilized and included a total sample of 115 private nurses with at least 1 year worth of experience from tertiary hospitals in Angeles City, data were collected using both paper-based questionnaire and Google Forms. Data collected were analyzed using descriptive mean, T-test and ANOVA, Pearson’s Product Moment Correlation Coefficient, Linear Regression, and Post HOC analysis. Results showed there were high positive relationships between financial well-being and work engagement (r= .620, p= .000) as well as financial well-being and job motivation (r= .711, p= .000). Result of the regression analysis elucidated that the job motivation explained the 52.1 percent variance [F(2, 112)= 60.837,p= .000]. The findings of the study indicate a positive correlation between nurses’ financial well- being and their level of work engagement and motivation in their profession. Additionally, it was also found that work engagement was a factor in the financial well-being of the participants, but it did not provide a substantial impact. The study’s results indicate that job motivation is a substantial predictor of financial well-being among nurses.

A STUDY TO ASSESS THE EFFECTIVENESS OF STRUCTURED TEACHING PROGRAM ON KNOWLEDGE REGARDING HEALTH HAZARDS OF ELECTRONIC DEVICES AMONG ANTENATAL MOTHERS AT SELECTED MATERNITY HOSPITALS, BANGALORE

Background of the Study:Electronic devices are components for controlling the flow of electrical currents for the purpose of information processing and system control. Prominent examples include transistors and diodes. Electronic devices are usually small and can be grouped together into packages called integrated circuits. Electronic devices are now an integral part of modern life. In many countries over half the population use electronic devices and the market is growing very rapidly. The electromagnetic field radiations impart an adverse effect on physical, mental and the emotional state of a pregnant mother and deeply affect the brain cells of mother and her unborn child. It affects level of melatonin, a sleep- controlling hormone which causes sleep disturbances in mothers, that pass to unborn babies through the placenta resulting in growth retardation. Objectives: 1. To assess the level of knowledge regarding health hazards of electronic devices during pregnancy among antenatal mothers in selected maternity Hospitals, Bangalore. 2. To evaluate the effectiveness of structured teaching program on knowledge regarding health hazards of electronic devices during pregnancy among antenatal mothers attending in selected maternity Hospitals, Bangalore. 3. To determine the association between knowledge score with selected demographic variables among antenatal mothers attending in selected maternity Hospitals, Bangalore. Methods:The research approach adopted was an evaluative research approach and research design adopted for this study was Pre experimental one group pretest and post-test research design. Simple random sampling technique was used to collect 60 antenatal mother sample. Data was collected over a period of one month. Structured knowledge questionnaire used as a tool to assess knowledge regarding the health hazards of electronic devices. Data obtained was analysed in terms of objectives which was calculated and tabulated. Results:In pretest, out of 60 antenatal mothers, 33.3% had inadequate knowledge and 66.7% had moderately adequate knowledge and none of the antenatal mothers had adequate knowledge. In post-test, out of 60 antenatal mothers, 73.3% had adequate knowledge, 26.7% had moderately adequate knowledge and none of the antenatal mother had inadequate knowledge. Over all knowledge the paired mean difference was 11.08 and SD of difference was 0.263 and enhancement mean percentage was 37 It was found significant (t=24.464, df=59) (i.e, p<0.05). The paired t-test was also worked out and it was found to be invariably significant (i.e p<0.05). The present study found the significant association between knowledge regarding health hazards of electronic devices at selected demographic variables of antenatal mothers. Interpretation and Conclusion: The study concluded that the post-test knowledge of antenatal mothers was improved after undergone Structured teaching programme regarding health hazards of electronic devices. The overall findings of the study clearly showed that the structured teaching programme was effective in improving the knowledge level of antenatal mothers regarding health hazards of electronic devices.

Surgical antisepsis in Russia: historical parallels

The origins of surgery and the improvement of surgical practices did not always guarantee a favorable surgical outcome. Even a masterfully performed operation could lead to a purulent-inflammatory process. Even at the stage of empirical development of medicine, before the great microbiological discoveries, an understanding was formed about a certain “substrate” that penetrates the surgical wound from the external environment and causes its suppuration. Supporters of the contagionist theory called it contagium, and supporters of the miasmatic theory considered the external (air) environment as such a source. The ideas of the supporters of the contagionist theory of the occurrence of purulent-septic complications in surgery in the middle of the 18th century are reflected in the works of D.S. Samoilovich and I.F. Semmelweis; the latter proved the transmission of the infectious principle by the surgeon’s hands, which became the starting point for the development of preventive antiseptics. In historical retrospect, the technology and means used for surgical hand antisepsis in Russia were analyzed from the mid-18th to the early 20th centuries. Moreover, literary sources for the period from 1783 to 2023 were analyzed, containing information about the antiseptic treatment of surgeons’ hands. Historical, descriptive, evaluative, analytical, and bacteriological research methods were used. As an experiment, the original antiseptic recipe described by D.S. Samoilovich (1783) was restored, and its effectiveness was confirmed. During surgical practices in Russia, starting from the mid-18th century, facts about the use of drugs with antiseptic activity for hand treatment are described. It is shown how, in historical retrospect, views on the final result of surgical antisepsis changed: from partial to complete destruction of microorganisms, preventing their entry from the deep layers of the skin, to the destruction of transient microflora and reduction of the abundance of resident microflora. Despite the influence of the English (J. Lister) and German (P. Fürbringer, J.F. Ahlfeld) surgical schools on the development of antiseptics in Russia, it retained its originality, determined by the needs of society and the possibility of using certain chemical antiseptics and their compositions for its implementation. The contribution of domestic surgeons to the improvement of antiseptic technologies and drugs in military field conditions is shown (N.I. Pirogov, A.M. Zabludovsky, D.I. Tatarinov et al.).

The Tpack model as a hybrid training oportunity in masters and postgraduates degrees

In the 2020-2021 academic year, during the pandemic, the teaching team of the Master Social and Educational Interventions of the University of Barcelona applied the Technological Pedagogical Content Knowledge (TPACK) model, which integrates technology in the teaching-learning processes (Mishra & Koehler, 2006). A participatory evaluative research (Redice Redice20-2861), with the participation of faculty and students involved, was carried out to identify the elements that facilitate and hinder virtual teaching, to know the impact of its application and to formulate recommendations applicable to online university higher education. A mixed quantitative and qualitative methodology was used, employing different instruments such as questionnaires, focus groups, interviews and observations during the 2020-2021 academic year, analyzed with NVivo 11, Atlas.ti 8.0 and Excel. The results obtained show that the application of digital environment activities within face-to-face training improve, among others, the connection with the professional world and offer dynamization strategies in the classroom that contribute to the creation of learning communities as an effective strategy.

Consensus Statement: Technical Standards for Thoracoabdominal Normothermic Regional Perfusion

BackgroundThoracoabdominal normothermic regional perfusion (TA-NRP) has emerged as a powerful technique for optimizing organ procurement from donation after circulatory death donors. Despite its rapid adoption, standardized guidelines for TA-NRP implementation are lacking, prompting the need for consensus recommendations to ensure safe and effective utilization of this technique. MethodsA working group composed of members from The American Society of Transplant Surgeons, The International Society of Heart and Lung Transplantation, The Society of Thoracic Surgeons, and The American Association for Thoracic Surgery was convened to develop technical guidelines for TA-NRP. The group systematically reviewed existing literature, consensus statements, and expert opinions to identify key areas requiring standardization, including predonation evaluation, intraoperative management, postdonation procedures, and future research directions. ResultsThe working group formulated recommendations encompassing donor evaluation and selection criteria, premortem testing and therapeutic interventions, communication protocols, and procedural guidelines for TA-NRP implementation. These recommendations aim to facilitate coordination among transplant teams, minimize variability in practice, and promote transparency and accountability throughout the TA-NRP process. ConclusionsThe consensus guidelines presented herein serve as a comprehensive framework for the successful and ethical implementation of TA-NRP programs in organ procurement from donation after circulatory death donors. By providing standardized recommendations and addressing areas of uncertainty, these guidelines aim to enhance the quality, safety, and efficiency of TA-NRP procedures, ultimately contributing to improved outcomes for transplant recipients.

A preliminary study to identify critical factors for evaluating the effect of car-lock sounds on drivers

Car-lock sounds are designed to inform the lock status of vehicles. However, drivers often experience a lack of confidence regarding whether the car is locked, and car thefts persistently occur, frequently attributed to unlocked doors. Without identification of critical factors for evaluating effects of car-lock sounds on drivers, a strategy to car-lock sound design with increased locking efficiency remains implicit. This study proposes a method to identify critical factors influencing drivers’ perceived certainty of car-lock status and behaviours during car-locking. An experiment was conducted to simulate the locking process and verbal protocol analysis was employed to comprehend participants’ cognitive processes and behaviours. The results show that mechanical sound yielded high certainty and few hesitations, while tonal and crisp sound elicited low certainty and frequent hesitations. Seven critical factors on participants’ behaviours and cognitive processes were identified, which provides a data-driven approach for future research in car-lock sounds evaluation and design.

Consensus Statement: Technical Standards for Thoracoabdominal Normothermic Regional Perfusion.

Thoracoabdominal normothermic regional perfusion (TA-NRP) has emerged as a powerful technique for optimizing organ procurement from donation after circulatory death donors. Despite its rapid adoption, standardized guidelines for TA-NRP implementation are lacking, prompting the need for consensus recommendations to ensure safe and effective utilization of this technique. A working group composed of members from The American Society of Transplant Surgeons, The International Society of Heart and Lung Transplantation, The Society of Thoracic Surgeons, and The American Association for Thoracic Surgery was convened to develop technical guidelines for TA-NRP. The group systematically reviewed existing literature, consensus statements, and expert opinions to identify key areas requiring standardization, including predonation evaluation, intraoperative management, postdonation procedures, and future research directions. The working group formulated recommendations encompassing donor evaluation and selection criteria, premortem testing and therapeutic interventions, communication protocols, and procedural guidelines for TA-NRP implementation. These recommendations aim to facilitate coordination among transplant teams, minimize variability in practice, and promote transparency and accountability throughout the TA-NRP process. The consensus guidelines presented herein serve as a comprehensive framework for the successful and ethical implementation of TA-NRP programs in organ procurement from donation after circulatory death donors. By providing standardized recommendations and addressing areas of uncertainty, these guidelines aim to enhance the quality, safety, and efficiency of TA-NRP procedures, ultimately contributing to improved outcomes for transplant recipients.

KEPALA SEKOLAH SEBAGAI PEMIMPIN PENDIDIKAN MENGENAI KESENJANGAN DAN PENYIMPANGAN DI SDN NO 50/VI LUBUK MENTILIN

Abstract This research aims to find out the principal as an educational leader regarding gaps and deviations at SDN 50/VI Lubuk Mantilin Kec. Anchor. This research includes evaluative research using the Descrepancy Model. Measuring the effectiveness of the program is carried out by comparing two things at the end of the program, namely the beginning and end of program implementation, namely comparing ideal conditions with real conditions regarding school principal standards. Variables are measured with instruments in the form of questionnaires. The results show that there are two main problems, namely gaps and deviations in the elementary school environment in Lubuk Mantilin Village.. The resuits of the first research are as follows, firstly, the gap includes several things a) leaderships quality gap, b) resource gap, c) treatment gap, d) implementations gap, e) communication gap. And as for the second result, the following are several devications that may accur, a) administrative deviatus, b) ethial deviations, c) devications in curriculum implementation, d) principal deviations in assesment. The results of the two studies show that there are action to overcome and reprimand the accurrenre of gaps and deviations at SDN 50/VI Lubuk Mentilin based on 8 national educatonal standards, namely content standards, process standards, graduate competency standards, standards for educators and education staff. Keywords : school principal, educational leader, disparities and iregularities

Effectiveness Evaluation of the Student Research Proposal Presentation Program at Scientific Writing Course

Higher education increasingly emphasizes the importance of developing students' academic and professional abilities, including scientific communication and presentation skills. The research proposal presentation program is a strategic initiative in responding to these needs, but challenges in its implementation still exist. Evaluate the effectiveness of student research proposal presentation programs in the Scientific Writing course at UNINDRA and identify strengths and weaknesses in the delivery of material and student argumentation structure. Evaluative research involves collecting data from the results of presentation evaluations, feedback from lecturers or other participants, and assessing the quality of presentations. Qualitative and quantitative data analysis techniques were used, with thematic analysis techniques to identify patterns or themes in participant feedback and evaluation. The evaluation showed that the research proposal presentation program provided significant benefits for lecturers and students. However, there are challenges in the learning process due to inadequate support in CPMK and material distribution. The high lecturer-student ratio also hinders the implementation of the ideal presentation. Internal obstacles such as lack of courage and suboptimal preparation of presentation materials also affect the quality of presentations. The need for adjustments in the preparation of the curriculum and teaching methods, including aligning the CPMK with the CPL and ensuring that the plan for the presentation of research proposals is included in the RPS. It is necessary to provide adequate time and improvement of presentation equipment to improve the quality of presentation and feedback. Better integration between CPMK and CPL will create a globally competitive generation of academics and professionals. Thus, this program can be more effective in achieving higher education goals in developing students' academic and professional abilities.

Automatic Identification of Sea Rice Grains in Complex Field Environment Based on Deep Learning

The number of grains per sea rice panicle is an important parameter directly related to rice yield, and it is also a very important agronomic trait in research related to sea rice breeding. However, the grain number per sea rice panicle still mainly relies on manual calculation, which has the disadvantages of being time-consuming, error-prone, and labor-intensive. In this study, a novel method was developed for the automatic calculation of the grain number per rice panicle based on a deep convolutional neural network. Firstly, some sea rice panicle images were collected in complex field environment and annotated to establish the sea rice panicle image data set. Then, a sea grain detection model was developed using the Faster R-CNN embedded with a feature pyramid network (FPN) for grain identification and location. Also, ROI Align was used to replace ROI pooling to solve the problem of relatively large deviations in the prediction frame when the model detected small grains. Finally, the mAP (mean Average Precision) and accuracy of the sea grain detection model were 90.1% and 94.9%, demonstrating that the proposed method had high accuracy in identifying and locating sea grains. The sea rice grain detection model can quickly and accurately predict the number of grains per panicle, providing an effective, convenient, and low-cost tool for yield evaluation, crop breeding, and genetic research. It also has great potential in assisting phenotypic research.

Analysis of Multiple Choice HOTS Test Questions on the Final Semester Assessment

Assessment is essential in the education system to monitor educational development and make appropriate instructional decisions. This study aims to analyze the quality of multiple choice questions based on high-level thinking skills (HOTS) in the Geography subject using Anates Windows Version 4.0.9 software. This evaluative research involves analysis of final semester exam questions for class X Geography at SMA Islam Al Azhar 14 Semarang. Data consisting of questions, answer keys and exam results were collected from 32 students. The analysis was carried out descriptively and quantitatively to assess the level of difficulty, discriminating power, reliability, effectiveness of distractors, and validity of the questions. The analysis results show that the difficulty level of most of the questions is in the medium category (50%). The discriminating power is mostly good, although several questions are wrong or bad. Most of the distractors worked well, but some questions had ineffective distractors. Regarding validity, 17 questions were valid, while 11 questions were invalid. The reliability of the test is in the high category, with a reliability coefficient of 0.71. Anates Windows Version 4.0.9 software effectively analysed the quality of HOTS questions. These findings indicate that although most of the questions are of good quality, some require revision to increase their validity and effectiveness. This research emphasizes the importance of using technology in educational assessment to ensure the quality and fairness of exam questions. It contributes to developing assessment tools to improve students' higher-order thinking skills.Keywords: Item Analysis; Multiple Choice; HOTS; SMA Islam Al-Azhar

Creating virtual patients using large language models: scalable, global, and low cost

Virtual patients (VPs) have long been used to teach and assess clinical reasoning. VPs can be programmed to simulate authentic patient-clinician interactions and to reflect a variety of contextual permutations. However, their use has historically been limited by the high cost and logistical challenges of large-scale implementation. We describe a novel globally-accessible approach to develop low-cost VPs at scale using artificial intelligence (AI) large language models (LLMs). We leveraged OpenAI Generative Pretrained Transformer (GPT) to create and implement two interactive VPs, and created permutations that differed in contextual features. We used systematic prompt engineering to refine a prompt instructing ChatGPT to emulate the patient for a given case scenario, and then provide feedback on clinician performance. We implemented the prompts using GPT-3.5-turbo and GPT-4.0, and created a simple text-only interface using the OpenAI API. GPT-4.0 was far superior. We also conducted limited testing using another LLM (Anthropic Claude), with promising results. We provide the final prompt, case scenarios, and Python code. LLM-VPs represent a ‘disruptive innovation’ – an innovation that is unmistakably inferior to existing products but substantially more accessible (due to low cost, global reach, or ease of implementation) and thereby able to reach a previously underserved market. LLM-VPs will lay the foundation for global democratization via low-cost-low-risk scalable development of educational and clinical simulations. These powerful tools could revolutionize the teaching, assessment, and research of management reasoning, shared decision-making, and AI evaluation (e.g. ‘software as a medical device’ evaluations).