Development Of Learning Tools Research Articles

Penerapan keterampilan abad 21 di Kelas V sekolah dasar

<p><em>The 2013 curriculum was designed and developed to facilitate skilled teachers and students in increasing learning interactions, especially in the 21st century. 21st century skills are a challenge for teachers and students to be able to adapt to the current era. 21st century skills need to be integrated and implemented in learning with the aim of creating generations according to current needs and demands. This study aims to implement 21st century skills by developing learning tools with TPACK, HOTS, and STEAM or STEM content, with problem- and project-based learning models (PBL and PJBL). The method used is descriptive analysis through the stages of observation, interviews to identify problems, conducting group discussion forums, making instrument angles, observation guidelines, and interview guidelines. Data collection was carried out on V grade elementary school students, totaling 30 people by using an interview guide questionnaire and a checklist for observation. The research findings are evidenced by three stages, namely learning planning through learning tools that include 21st century or 4C skills, implementing the learning process and evaluating the implementation of learning. The data shows that students are able to participate in activities oriented to the 4C skills, Critical Thinking and Problem Solving, Creativity and Innovative, Collaborative, and Communicative. Through 21st century or 4C skills, students are more active and able to collaborate with other students, and have self-confidence.</em></p>

Pengembangan Media Poster Berbasis Pictorial Riddle Model 4D Sebagai Bahan Ajar Mata Kuliah Pendidikan Agama Program Studi Pendidikan Agama Hindu

The purpose of this study was to determine the development of 4-D pictorial riddle-based poster media as teaching materials for religious education courses in the Hindu Religious Education Study Program; find out whether poster media is feasible or not to be used based on the assessment of material experts, learning practitioners and media experts; knowing the interest in learning and student learning outcomes after using poster media. This type of research and development refers to the 4D model of the development process. At the development stage (Develop) learning tools, all the instruments used and the poster media developed were validated by learning experts and practitioners. Expert validators and practitioners are Lecturers of STAHN Mpu Kuturan Singaraja. The develop stage consists of a limited field test and an extensive test. The results showed that Pictorial Riddle-Based Poster Media was developed in four stages, namely Define, Design, Develop, and Disseminate; Pictorial Riddle-based poster media is said to be feasible in terms of SBI, PA (Percentage of Agreement) scores, and based on the results of student response questionnaires. The poster media has an SBI score of 3.92 in the good category, the PA score is 92.2%, and the result of the student response questionnaire is 3.86 in the good category and is included in the appropriate category for use. Based on the normalized gain value, the increase in student interest in the second semester of the Hindu Religious Education Study Program after applying the poster media is 0.13 with a low significant level. Based on the normalized gain value, the increase in student learning outcomes in the second semester of the Hindu Religious Education Study Program after applying the poster media was 0.42 with a moderate significance level. The results showed that the poster media based on pictorial riddles that was developed was very well used as teaching material in the Hindu Religious Education Study Program.

Unsupervised machine learning, QSAR modelling and web tool development for streamlining the lead identification process of antimalarial flavonoids

ABSTRACT Identification of lead compounds with the traditional laboratory approach is expensive and time-consuming. Nowadays, in silico techniques have emerged as a promising approach for lead identification. In this study, we aim to develop robust and predictive 2D-QSAR models to identify lead flavonoids by predicting the IC50 against Plasmodium falciparum. We applied machine learning algorithms (Principal component analysis followed by K-means clustering) and Pearson correlation analysis to select 9 molecular descriptors (MDs) for model building. We selected and validated the three best QSAR models after execution of multiple linear regression (MLR) 100 times with different combinations of MDs. The developed models have fulfilled the five principles for QSAR models as specified by the Organization for Economic Co-operation and Development. The outcome of the study is a reliable and sustainable in silico method of IC50 (Mean ± SD) prediction that will positively impact the antimalarial drug development process by reducing the money and time required to identify potential antimalarial lead compounds from the class of flavonoids. We also developed a web tool (JazQSAR, https://etflin.com/news/4) to offer an easily accessible platform for the developed QSAR models.

QR Code-Practicum Guide Based on Guided Inquiry as The Novel Biology Lab Activities Learning Assisted

Educators play an important role in developing learning tools that are interesting and by the needs of students. This aims to develop a QR Code-Practical Guide based on Guided Inquiry for Class XII MIPA at the High School Level on growth and development in plants and photosynthesis and to determine the validity, practicality, and effectiveness. This type of research is Research and Development with the ADDIE model which consists of 5 stages, namely Analyze, design, development, Implementation, and evaluation. The research location is at SMAN 2 Majene, East Banggae District, Majene Regency, West Sulawesi Province. The research subjects were class XI MIPA 5 which consisted of 31 students. The results of the validation of the QR Code Practical Guide based on Guided Inquiry using the assessment sheet of media experts and material experts who were assessed by 2 expert validators obtained a result of 3.8 which is included in the valid category. The results of the practicality of the QR Code Practical Guide based on Guided Inquiry using teacher response questionnaires and student response questionnaires. The results of the questionnaire analysis show that the value of the teacher's response is 92.6%, getting the very practical category and the value of the student's response is 86.6% getting the very practical category. The results of the QR Code Practical Guide based on Guided Inquiry using test questions, the learning outcomes obtained are 80.7% in the effective category. Thus the QR Code Practical Guide based on Guided Inquiry is declared valid, practical, and effective to be used as teaching materials.

Development of Hypercontent-Based Open-Ended Approach-Based Learning Devices to Improve SMK Students' Creative Thinking Abilities

This study aims to: 1) obtain learning tools developed using an open-ended approach using hypercontent that meet validity, practicability, and effectiveness to improve students' creative thinking skills; and 2) describe how to apply learning tools developed using an open-ended approach using hypercontent to improve students' creative thinking skills. This study is concerned with development. The ADDIE development paradigm was used in this study. 1) The development of learning tools based on an open-ended approach using hypercontent meets the criteria of validity, practicability, and effectiveness of learning tools; 2) In trial I and trial II, the ability to improve creative thinking skills using learning tools based on an open-ended approach using hypercontent increased from 0.22 with low criteria to 0.39 with moderate criteria.

Pengembangan Model Latihan Sistem Menyerang Dengan Pola 3-5-2 Pada Ekstrakurikuler Sepak Bola SMPN 19 Malang

The application of formations in football is an important element in determining the outcome of a team's match. If the team has players who are proficient, but apply the wrong formation and even the formation does not match the existing conditions, it will have an impact on the results of the match. The method in this research is research & development. The method used is research & development of 4D models. The 4-D (Four-D) development model is the development of learning tools developed by S. Thiagarajan. From the small group test data it can be concluded that the ease aspect obtained a percentage of 87.5%, while the attractiveness aspect obtained a percentage of 83.59%. From the large group test data it can be concluded that the ease aspect obtained a percentage of 86.35%, while the attractiveness aspect obtained a percentage of 82.1%.

Pengembangan Model Pembelajaran Berbasis Pendidikan Matematika Realistik Berbantuan Media Animasi untuk Meningkatkan Kemampuan Komunikasi Matematis

This study aims to produce: the validity, practicality and effectiveness of learning tools based on the Realistic Mathematics Education (PMR) model, improving students' mathematical communication abilities using the developed learning tools and the process of student answers in solving mathematical communication skills questions. This research is development research with the ADDIE development model. From the results of the research, it is known that: (1) The learning model of Realistic Mathematics Education (PMR) assisted by animation media which was developed obtained an interactive learning model declared valid with an average of 4.31; (2) The Realistic Mathematics Education (PMR) learning model assisted by animation media that was developed meets the practical criteria of the learning model in terms of the analysis of the results of observations of the implementation of learning. The value obtained in trial I was 3.03 (medium category). In trial II, the score for observing the implementation of learning increased to 3.88 (category "high"); (3) The learning model of Realistic Mathematics Education (PMR) assisted by animation media developed meets the criteria of effectiveness. (4) Based on the normalized gain index it was found that in trial I there was an increase in students' mathematical communication skills with the "low" criteria with a score of 0.30 (g ≤0.3) and in trial II there was an increase in scores with the "moderate" criteria with a score of 0.42 (0.3 < N-Gain ≤0.7).

Development of RBL-STEM Learning Tools to Improve Students’ Computational Thinking Skills Solving Rainbow Antimagic Coloring Problems and Their Application to Traffic Flow Problems with Spatial Temporal Graph Neural Network

Computational thinking is thinking process that is needed in formulating problems and solutions, so that these solutions can be effective information processing agents in solving problems. Indicators of computational thinking consist of problem decomposition, algorithmic thinking, pattern recognition, abstraction and generalization. To improve higher-order thinking skills, we apply RBL learning integrated with STEM approach. To improve students’ thinking skills, it is necessary to develop tools that support the success of learning activities. The learning tools that have been developed meet the criteria of valid, practical, and effective. The validity scores obtained by each device are 3.5 for the face-to-face plan, 3.41 for the student worksheet, and 3.56 for the learning outcomes test. The observation result of the learning implementation score was 3.8 with a percentage of 95%. There were 23 students who completed or around 88,46%, percentage of average score of student activities was 94.17%, and as many as 94.47% of students gave a positive response.

Animated teaching improves student learning of human gastrulation and neurulation

Human embryology is a complex topic that brings together core components of anatomy and physiology to describe the developmental process from fertilisation to birth. Embryonic development is a challenging topic of study that is core to the curricula for health science students. There are challenges ingrained in teaching and learning embryology, due to the three-dimensional dynamic processes that occur as the embryo develops. This study aimed to develop and assess two newly developed animations depicting key processes in embryology, namely gastrulation and neurulation, as supplemental learning aids for students. Indeed, animated teaching tools to enhance the learning of gastrulation and neurulation are not widely available. A multi-disciplinary team of physiologists, biochemists, anatomists, and a computer scientist developed the animation sets. A student cohort of 81 first-year health science students were enrolled in this study over a period of three academic years. Both animations are in line with the course content of the first-year health science students undertaking the Human Health and Disease BSc at Trinity College Dublin, who were the study participants. Participants were randomly assigned to a non-animation control group and an animation group. Each set of animated teaching aids was broken down into individual clips which were given identifiable headings to allow the user to interchange between clips to facilitate a more personal learning experience. The animation group had open access to the animations for a three-week period. Questionnaires were designed to assess participants’ attitude to the animations and their knowledge of embryology, both at the start of the study and three weeks later following access to the animations. Data presented herein indicate that students incorporated the animated teaching aids into digital home study and that the use of the animations acted as a supplemental tool that increased student knowledge in key areas of human embryology. From a qualitative point of view, students described the animations as enjoyable and helpful in visualising complex processes. This study indicates that the development of gastrulation and neurulation animated learning tools allow for a more engaging learning experience, facilitating student’s engagement with academically challenging concepts in human embryology.

Agreement Between Experts and an Untrained Crowd for Identifying Dermoscopic Features Using a Gamified App: Reader Feasibility Study.

Dermoscopy is commonly used for the evaluation of pigmented lesions, but agreement between experts for identification of dermoscopic structures is known to be relatively poor. Expert labeling of medical data is a bottleneck in the development of machine learning (ML) tools, and crowdsourcing has been demonstrated as a cost- and time-efficient method for the annotation of medical images. The aim of this study is to demonstrate that crowdsourcing can be used to label basic dermoscopic structures from images of pigmented lesions with similar reliability to a group of experts. First, we obtained labels of 248 images of melanocytic lesions with 31 dermoscopic "subfeatures" labeled by 20 dermoscopy experts. These were then collapsed into 6 dermoscopic "superfeatures" based on structural similarity, due to low interrater reliability (IRR): dots, globules, lines, network structures, regression structures, and vessels. These images were then used as the gold standard for the crowd study. The commercial platform DiagnosUs was used to obtain annotations from a nonexpert crowd for the presence or absence of the 6 superfeatures in each of the 248 images. We replicated this methodology with a group of 7 dermatologists to allow direct comparison with the nonexpert crowd. The Cohen κ value was used to measure agreement across raters. In total, we obtained 139,731 ratings of the 6 dermoscopic superfeatures from the crowd. There was relatively lower agreement for the identification of dots and globules (the median κ values were 0.526 and 0.395, respectively), whereas network structures and vessels showed the highest agreement (the median κ values were 0.581 and 0.798, respectively). This pattern was also seen among the expert raters, who had median κ values of 0.483 and 0.517 for dots and globules, respectively, and 0.758 and 0.790 for network structures and vessels. The median κ values between nonexperts and thresholded average-expert readers were 0.709 for dots, 0.719 for globules, 0.714 for lines, 0.838 for network structures, 0.818 for regression structures, and 0.728 for vessels. This study confirmed that IRR for different dermoscopic features varied among a group of experts; a similar pattern was observed in a nonexpert crowd. There was good or excellent agreement for each of the 6 superfeatures between the crowd and the experts, highlighting the similar reliability of the crowd for labeling dermoscopic images. This confirms the feasibility and dependability of using crowdsourcing as a scalable solution to annotate large sets of dermoscopic images, with several potential clinical and educational applications, including the development of novel, explainable ML tools.

Design, Implementation, and Evaluation of Online Bioinformatics and Neuroinformatics Labs

In recent years, online laboratories have become highly integrated into the educational process due to the development of distance learning tools as well as the pandemic. As part of a master's degree program in bioinformatics and neuroinformatics, in the academic years 2020-2021 and 2021-2022, the mandatory module "Laboratory Education (LE)" included 9 labs and a special session took place online. For the evaluation of the LE a survey was carried out among the participants on issues of satisfaction with each lab, the knowledge they acquired, the educational material given to them, with the facilitator per lab as well as their self-reported learning. 73 postgraduate students participated in the survey. According to the results, the overall satisfaction from each laboratory ranged from 3.94 to 4.49/5.00. Furthermore, there is a variety of values in self-reported learning ranging from 23 to 50/50. Finally, although 7 out of 10 students say they are satisfied with the distance structure of LE, 8 out of 10 say they prefer LE to be carried out with a physical presence in the labs.

Development of eConsult reflective learning tools for healthcare providers: a pragmatic mixed methods approach

BackgroundElectronic consultation (eConsult) programs are crucial components of modern healthcare that facilitate communication between primary care providers (PCPs) and specialists. eConsults between PCPs and specialists. They also provide a unique opportunity to use real-world patient scenarios for reflective learning as part of professional development. However, tools that guide and document learning from eConsults are limited. The purpose of this study was to develop and pilot two eConsult reflective learning tools (RLTs), one for PCPs and one for specialists, for those participating in eConsults.MethodsWe performed a four-phase pragmatic mixed methods study recruiting PCPs and specialists from two public health systems located in two countries: eConsult BASE in Canada and San Francisco Health Network eConsult in the United States. In phase 1, subject matter experts developed preliminary RLTs for PCPs and specialists. During phase 2, a Delphi survey among 20 PCPs and 16 specialists led to consensus on items for each RLT. In phase 3, we conducted cognitive interviews with three PCPs and five specialists as they applied the RLTs on previously completed consults. In phase 4, we piloted the RLTs with eConsult users.ResultsThe RLTs were perceived to elicit critical reflection among participants regarding their knowledge and practice habits and could be used for quality improvement and continuing professional development.ConclusionPCPs and specialists alike perceived that eConsult systems provided opportunities for self-directed learning wherein they were motivated to investigate topics further through the course of eConsult exchanges. We recommend the RLTs be subject to further evaluation through implementation studies at other sites.

New Perspectives on Reverse Translation: Brief History and Updates

Since the 1950s, reverse translation has been an enigmatic part of Crick’s central dogma of molecular biology. It might be described as the possibility to back-translate information from proteins to nucleic acids (or codons). A few studies have attempted to theorize and/or conduct in vitro experiments to test the likelihood of reverse translation, with ideas often involving the creation of peptide recognition sites that bridge the peptide and the codon. However, due to many constraints including an asymmetrical informational transfer, the stability of protein-peptide bonds, the structural non-uniformity of protein R-groups, and the informational loss in post-translational protein modifications, this concept requires follow-up studies. On the other hand, current bioinformatic tools that rely on computational programs and biological databases represent a growing branch of biology. Bioinformatics-based reverse translationcan utilize codon usage tables to predict codons from their peptide counterparts. In addition, the development of machine learning tools may allow for the exploration of biological reverse translation in vitro. Thus, while in vivo reverse translation appears to be nearly impossible (due to biological complexity), related biological and bioinformatics studies might be useful to understand better thecentral dogma’s informational transfer and to develop more complex biological machinery.

A review on recent machine learning applications for imaging mass spectrometry studies

Imaging mass spectrometry (IMS) is a powerful analytical technique widely used in biology, chemistry, and materials science fields that continue to expand. IMS provides a qualitative compositional analysis and spatial mapping with high chemical specificity. The spatial mapping information can be 2D or 3D depending on the analysis technique employed. Due to the combination of complex mass spectra coupled with spatial information, large high-dimensional datasets (hyperspectral) are often produced. Therefore, the use of automated computational methods for an exploratory analysis is highly beneficial. The fast-paced development of artificial intelligence (AI) and machine learning (ML) tools has received significant attention in recent years. These tools, in principle, can enable the unification of data collection and analysis into a single pipeline to make sampling and analysis decisions on the go. There are various ML approaches that have been applied to IMS data over the last decade. In this review, we discuss recent examples of the common unsupervised (principal component analysis, non-negative matrix factorization, k-means clustering, uniform manifold approximation and projection), supervised (random forest, logistic regression, XGboost, support vector machine), and other methods applied to various IMS datasets in the past five years. The information from this review will be useful for specialists from both IMS and ML fields since it summarizes current and representative studies of computational ML-based exploratory methods for IMS.

AAPM task group report 273: Recommendations on best practices for ai and machine learning for computer-aided diagnosis in medical imaging.

Rapid advances in artificial intelligence (AI) and machine learning, and specifically in deep learning (DL) techniques, have enabled broad application of these methods in health care. The promise of the DL approach has spurred further interest in computer-aided diagnosis (CAD) development and applications using both 'traditional' machine learning methods and newer DL-based methods. We use the term CAD-AI to refer to this expanded clinical decision support environment that uses traditional and DL-based AI methods. Numerous studies have been published to date on the development of machine learning tools for computer-aided, or AI-assisted, clinical tasks. However, most of these machine learning models are not ready for clinical deployment. It is of paramount importance to ensure that a clinical decision support tool undergoes proper training and rigorous validation of its generalizability and robustness before adoption for patient care in the clinic. To address these important issues, the American Association of Physicists in Medicine (AAPM) Computer-Aided Image Analysis Subcommittee (CADSC) is charged, in part, to develop recommendations on practices and standards for the development and performance assessment of computer-aided decision support systems. The committee has previously published two opinion papers on the evaluation of CAD systems and issues associated with user training and quality assurance of these systems in the clinic. With machine learning techniques continuing to evolve and CAD applications expanding to new stages of the patient care process, the current task group report considers the broader issues common to the development of most, if not all, CAD-AI applications and their translation from the bench to the clinic. The goal is to bring attention to the proper training and validation of machine learning algorithms that may improve their generalizability and reliability and accelerate the adoption of CAD-AI systems for clinical decision support. This article is protected by copyright. All rights reserved.

IMPLEMENTATION OF INDEPENDENT LEARNING CURRICULUM AS A 21st CENTURY LEARNING MODEL IN HIGHER EDUCATION

This study aims to describe the implementation of independent learning curriculum in higher institutions. The independent learning curriculum is newly implemented to respond to the needs of industry 4.0. The Independent learning curriculum is implemented by applying student-centered learning. Students are free to choose the field of science they are interested in even though they are cross-departmental. The method used is a systematic review using the prism method by analyzing research manuscripts. The steps in a systematic review using the Prisma model include; defining eligibility criteria, defining sources of information, selecting literature, collecting data, and selecting data items. Based on the results of the review, it was found that the independent learning curriculum can answer the challenges of industry 4.0. The implementation of the independent learning curriculum is carried out through three aspects, namely, 1) collaboration of the campus curriculum with the independent learning curriculum, 2) the availability of learning facilities, and 3) the development of learning tools. The fulfillment of these three aspects will positively impact the multidisciplinary competence of students.

Peran Pesantren dalam Era Digital

This research aims to find out more deeply about the role of Islamic boarding schools in the digital era that emerged amid the Industry 4.0 and Society 5.0 markets. This research uses a type of qualitative research with a descriptive analysis method and describes it in a systematic, comprehensive, and holistic manner. Then the results of research on the role of Islamic boarding schools in the digital era can utilize various kinds of technology instead of being tossed around by the world of technology. Islamic boarding schools can also take advantage of digital technology in bureaucracy, administration, and in preaching. There are at least two constrictive steps that can be taken by Islamic boarding schools, namely Islamic boarding schools must be able to adapt to the times without losing their characteristics as Islamic boarding schools, and be able to fill digital space wisely. Islamic boarding schools are also undergoing reconstruction in facing the era of Society 5.0, especially in terms of curriculum, they must develop leadership competencies, language skills, IT literacy, writing skills, problem-solving, and critical thinking. As well as the efforts of Islamic boarding schools to enter into PISA, namely implementing HOTS Literacy-based learning, carrying out trials of questions in a format like the PISA test questions, increasing the competence of educators in Islamic boarding schools in HOTS Literacy-based learning, increasing the competence of educators in Islamic boarding schools in developing learning tools. which leads to PISA through Micro-Learning workshops, and provides links to simulation questions for PISA that are easily accessible to students.

Investigating digital skills among Russian higher education students

Learning in the digital age is a pervasive idea that encompasses all aspects of a person's life, including work and leisure. As a result of the development of new teaching and learning tools, an increasing number of students are acquiring knowledge on the Internet- connected to the Internet. Therefore, all citizens must develop digital literacy as a lifelong learning skill. Studies have been conducted on students' digital skills in higher education institutions. In this context, this study aimed to investigate the skills of college students. The participants were students from a university in the Kazan region of Russia who volunteered to participate. Three hundred and eighty students completed the questionnaire online. The scale consists of a total of 25 questions and six dimensions. Since our independent variables are binary values, we applied the Bayesian t-test. We obtained the values of the Bayes factor (BF10) for each dimension and the total scale. In general, students' digital skills are well-developed. However, it was found that creating and using digital information requires fewer skills than in other areas. The hypothesis that there is no difference based on student gender was supported to a higher degree but not to a very high degree. The hypothesis that there is no difference based on students' fields of study was supported to a greater extent, but only to a moderate extent.

Deep learning-based 2D keypoint detection in alpine ski racing – A performance analysis of state-of-the-art algorithms applied to regular skiing and injury situations

ObjectivesIn this study, we examined the practicability of deep learning-based 2D keypoint detection applied to regular skiing and injury situations (i.e., out-of-balance situations and fall situations) on an alpine ski racing track. MethodsWe therefore created a regular skiing- and injury situation-specific dataset (hereinafter called "Injury Ski Dataset"), on which the state-of-the-art keypoint detection algorithms OpenPose, Mask-R-CNN, AlphaPose and DCPose were compared. The performance of each keypoint detector was evaluated by calculating the mean per joint position error (MPJPE) and the percentage of correct keypoints (PCK). Failure cases and common error patterns were further investigated by a visual analysis. ResultsWe observed the best results for regular skiing, with 81%–92% of all keypoints detected correctly at an MPJPE of 9 (2) to 14 (3) pixels. In injury situations, self-occlusions and rare poses became more likely, similar to occlusions due to snow spray and motion blur. As a result, the performance in out-of-balance situations decreased to 68%–80% (PCK), while in fall situations, only 35%–54% of all keypoints were detected correctly, with mean errors of 26–36 pixels. Among all algorithms, AlphaPose was the most robust and achieved the best results. ConclusionsPCK and MPJPE for regular skiing were in the range of manual annotation errors and can be considered low enough for further biomechanical analysis. For fall situations, keypoint detection should be further improved. Regarding the development of a deep learning tool for injury analysis in alpine skiing in the future, we propose to fine-tune a well-performing keypoint detector, such as AlphaPose, on a ski- and injury-specific dataset, such as ours.

The Development of RBL-STEM Learning Materials to Improve Students' Conjecture Thinking Skills in Solving the Antimagic Rainbow Coloring Problem in Greenhouse using Artificial Neural Networks

Learning materials are instruments or equipment used to carry out learning activities. Learning tools serve as guidance during teaching and learning, whether in the classroom, laboratory, or outside the class. This research aims to determine the validity, practicality, and effectiveness of the developed learning tools. This development research refers to the Thiagarajan model (4D), which includes the stages of defining, designing, developing, and disseminating. The developed teaching materials have met the criteria for validity, practicality, and effectiveness. The validity scores obtained for each tool are 3.6 for RTM, 3.5 for LKM, and 3.6 for THB. Practicality scores were obtained from observations of the implementation of learning, resulting in a score of 3.8 with a percentage of 94%, indicating that the learning was well-implemented. Based on the test results, the researcher found that 19 students scored above 60, which means that 83% of the students in this class have completed the course and met one of the effectiveness criteria.