Performance-based Assessment Research Articles

Predictors of student pharmacist success on an advanced pharmacy practice experience-readiness performance-based assessment

IntroductionThe study objective was determining the relationship of certain predictor variables with student performance on an advanced pharmacy practice experience (APPE)-readiness performance-based assessment (PBA) in the third professional year (P3). MethodsThis retrospective study aimed to identify if student predictors and/or curricular-related predictor variables correlate with student performance on a patient interview and documentation activity (PIDA), an APPE-readiness assessment. Student predictors included Pharmacy College Admission Test score, pre-pharmacy cumulative grade point average (GPA), end of first professional year (P1) GPA, end of second professional year GPA, end of fall P3 GPA, and elective Pharmacists' Patient Care Process (PPCP) score. Curricular-related predictor variables comprised specific activities with emphasis on patient care skill development in P1 through P3 years. Adjusted and unadjusted linear regression models assessed correlations between PIDA score and student predictors as well as curricular variables. ResultsStudents from the Classes of 2018 through 2020 at Southern Illinois University Edwardsville School of Pharmacy (N = 231) were included. Two student predictors, P3 GPA and PPCP elective score, showed statistically significant correlations with PIDA performance. One curriculum-related variable, P3 activity with an emphasis on medication reconciliation, demonstrated a statistically significant correlation with student performance on the PIDA. ConclusionsStudent performance on a cumulative PBA may be predictive from prior curricular learning experiences that assess similar patient care skills. The predictive impact of the elective PPCP score on a PBA needs additional research. Recent implementation of a new curriculum may lead to identification of other predictors of student success on PBAs.

A Proposal for Performance-based Assessment of the Learning of Machine Learning Concepts and Practices in K-12

Although Machine Learning (ML) is used already in our daily lives, few are familiar with the technology. This poses new challenges for students to understand ML, its potential, and limitations as well as to empower them to become creators of intelligent solutions. To effectively guide the learning of ML, this article proposes a scoring rubric for the performance-based assessment of the learning of concepts and practices regarding image classification with artificial neural networks in K-12. The assessment is based on the examination of student-created artifacts as a part of open-ended applications on the use stage of the Use-Modify-Create cycle. An initial evaluation of the scoring rubric through an expert panel demonstrates its internal consistency as well as its correctness and relevance. Providing a first step for the assessment of concepts on image recognition, the results may support the progress of learning ML by providing feedback to students and teachers.

Diversity pedagogical content knowledge: a new conceptual framework and assessment across different teacher education programmes

ABSTRACT Previous work on multicultural teaching competence has mainly focused on teachers’ beliefs and attitudes towards ethnocultural diversity. Little attention has been paid to examine the nature of teachers’ ethnocultural diversity knowledge. Drawing upon Shulman’s model of pedagogical content knowledge, this study presents a novel theoretical framework and a performance-based assessment for examining teachers’ ethnocultural diversity knowledge. We collected data from 819 preservice teachers across six teacher education programmes in Belgium, Flanders. Our findings indicate that student teachers in Flanders have limited and low levels of cultural diversity knowledge and there is a significant variation between the mean scores of teacher education programmes. We conclude by arguing that teacher education institutions should make meaningful reforms to develop student teachers’ knowledge base regarding ethnocultural diversity.

Frequency of Performance-Based Assessments in Secondary School Computer Studies and Its Influence on Students’ Innovation Capacity in Kandara Sub-County, Kenya

The central purpose of carrying out this study was to establish the extent to which the frequency of Performance-Based Assessments (PBAs) in secondary school Computer Studies influences students’ innovation capacity. A correlational research design was used to conduct the study. Stratified sampling using proportional allocation and random sampling were used to select a total of 313 Computer Studies students from a target population of 1561 students. Data on the frequency of PBAs and students’ innovation capacity was collected through Computer Studies Students questionnaire whose Cronbach Alpha coefficient,a for internal consistency was 0.817. Pearson moment correlation and simple linear regression were used to analyze data inferentially. The study found a low frequency of PBA use in Computer Studies in the secondary schools which had negatively influenced students’ innovation capacity. The study recommends an increased frequency of PBAs in the subject’s curriculum implementation and close monitoring of Computer Studies PBAs implementation in order to ensure that teachers implement them more often. Keywords: Performance-Based Assessments, innovation, innovation capacity, secondary school, Computer Studies DOI: 10.7176/JEP/12-30-08 Publication date: October 31 st 2021

Students First: Equity, Access, and Opportunity in Higher Education

Students First: Equity, Access, and Opportunity in Higher Education

Probabilistic Model Based on Bayesian Model Averaging for Predicting the Plastic Hinge Lengths of Reinforced Concrete Columns

A probabilistic model is devised for predicting the plastic hinge lengths (PHLs) of RC columns. Seven existing parametric models are evaluated first using a comprehensive database comprising PHL measurements from 133 RC column tests. It is observed that the performances of these seven models are fair (as opposed to strong), and their predictions bear significant uncertainties. A novel technique is devised to combine them into a weighted-average supermodel wherein the weights are determined via Bayesian inference. This approach naturally produces the weights’ statistical moments, and thus, the resulting model is a probabilistic one that is amenable for performance-based seismic design and assessment analyses. Prediction comparisons indicate that the proposed supermodel has a higher performance than all prior models. The new model is easily expandable should more test data become available.

Performance-based probabilistic assessment of liquefaction-induced building settlements

The current state of practice in the estimation of liquefaction-induced building settlements (LIBS) relies on pseudoprobabilistic approaches in which the estimation of ground motion intensity measures (IMs) is separated from the estimation of LIBS. In contrast, in a performance-based probabilistic approach, the estimation of the IM hazard is coupled with the estimation of the LIBS hazard. Thus, engineers can obtain LIBS estimates that are directly related to a selected design hazard level (or return period), which is more consistent with performance-based design.In this study, we present new developments for the performance-based probabilistic evaluation of LIBS, including 1) the performance-based assessment of LIBS considering the hazard from a single IM in the context of scalar probabilistic seismic hazard assessment (PSHA), 2) the performance-based assessment of LIBS considering the hazard from multiple IMs in the context of vector PSHA, 3) deaggregation of earthquake scenarios from LIBS hazard curves, 4) estimation of LIBS hazard curves in areas where earthquakes from multiple tectonic settings can occur (e.g., shallow crustal, subduction), and 5) treatment of uncertainties (i.e., aleatory and epistemic). The developments are implemented in a computational platform named "LIBS", which is fully coupled with PSHA assessments and facilitates the straightforward performance-based estimation of LIBS in engineering practice. Finally, we perform comparisons of performance-based and pseudoprobabilistic-based estimates of LIBS and share insights from the comparisons.

Performance-based seismic design and assessment of rocking timber buildings equipped with inerters

Over the last decades, performance-based design objectives have shifted towards damage control and continuity of operation after a design-level earthquake. In this context, the advantages of rocking have been applied to the development of a family of self-centring systems that can sustain large lateral deformations without noticeable damage. However, the bending moments and shear forces in uplifting structures can increase significantly due to the effects of higher modes, an issue to which timber structures are particularly prone. This paper assesses the seismic response of post-tensioned timber rocking walls combined with inerters as a means to control the rotation amplitude and suppress higher-mode effects on the system. To this end, a representative set of three post-tensioned rocking walled structures, comprising 3, 6 and 9 storeys, are designed following direct-displacement based design procedures. A simplified method to pre-dimension the inerter device is proposed and used to design a set of ball screw and gear inerters, with and without clutches. The performance of bare and protected structures with different levels of apparent mass ratios is assessed and compared considering a set of 7 records consistent with the displacement design spectrum. Special attention is paid to the resisting force developed in the inerter and the mechanism to transfer it safely to the structural diaphragm. Finally, a detailed performance-based assessment is conducted considering a database of 202 pulse-like ground motion records. It is concluded that the innovative combination of inerters and rocking is an efficient way to improve the seismic control of self-centring structures.

Effects of household chaos and parental responsiveness on child executive functions: a novel, multi-method approach

BackgroundExecutive functions can be adversely affected by contextual risks in the home environment including chaos and parenting challenges. Furthermore, household chaos negatively influences parenting practices. Few studies, however, have examined the role of parenting in the association between household chaos and child executive functions.MethodsUsing a sample of 128 school-aged children (mean = 61.9 months, SD = 2.0, range 58–68 months) and their mothers, the present study examined direct and indirect effects (via parental responsiveness) of household chaos on child executive functioning. Multi-measures were used including performance-based assessments, behavioural observations, questionnaires, and video-home tours.ResultsHousehold chaos had both a direct effect on child executive functions (β = − .31, 95% CI [− .58, − .04]) and an indirect effect (β = − .05, 95% [− .13, − .01]) via parental responsiveness. Further, the indirect effect was only significant for household instability.ConclusionThese findings indicate that parental responsiveness may be compromised by household chaos, with implications for the executive functions of school-aged children. Preventative strategies are needed to improve the stability in the home and strengthen parenting practices.

Seismic performance of steel-reinforced concrete composite columns in existing and modern construction

The seismic performance of steel-reinforced-concrete (SRC) composite columns with non-seismic details in older buildings is not sufficiently characterized due to lack of cyclic test data. Formal recommendations for cyclic backbone curves of these columns for performance-based seismic assessment are not currently available. In addition, the effect of axial loads on the deformation capacity of SRC columns is uncertain. Furthermore, SRC composite columns with modern seismic details lack formal recommendations for nonlinear modeling parameters or acceptance criteria. This study aims to fill some of these knowledge gaps via experimental and analytical means. The cyclic behavior of five SRC composite columns with non-seismic and seismic details failing in flexural-tension or flexural-compression modes has been experimentally investigated. Test results show that tension-controlled SRC columns with non-seismic details can sustain up to 6% drift ratio with minor strength degradation, satisfying modern seismic assessment criteria and suggesting no need to be retrofitted. However, high axial loads were found to jeopardize drift capacity of compression-controlled SRC columns with non-seismic details (2% peak-load drift capacity), leading to premature axial failure. Test results suggest the possibility of relaxing the ACI 318–19 confinement requirements for tension-controlled SRC columns with seismic details. A database of past cyclic tests of SRC composite columns with seismic and non-seismic details has been developed. Database response parameter trends have been analyzed and compared. Based on the current test results and the database trends, new drift and ductility capacity expressions for SRC columns with non-seismic details in older buildings are proposed. Furthermore, based on current test results and the database trends, new ASCE 41-17-style lower bound nonlinear modeling parameters for flexural-controlled SRC composite columns with non-seismic details are proposed for performance-based seismic assessment.

In-plane seismic behaviour of ashlar three-leaf stone masonry walls: Verifying performance limits

ABSTRACT In light of the forthcoming second generation Eurocodes (EC), the results of conducted systematic in-plane cyclic and compressive tests on three-leaf stone masonry walls are discussed following new requirements and provisions. The new proposal for EC8-3 for existing buildings is based on partial factors safety approach, though it considers different uncertainties in defining input parameters for effective seismic performance-based assessment. Prior to its application, massive calibration effort will be needed since there is no standardized method for shear testing of masonry walls. In this paper, the performance limit states damage, resistance, and displacement capacities from conducted test results were evaluated and assessed through comparison with analytical solutions and imposed limit values, as stated in existing codes. The test results provide a much higher deformation capacity than the limits provided in both existing and new proposal of EC8-3 as well as those in the ASCE code provisions. The reason for this lies in the soft, “ductile” mortar for which the presumed resistance according to code provisions should be significantly higher when considering good quality ashlar three-leaf stone masonry.

Surrogate models for endplate beam-column connections with shape memory alloy bolts

This paper presents the development and validation of moment-rotation backbone curve parameters for self-centering steel endplate connections with superelastic SMA bolts. In terms of influential design factors, surrogate models of response parameters are proposed using detailed finite element models of self-centering beam-column connections and statistical analysis of response parameters. The predictive surrogate models are useful for characterizing the connection backbone and self-centering response, which is essential for the performance-based design and assessment of extended endplate connections equipped with SMA bolts. Through a verification study, the accuracy of predictions using the proposed models is confirmed. The proposed surrogate models are used to assess the fragility of SMA-based beam-column connections. The results demonstrate that endplate beam-column connections with SMA bolts can provide an acceptable level of deformation capacity. Furthermore, it is found that larger diameter SMA bolts and smaller depth beam sections can reduce the probability of experiencing SMA bolt fracture by 31% and 77%, respectively.

The Implementation of Project-Based Learning in Pilot Project Elementary Schools in Banyumas District

This descriptive research examines the tendency of teachers in implementing project-based learning on thematic materials in pilot project schools of the 2013 curriculum. This research aims to describe: 1) the types of product produced in the implementation of project-based learning, 2) the realization of student's involvement in the implementation of project-based learning, 3) the student's skills in the implementation of project-based learning; 4) the assessment instruments used by teachers in the implementation of project-based learning. The result of this research shows that 1) the produced products are real objects, written reports, oral presentations, and dance. 2) the student's involvement includes seeking information, exploring information and data, creating products, sharing, and only two teachers involved the students on solving problems. 3) several skills given to the students are the use of material, collaboration and cooperation, oral and written communication, technology mastery, and there were two teachers who provided critical thinking skill. The assessment instruments used were: product-oriented performance-based assessment, practice-oriented performance-based assessment, and there were two teachers who used exhibition. No teachers applied Journal assessment. Based on the result of the research, it can be concluded that the tendency of teachers on the implementation of project-based learning in elementary school level has not provided the critical thinking skill and complex problems.

The Rorschach and Violent Crime

Abstract. Over the years, a significant number of Rorschach studies have been conducted with forensic adult and adolescent samples, partly motivated by the use of the test in forensic psychological evaluations. Could the Rorschach, as a performance-based personality assessment tool, provide unique information that is not as vulnerable to distortion on the part of the examinee as self-report measures are? This article provides a review of Rorschach studies on relevant Rorschach variables, including those with different forensic samples. Empirical findings are mixed; there is not a one-on-one relationship between certain Rorschach variables and forensically relevant traits, such as psychopathy or hostility. This does not mean the Rorschach cannot provide useful information in answering psychological questions before the court. A case illustration of a male college student, who committed a (first) violent offense, illustrates the unique contribution of the Rorschach for understanding the psychological dynamics behind a violent act that was seemingly out of character.

Longitudinal Outcome of Programmatic Assessment of International Medical Graduates.

IntroductionAustralia depends on international medical graduates (IMGs) to meet workforce shortages. The current standard assessment for IMGs is by clinical examination in observed structured clinical encounter (OSCE) format lasting 200 minutes. There are concerns about adequateness of this assessment as it does not test the qualities required to practice in a new country. We introduced a programmatic performance-based assessment for IMGs to prepare them to meet these challenges. The workplace-based assessment (WBA) program involves six-month longitudinal programmatic assessments comprising of 12 mini-clinical evaluation exercises (Mini-CEX), five case-based discussions (CBD), two in-training assessments (ITAs) and two sets of multisource feedback (MSF) assessments. We assessed 254 IMGs since 2010. We conducted a survey to evaluate the satisfaction with the program and the outcomes of these doctors.MethodsWe surveyed 254 candidates from 2010 to 2020. The survey used “SelectSurvey” tool with 12 questions and free-text comments. All candidates were sent the survey link to their last registered mobile phone using “Telstra Instant Messaging Service”. We analysed the data using Microsoft “Excel”.ResultsWe received 153 (60%) responses. Amongst them, 141 (92%) candidates did not require further supervised practice for general registration and 129 (84%) candidates hold general/specialist registration. The candidates found the program useful and felt well supported. They appreciated real patient encounters. The feedback with positive critiquing was helpful in improving their clinical practice. The negative themes were program costs and frustration with the length of the program.ConclusionUpon completion of the WBA program and obtaining the AMC certificate, most of the doctors were able to gain general registration. Seventy-eight (50%) candidates chose to continue their careers within the local area with 124 (80%) of them within the state. Our survey shows a comprehensive assessment program with immediate constructive feedback produces competent doctors to fill the medical workforce shortages.

The Influence of Work Autonomy, Workload, Work Bornage and Interpersonal Conflict on Nurse Performance in Patient

The problem of work stress is still a trend in various parts of the world. In Europe, the problem of work stress ranks second after musculoskeletal disorders. Sources of work stress which consist of autonomy, workload, work saturation and interpersonal conflicts are a challenge for hospitals in terms of employee performance issues. The purpose of this study was to determine the effect of work autonomy, workload, work saturation and interpersonal conflict on the performance of nurses in the Inpatient Installation of RSUD Bima.The research design used was non-experimental quantitative analytical observational. The research sample was 154 nurses in the inpatient room. The analysis in this study used univariate, bivariate and multivariate analysis with multiple linear regression with 95% confidence level or nilai <0.05.The results showed that the value of work autonomy (0.000), workload (0.013), work boredom (0.014) and interpersonal conflict (0.006) < 0.05 which means that all variables affect the performance of nurses and result of logistic regresion it was found that the most dominant variable was interpersonal conflict with the highest Odds Ratio value of 3.764,which means that interpersonal conflict was the most influential variable on the performance of nurses.Based on the results of the study, it can be concluded that work autonomy, workload, work saturation and interpersonal conflict affect the performance of nurses, and the most influential variable is work saturation with the largest t value of 2.486. It is recommended for the Head of Nursing and Head of Room to evaluate the performance of nurses in the room to the maximum, and make a performance-based supervision program and assessment of service users

Caregiver rating of early childhood development: Reliability and validity of the ASQ-3 in rural Guatemala

BackgroundAlthough performance-based assessment of early childhood development is preferred, there are a number of limitations to this methodology in low resource settings (LRSs). Hence, clinicians and researchers often rely on caregiver report screening tools. The Ages and Stages Questionnaire 3 (ASQ) is one of the most widely used caregiver report measures globally. Adequate psychometric properties have been demonstrated in high income settings, especially when used in older children, high- risk children, or those with severe neurodevelopmental delays. However, its utility is more variable within very young children and for use in LRSs. MethodsThe reliability and validity of the ASQ was determined for children ages 0–5 years living in rural Guatemala. Internal consistency and test-retest reliability were assessed, as well as concurrent and predictive validity. Sensitivity, specificity, positive and negative predictive values related to performance-based developmental assessment (Mullen Scales of Early Learning; MSEL) and growth status (i.e. stunting) were also calculated. ResultsInternal consistency reliability for the ASQ was adequate, except when results were limited by small sample size. Test-retest reliability ranged from low to moderate (r = 0.08–0.43; p < 0.05–0.01). However, there was significant variability in mean scores over time across ASQ domain scores. In terms of validity, the ASQ did not discriminate adequately between children who performed within or below age-expectations on performance-based developmental testing or those with and without stunting. ConclusionsThe ASQ did not demonstrate adequate psychometric properties in rural Guatemala, consistent with concerns documented in other LRSs. These results indicate that existing caregiver report screening measures of early childhood development should be utilized with caution in LRSs, and alternative methods for assessment or in the development and utilization of caregiver report measures should be considered.

Performance-based assessment: Self-efficacy, decision-making, and problem-solving skills in learning science

Performance-based assessment is a purposive-oriented assessment activity in revealing student understanding of unwrapped concepts, skills, and big ideas. This study aimed to determine the effects of performance task assessment on the students' self–efficacy, problem–solving, and decision–making skills in learning science that used the quantitative and correlation methods of research. There were 35 Grade 9 students of the University of San Carlos, Basic Education Department – South Campus in science class as respondents of the study. The assessment was given after the engagement session of the instructional plan to be accomplished after the transfer task session and evaluated using specific rubrics. A modified Albert Bandura's Instrument was used to measure self–efficacy and Mind Tool: Essential Skill for Excellent Career questionnaires for the problem–solving and decision–making skills of the students in learning science. The study's findings indicated that the Performance Assessment developed students' self–efficacy in learning science and had a significant impact on how students think, feel, and behave in the process of learning. Moreover, the assessment conveyed positive influence in developing students' problem–solving and decision–making skills. This assessment encouraged students to apply the scientific processes in evaluating the identified options before deciding the problem's solution.

An alternative approach for measuring computational thinking: Performance-based platform

Educators emphasize that computational thinking is especially helpful in raising problem-solving skills, improving logical reasoning, and developing analytical thinking. In this context, the aim of this study is to develop an online, interactive, valid, reliable and useful performance-based assessment tool for measuring computational thinking skills. Data were collected in two stages with three different measurement tools. In the data collection phase, a psychometric scale, a skill test and an interactive web application developed in this study were used. In the study in which a total of 203 high school students participated, it was determined that the students' computational thinking skills were at average level. as a result, a valid and reliable performance-based platform was developed for measuring computational thinking skills. It has also been concluded that the performance-based measurement tool makes a better measurement than the psychometric scale. It is emphasized that computational thinking should be treated as a skill or group of skills, not as a theoretical knowledge, product/portfolio or attitude/perception. Therefore, it is believed that developing performance-based tools for evaluating computational thinking skills and measuring them by using multiple measurement tools will give more accurate results. Considering the developments in computational thinking, it is thought that the study will contribute to the literature.

Performance-based assessment of social skills in a large sample of participants with schizophrenia, bipolar disorder and healthy controls: Correlates of social competence and social appropriateness

IntroductionPerformance-based assessments of social skills have detected impairments in people with severe mental illness and are correlated with functional outcomes in people with schizophrenia and bipolar disorder. The most common of these assessments, the Social Skills Performance Assessment (SSPA), has two communication scenarios and items measuring both social competence and appropriateness. As real-world competence and appropriateness appear to have different correlates, we hypothesized that SSPA Items measuring competence and appropriateness would be distinct and have different correlations with other outcomes. MethodsWe aggregated data from 557 people with schizophrenia, 106 with bipolar disorder, and 378 well controls from 4 separate research studies. All participants were assessed with both SSPA scenarios and other performance based and clinician-rated measures. A single expert rated the SSPA interactions for competence and appropriateness while blind to participant diagnoses. ResultsParticipants with bipolar disorder and schizophrenia performed more poorly on every item of the SSPA than healthy controls. Items measuring social competence and appropriateness in communication were intercorrelated across scenarios, as were elements of socially competent communication, although the items measuring competence did not correlate substantially with appropriateness. Items assessing social competence, but not social appropriateness, correlated with better cognitive and functional performance and residential and financial independence. DiscussionSocial competence and social appropriateness were distinct elements of performance-based social skills with potential differences in their functional correlates. As both social competence and appropriateness impact functional outcomes, improvement in the measurement and treatment of appropriate communication seems to be an important goal.