Integrated Performance Assessment Research Articles

An integrated performance assessment method for SMEs fruit processing sustainability in Indonesia

An integrated performance assessment method for SMEs fruit processing sustainability in Indonesia

Integrated Performance Assessments: Providing Equitable Instruction and Assessment for ELLs/MLLs

AbstractIn this Teaching Issues article, we make the case for using Integrated Performance Assessments (IPAs) to get a better picture of ELLs/MLLs learning progress and language development and thus, contributing to more equitable learning environments in both instruction and assessment practices. An Integrated Performance Assessment allows learners to use the three modes of communication—interpretive, interpersonal, and presentational—to communicate their learning and demonstrate language development in an integrated way. We show how IPAs can be used as assessments for, of, and as learning; they promote more authentic teaching and assessment methods, in both formative and summative ways. To illustrate the potential opportunities IPAs can offer over traditional teaching and assessment methods, we share a classroom‐ready example designed by a teacher who participated in a 15‐month professional development project. We posit that IPAs value the resources ELLs/MLLs bring to the classroom and allow them to use these resources in their learning, as well as promote authentic language use and provide a window into ELLs’/MLLs’ progress in ways that advance their learning and language development through collaboration and interaction.

An implementation of integrated performance assessment model in vocational English for communication learning

This research implemented the Integrated Performance Assessment (IPA) model within the context of vocational English for communication learning. The IPA model, proposed by the American Council on the Teaching of Foreign Languages (ACTFL), serves as a comprehensive framework aimed at unifying teaching and assessment methodologies to enhance students' speaking abilities. The primary objective of this research is to evaluate the efficacy of the IPA model in improving students' communication skills within a vocational English setting. First-semester students at AKN Blitar serve as the subjects of this research. A mixed-method approach is employed, encompassing both quantitative and qualitative analyses. Pre-test and post-test stages are conducted, with student scores calculated using the Shapiro-Wilk test to ensure the validity of the findings. The results of this research demonstrate a significant improvement in students' communication skills following the implementation of the IPA model. Despite the observed increase in scores between the pre-test and post-test stages, further investigation reveals that the significance value (p) remains above 0.05 (sig. > 0.05), underscoring the need for continued assessment and refinement of the IPA model in vocational English education.

A case study in eastern Indonesia for the development of integrated real-time performance assessment on science practicum in universities

The purpose of this study was to collect preliminary data for the design of real-time and integrated performance assessments in science practicum at two public universities and two private universities in Eastern Indonesia. The case study design and quantitative and qualitative approaches were adopted in this study. Cluster random sampling technique was used to select four universities, 376 participants respectively from university students and lecturers, four vice deans, four heads of study programs, four heads of laboratories, and four teaching assistants. Data were gathered through questionnaires, in-depth interviews, and observations. The chi-square test was used to analyze the data descriptively. Findings indicated that a performance assessment was not carried out since standard measurement instruments that provided real-time and integrated feedback had not been developed. There was no significant difference (p > 0.05) between public and private universities on the science practicum assessment, implying that both public and private universities still used paper-based practicum assessments. There was also no significant difference between the responses of university students and lecturers on practicum performance assessments (p > 0.05). University students’ experience in assessing science practicums was similar to those of lecturers’, which suggests that practicum of performance assessments was not real-time and integrated with feedback. This study recommends developing a real-time and integrated performance assessment with automated feedback on the science practicum, in order to provide continuous improvement of students and monitoring their learning progress.

Motivational Dynamics of Intermediate Chinese Learners in a COVID-Induced Remote IPA-Informed CSL Curriculum: A Case Study From a CDST Lens

Abstract Grounded in a complex and dynamic systems theory (CDST) perspective of L2 learning motivation, this year-long longitudinal study examines evolving motivational dynamics of two intermediate Chinese as a Second Language (CSL) learners at a Sino-US joint-venture university in China during the COVID-19 pandemic. Adopting Higgins’ framework of motivation, this case study focuses on the interplay of learner motivation for value, control, and truth effectiveness and contextual factors in shaping the contrastive motivational trajectories of two CSL learners who displayed discrepancy in the improvement of Mandarin oral proficiency. In particular, the two contextual factors of interest are an Integrated Performance Assessment (IPA) -informed CSL curriculum and COVID-induced remote online learning. We focus on how learners co-adapted with the “here-and-now” manifestations of these contextual factors and how their motivational dynamics and learning behaviors evolved based on multiple interrelated layers of the temporal-spatial context. The findings show that CSL learners’ motivational dynamics were shaped by multiple overlapping and interrelated motivations related to value, control, and truth effectiveness, which emerged out of the contingencies and affordances in the learning context. The learner-context co-adaptations play a significant role in directing relevant motivational dimensions and thereby shaping the overall identity of the whole system. This empirical study contributes to CSL and L2 motivational studies by exploring the applicability of a CDST approach and a global motivational framework to study intermediate CSL learners’ motivational dynamics and how they individually co-adapted with an innovative curricular design and the perturbance during the COVID-19 online teaching and learning process. The paper reflects on the effectiveness and applicability of IPA in an intermediate-level CSL curriculum from a motivational perspective and offers insights into understanding and promoting the complex learner motivation during COVID-19 for language educators.

Development of an early design tool for the sustainability assessment of positive energy districts: methodology, implementation and case-studies

The concept of Positive Energy District is one of the research ideas that embody the ambitions of decarbonization, renovation (both literal and in a wider perspective) and inclusivity for the urban environment portrayed in the EU activities. In this framework, the paper presents a modeling and simulation tool which allows for an early-design depth to be applied in the field of Positive Energy Districts renovation design and integrated performance assessment. The work aims at creating a tool for stakeholders and designers that would allow them to: a) Calculate carbon impacts along the life cycle for different technical systems and materials used for retrofitting; b) Compute use stage carbon emissions, including import-export of electricity; c) Computations of PED carbon emission balances, along the expected useful life of the district computing both embodied and the use stage carbon emissions. The tool has been created as a spreadsheet including typical profiles of energy use per building archetype, with the inclusion of available and free Life Cycle Assessment data within the life cycle carbon assessment and aims at jointly developing use stage and life cycle considerations. It was tested on a district case studies in the EU.

Integrated Performance Assessment Instrument of Tri Hita Karana's Priority Values in Vocational Learning

This study aims to analyze the success of developing an integrated performance assessment instrument for the priority values of Tri Hita Karana (THK) in vocational learning, in this case, in the practicum course for making men's clothing. This type of research is development research, which was developed using the ADDIE model. The subjects in this study were colleagues and learning and assessment experts. The focus of the research was the development of an integrated portfolio assessment instrument of the priority values of Tri Hita Karana in the practicum course in the manufacture of menswear. Data collection in this study was carried out using the method of questionnaires, with the research instrument being a validity sheet. The data obtained in the study were then analyzed using data analysis techniques using quantitative descriptive analysis. The feasibility of the resulting set of instruments is tested for validation using the Gregory formula using two experts and testing the reliability of the instrument using the inter-rater reliability technique. The results showed that the developed performance assessment instrument met the validity criteria with an average 4.0 (relevant) score and obtained very high validity through calculations using the Gregory formula (1.0). Calculation of reliability through an inter-rater agreement from two experts, which was very consistent with a score on the performance instrument of 0.96. Based on these results, it can be concluded that the design of the integrated performance assessment instrument set of priority values of Tri Hita Karana karana in the Men's Clothing Making Practical Course developed is in the valid and reliable category, so it is very feasible to use to measure students' abilities.

Measurement Criteria and Metrics of Retail Store Performance: An Empirical Study

In this work, we analyze the need for evolution in conventional decision support techniques while keeping in mind the complex and ever-changing requirements of the corporate environment. The goal is to make integrated performance assessment and management more crucial to improving the current decision-making tools, which are now dependent on historical data. It is regarded as a suitable application scenario due to the nature of the difficulties and developments in the retail sector. A framework is also suggested, and a case study is provided as evidence for our claim. Measurement of retailer productivity is necessary to comprehend performance entirely. There are far too many metrics that rely on accounting and financial metrics. Nevertheless, these metrics do not consider the underlying factors influencing bottom-line success. This article examines the idea of productivity concerning retail businesses, emphasizing input and output components and parametric and non-parametric productivity measurements. Furthermore, given are a research agenda and managerial implications.

Integrated performance assessment of urban water systems: Identification and prioritization of one water approach indicators

Conventional water systems are facing challenges of increasing population, aging infrastructure, and climate change incidents. A paradigm shift towards integrated urban water management is desired, which is underpinned by the One Water Approach (OWA) concept. Assessment of integrated urban water system (IUWS) performance is required to manage urban water systems holistically. For this purpose OWA indictors are identified and prioritized in the present study. These indicators represent essential characteristics of water systems relationship, global sustainability attributes, infrastructure features in accommodating increasing water services demands, and holistic building of One Water Community. Twenty-nine indicators have been identified across five performance objectives and ranked by 35 municipal representatives. Fuzzy- Technique for Order Performance by Similarity to Ideal Solution) (F-TOPSIS) was employed to understand the impact of three decision criteria (sustainability, resiliency, and reliability) in identifying indicators and calculate their corresponding scores. Sustainability is a major criterion in identifying indicators, followed by resiliency and reliability. The achieved criteria scores were 0.764, 0.310, and 0.250, respectively. Fuzzy AHP (Analytical Hierarchical Process) was used to prioritize the OWA indicators. Rapid recovery from natural calamities was the top-ranked indicator followed by water leakage reduction, and public willingness to change behaviour, whilst GHG emissions from treatment plants is the bottom-ranked. Statistical analyses were conducted to discern variability in performance scores and determine relationship between decision criteria in indicators identification process. The findings from this study will open new avenues for policy-makers to improve urban water sustainability, resiliency, and reliability by assessing water system performance in an integrated manner to achieve OWA goals.

Integrated Performance Assessment (IPA): Implementation, task types, and feedback

AbstractThis longitudinal, multi‐semester study investigates the Integrated Performance Assessment (IPA) in university‐level communicative Spanish classrooms to find out how instructor feedback affects student performance on IPA assessments, if at all, and in what ways do teachers and students use feedback from the IPAs in their teaching and learning, if at all, to navigate the relationship of assessment and feedback within the IPA framework. A mixed‐methods analysis of the effect of instructor (n = 7) feedback on student (n = 103) progress in the performance of oral and written communication revealed that corrective feedback benefitted learners on interpretive and presentational tasks, but hindered their progress on interpersonal tasks. The findings suggest that instructors should provide feedback in a modality that corresponds with the task type.

Drinking water management strategies for distribution networks: An integrated performance assessment framework

Due to rapid population growth, urbanization, water contamination, and climate change, global water resources are under increasing pressure. Water utilities apply drinking water management strategies (DWMS) to ensure that water is safe for drinking. However, in recent years, due to increased inclination towards climate change, environmental emissions, and sustainable development goals; the environmental and economic performance of DWMSs is getting attention. An integrated framework combining life cycle thinking and water quality assessment techniques was developed in this study to evaluate the DWMSs' performance in terms of water quality, environment, and economics. Six DWMSs were assessed using the integrated framework as a case study. The environmental impacts in terms of human health, ecosystem, and resource use ranged from 1.46E-06 to 4.01E-06 DALY, 9.35E-10 to 3.80E-09 species.yr, and 0.0025–0.0071 USD-$, respectively. Pollution water index (PWI) and cost-benefit analysis (CBA) were used as decision-making techniques to assess the overall performance and suitability of DWMSs under given settings. The DWMSs with surface water as a source or ones providing relatively more degree of treatment have a relatively high PWI score (i.e., ≈0.31), reflective of high environmental impacts and water pollution compared to other alternatives. The CBA scores of selected alternatives ranged between 0.22 and 1.0. Furthermore, it was identified that DWMSs applied on relatively bigger water distribution systems can outweigh their costs (i.e., environmental and economic impacts). The proposed framework and approaches are flexible as they can incorporate different criteria in evaluating the performance and applicability of DWMSs.

The Integrated Performance Assessment at the Novice Level: An Experiment that Works

The Integrated Performance Assessment at the Novice Level: An Experiment that Works

The implementation of the integrated performance assessment (IPA) model to improve English students' speaking skills

This study aims to analyze the extent to which the application of the IPA model could improve Indonesian students’ speaking skill, especially speaking skills for discussion course. It was classroom action research in which the stages of its activities include planning, implementation, evaluation, and reflection. This research’s subjects were the third semester English Department University of Bengkulu. The researchers only took four cycles as the data of the research. Qualitative and quantitative data were analyzed to find a valid conclusion. The results show that the application of the IPA model improved the basic skills of students’ speaking skills in class. It can be seen from the improvement in almost every cycle. The mean score in cycle 1 was 80, followed by 80.3 in cycle 2, but slightly decreased in cycle 3 with a mean score of 79.6 and got higher again in cycle 4 with a mean score of 82. Internal and external factors also contribute to the students’ speaking skills.

Performance Assessment of a nZEB Carbon Neutral Living/Office Space and Its Integration into a District Energy-Hub

The integrated performance assessment of buildings can orient their design in the early stages. Despite the wide availability of physics simulation-based, data-driven, and hybrid techniques, it is often difficult to rely on a single, appropriate technique to obtain reliable results. A set of methods, each featuring advantages and limitations, help to refine the performance assessment in an iterative comparative process until a comprehensive picture of the building is achieved. The approach was implemented on a nearly zero-energy building, recently built-up as a combined living and office space (e.g., the SolAce unit) on the NEST infrastructure in Dübendorf (Switzerland). The proposed approach showed that the unit reaches high energy performance accordingly requiring optimal cooling management, involving the control of the opening of blinds and windows. A sound convergence between the computer simulations and data-driven analysis were observed, attesting to the overall energy consumption, of around 26 kWh/m2year, in continuous decrease, aiming at an annual energy-positive balance. The unit was ranked first according to the dynamic energy exchange scheme of the energy trading hub within the NEST facility, which features high-level building modules as a testbed of future building technologies. Embodied energy is estimated at 39 kWh/m2year, which is below the commended limits of Swiss eco-building standards. By considering the carbon sequestration of the wood products during their lifespan, the unit is very close to carbon neutrality with the CO2 emitted annually by the unit over its lifetime being compensated by those stored within wood products during the same period. Highlights: An innovative combined living/office space is connected to an ‘Energy-hub’. The unit includes novel bioclimatic developments of the building industry. Energy performance is assessed in the concept phase, construction, and operation. Used and compared simulation vs data monitoring for energy and comfort assessment. High energy performance and neutral CO2 emission; plus-energy exported to the hub.

Integrated Approach to Evaluating the Effect of Indoor CO2 Concentration on Human Cognitive Performance and Neural Responses in Office Environment

According to a guideline published by the American Society of Heating, Refrigerating and Air-Conditioning Engineers (ASHRAE), people spend 80%–90% of their lives indoors; therefore, managing indoor environmental quality is becoming increasingly significant. In particular, a very high level of indoor CO2 concentration can seriously impair people’s comfort, health, and productivity. The ASHRAE recommends maintaining indoor CO2 concentration below 1,000 ppm. This study proposes a novel integrated approach to evaluating the effect of indoor CO2 concentration on human cognitive performance and neural responses in office environments from multiple perspectives. An integrated performance score (IPS) w developed to investigate objectively and comprehensively how indoor CO2 concentration affects the level of cognitive performance compared with the neural responses of the subjects, in which the brain regions were considered along with the brain-wave frequency bands. Thirty participants were asked to engage in nine (three×three) experiments, i.e., three cognitive tasks (i.e., working memory, mental workload, and visual concentration) under three CO2 concentration levels (i.e., a low CO2 level of 500 ppm; a moderate CO2 level of 1,000 ppm; and a high CO2 level of 2,500 ppm). During the experiments, electroencephalography (EEG) signals were recorded. Compared with the individual approach, the integrated approach can explain more clearly that an increase in indoor CO2 concentration had a detrimental effect upon working memory, mental workload, and visual concentration. The study breaks new ground in applying a biometric research strategy in the domain of management in engineering, using EEG-based integrated performance assessment. This study can contribute to properly designing the space size and the number of occupants during space programming. In addition, it can be used for optimally designing and operating ventilation systems at proper size and with proper strategy. Consequently, it is expected that the negative effect of indoor CO2 concentration on occupants can be minimized.

Nature inspired evolutionary algorithm integrated performance assessment of floating solar photovoltaic module for low-carbon clean energy generation

Development and deployment of FSPV systems are still in the nascent stages hence long-term performance, control and feasibility study of FSPV systems are not well addressed. Precise and robust estimation of FSPV panel parameters will play an important role in determining the actual performance, carbon savings and long-term feasibility studies of FSPV systems. Here, hybrid stochastic firefly algorithm (HSFA) is used for parameter estimation and optimization. The hybrid algorithm is used to find out the parameters of single diode model, double diode model and FSPV module. An experiment is also performed using FSPV modules under varying irradiance conditions. The accuracy of model is evaluated by comparing the simulated results with the experimental results and computing the relative error and root mean square error (RMSE). The parameters extracted using the proposed method has a very low RMSE value of 9.83002E-04. Assessment of the experimental and estimated results show that the relative error for measured electricity on a sunny day is 0.57% while for an overcast day it is 0.89%. For partial shading condition, the relative error and RMSE was 0.79% and 6.5%, respectively. The results which are in well agreement with the experimental values demonstrate the superior performance of the model in determining the FSPV parameters. Proper estimation of the FSPV parameters will help researchers, scientists, engineers and all actors associated with solar PV systems in making sound judgements towards the deployment of FSPV systems and help the society in developing a sustainable ecosystem towards implementation of industry 4.0 by adapting to low-carbon power generation methods.

The development of temporal-spatial meaning in personal recounts of beginning L2 writers of German

Personal recounts represent a core narrative genre that novice L2 learners typically encounter when learning about everyday experiences in the target culture. Expressing a “journey of significance” (Rothery & Stenglin, 1997, p. 248) through a succession of well-defined events, recounts are ideal textual environments for beginning learners to practice verb forms and cohesive devices (e.g., Abdel-Malek, 2019). A key feature of recounts are temporal-spatial circumstantial meanings, often realized lexico-grammatically through adverbial and prepositional phrases that guide movement through a sequence of habitual actions and reported happenings. While circumstances enhance (Halliday & Matthiessen, 2004) recounts by providing additional details to situate events, L2 learners sometimes avoid producing these meanings (Gleason, 2014).Adopting an Integrated Performance Assessment (Adair-Hauck, Glisan, & Troyan, 2013) and genre-based approach (Martin, 2009; Martin & Rose, 2008), this study explores how collegiate beginning L2 learners of German develop their ability to write stories of travel experiences. Data come from two second-semester German classes in fall 2019 (n = 10). Pre- and post-tests of learners’ written recounts of their own vacation experiences provide information about learners’ developing facility with the genre, including the ability to produce temporal-spatial meaning discursively. Implications for supporting and evaluating beginning L2 learners’ narrative writing within Integrated Performance Assessments are offered.

Performance assessment framework based on competitive priorities for sustainable freight transportation systems

Stringent environmental norms and growing demands for energy-efficient and cost-effective freight shipping have put organizations under immense pressure to consider sustainability aspects while assessing their performance. Moreover, amid growing awareness about competitive priorities, it has become prudent to understand how an organization’s sustainability performance is balanced against competitive priorities. Therefore, this study attempts to develop an integrated performance assessment framework (PAF) on the basis of competitive priorities for sustainable freight transportation (SFT) systems. A unified approach consisting of fuzzy group decision-making, fuzzy evidential reasoning approach, and expected utility concept is utilized to assess critical success factors of SFT based on four competitive priorities. The applicability of the developed model is exhibited through a case example, and sensitivity analysis is also performed to examine its robustness. This competitive performance score based PAF would facilitate logistics professionals in selecting the most competitive SFT system.

Developing culture-based materials with integrated performance assessment

The aims of this study were to identify the target needs, describe the learning needs and describe the appropriate culture-based material with Integrated Performance Assessment for the eighth grade students of SMP N 1 Muntilan. This study was classified as research and development (RD). The study procedure was adapted from ADDIE models proposed by Branch (2009). The data were collected through questionnaires (need analysis questionnaire, expert judgment rating scale, interview transcripts, and questionnaire for students about the materials). The data from the needs analysis, the expert judgment questionnaire, and the questionnaire for students about the materials were analyzed quantitatively using frequency and percentage. Meanwhile, the data from the interview were analyzed using descriptive. The materials in the developed materials were developed based on Curriculum 2013. The research findings showed that the developed materials was considered appropriate. The result of the materials evaluation questionnaire was 3.81 which categorized as very good.

Closing the pre and in‐service gap: Perceptions and implementation of the IPA during student teaching

AbstractThis research explored the experiences of nine preservice teacher candidates of Spanish in world language teacher education programs at two higher education institutions implementing the Integrated Performance Assessment (IPA) as Student Teachers (ST), as well as perceptions of seven Cooperating Teachers (CTs) and the University Field Supervisor. This qualitative, multiple‐case study included data from interviews, focus groups, and classroom observations. Four themes emerged from the data analysis: (a) STs’ preparation and CT's familiarity with the IPA; (b) STs’ implementation of IPA and CTs’ reactions; (c) impact of clinical experience on STs and CTs' pedagogical beliefs; and (d) professional development needs. The article concludes with recommendations for world language teacher certification programs and professional development opportunities for in‐service, CTs.