Performance Assessment Items Research Articles

Developing and validating the multidimensional industry commitment scales: The perspective of vocational high school students

<span>Industry commitment plays a vital role in vocational education programs, especially in vocational high schools. The involvement of the industry is the key to the success of implementing vocational education. However, how to measure industry commitment in vocational learning is still not discussed. This study aimed to develop and test the validity of an industry commitment questionnaire. The industrial commitment questionnaire validation test gradually used three techniques, namely, content validity test, pilot test, and confirmatory factor analysis (CFA). The respondents were 390 culinary students of seven state vocational schools in Yogyakarta Special Region Province, Indonesia. This study revealed that industrial commitment can be measured by 12 items of industry commitment questionnaire consisting of four career opportunity items, two performance assessment items, four involvement/participation items, and two empowerment items. This study fills a gap in the assessment of industry commitment to engagement in vocational education programs. This study has implications for vocational education practitioners to evaluate the extent of industry commitment to vocational education programs.</span>

ON INTERDISCIPLINARY COMPARATIVE STUDY OF ANALOGICAL FEEDBACK/ASSESSMENT MODELS APPLIED IN BLENDED LEARNING VERSUS COMPUTER AIDED LEARNING USING ARTIFICIAL NEURAL NETWORKS

This paper provides educationalists as well as researchers in computer science and engineering with a study of an interdisciplinary challenging pedagogical issue. More specifically, the presented study resulted in a set of interesting findings that originated from adopted realistic Artificial Neural Network (ANNs) modeling, which is associated with two educational analogical feedback/assessment processes. This piece of research considers comparative analysis and evaluation study of an educational phenomenon issue for two diverse teaching/learning methodologies namely :{ Blended Learning (BL) and Computer Aided Learning (CAL)}. More precisely, the introduced issue of this work addresses the two summative and formative assessment processes applied in educational field practice. Accordingly, this issue is concerned mainly with the modeling of two practical field case studies considering the two items of educational feedback/assessment. In other words, assessment is used in many ways in education, a great deal of attention is now given to its use in helping teaching and learning, described as the two performance assessment items (summative and formative). These are defined as: assessment for learning (A f L), or formative assessment, and assessment of learning (AO L), or summative assessment. It is noticed that (AO L), is focused on which is used to summarize what students know or can do at certain times in order to report achievement and progress. Herein, two parametric factor values of ANN (gain, and learning rate factors), have been considered for the two suggested instructional methodologies. That is considered in order to compare, analyze, and evaluate dynamically two items of academic performance namely: (Academic achievement outcome & Learning convergence time) for both methodologies. Interestingly, the running realistic ANN computer modeling for different numbers of neurons -that are contributing to the learning process- results in an investigative, comprehensive, and innovative systematic analysis of individual students’ differences. Finally, after performing a perceptive evaluation comparing two case studies of obtained experimental field results, two interesting findings have been concluded. Firstly, while comparing computed sets of statistical parameters, associated with the presented instructional methodologies (CAL & BL). That resulted in the observed analogy between both sets. Secondly, in the context of Feedback/Assessment performance, regarding both (BL& CAL); either Formative or Summative feedback /assessments, have been observed to be well analogous to each other.

Underlying factors influencing community pharmacies business in low- to medium-developed markets: a societal perspective

Objectives To identify (1) critical local factors affecting community pharmacy performance and (2) items that are important in assessing community pharmacy performance. Methods A cross-sectional survey based on the opinions of the pharmacists involved in managing a community pharmacy in Iran at least for 3 years by the time of the study was conducted, in 2014. The questionnaire was anonymous and self-administered and consisted of three parts, which constructed to address research questions. Descriptive and factor analysis were performed using spss (16.0.0). The size of the effect of each dimension was investigated by lisrel (8.54). Key findings Based on the respondents' opinions, product (0.89), leadership (0.86) and promotion (0.82) were the three most influencing intra-organisational dimensions, and price (0.37) was the least important one. Regard to extra-organisational factors, the highest and lowest scores were obtained for participants (0.86) and regulatory issues (0.58) respectively. Among performance assessment items, clients' results (0.76) and society results (0.22) set at the first and last priority respectively. Conclusions Authors believe that this study provides decision makers and researchers with a comprehensive collection of factors affecting the community pharmacy area, which after minor modifications based on local issues is applicable in countries that are similar to Iran with regard to the level of the reprofessionalisation of community pharmacies.

쉴드 TBM 터널의 구조물 성능 평가 기준 개발

In this study, the performance assessment criterion for reasonable maintenance of shield TBM tunnel was presented. The performance assessment items such as crack, leakage, breakage, spalling, exfoliation/detachment, efflorescence, quality condition, exposure of steel, carbonation, faulting step, bolts condition, drainage condition, ground condition, contact section condition and conduit condition were selected by analyzing domestic and foreign performance assessment criterions and investigating segment lining deterioration cases through the site investigation and in-depth inspection analysis result on the shield TBM tunnel. In addition, the reasonable weight using AHP (Analytic Hierarchy Process) were estimated.

Using Latent Class Analysis to Set Academic Performance Standards

This study investigated using latent class analysis to set performance standards for assessments comprised of multiple-choice and performance assessment items. Employing this procedure, it is possible to use a sample of student responses to accomplish four goals: (a) determine how well a specified latent structure fits student performance data; (b) determine which latent structure best represents the relationships in the data; (c) obtain estimates of item parameters for each latent class; and (d) identify to which class within that latent structure each response pattern most likely belongs. Comparisons with the Angoff and profile rating methods revealed that the approaches agreed with each other quite well, indicating that both empirical and test-based judgmental approaches may be used for setting performance standards for student achievement.

Concept Mapping and Science Achievement of Middle Grade Students

This study examined the effects of concept mapping on the science achievement of middle grade science students. The subjects were 182 eighth-grade students, distributed into eight intact earth science classes by ability levels. The ability level variable also was examined as a possible effect on student achievement. Two teachers were involved in teaching a unit on weather for nine weeks. An objective weather test and six performance assessment items were used to measure achievement. For this group of students, analyses indicated no significant overall effects of treatment on science achievement. A statistically significant effect was found between concept mapping and student achievement among the average students, as measured by combined performance assessment items. The results suggest that the effect of concept mapping on science achievement is not clear, but that lower ability students appear to have better success with concept mapping than higher ability students.

Using Concept Mapping to Aid African American Students' Understanding in Middle Grade Science

This article reports on the results of a nine-week investigation that examined the effectiveness of concept mapping on science achievement of 182 African American middle grade science students, distributed into eight intact earth science classes (by ability levels). Ability level was examined as a covariate on student achievement. For this sample of students, analyses of covariance indicated no significant overall effects of treatment on science achievement. A statistically significant effect was found between concept mapping and student achievement among the average students measured by combined performance assessment items. The results suggest that concept mapping has a positive effect on average (lower) ability level African American science students. Several years ago, science educators recommended major reforms regarding the teaching of science in the nation's public schools (National Research Council, 1996). Science educators issued this request for reform because a growing number of students were graduating from public schools without demonstrating competent scientific literacy enabling them to function in a technological and scientific community. Since the reform measures were initiated, data reported by the National Assessment of Educational Progress (NAEP, 2001) indicated that students in fourth and eighth grades performed better in 2000 than previous years, and overall, American students showed academic improvement in science compared to students in Canada and several European and Asian countries. Although gains were indicated overall, eighth-grade science students did not perform as well as fourth-grade science students. In spite of overall gains, African American eighth-grade science students still lag behind other eighth graders in achieving an adequate level of scientific literacy. This assertion is based on several factors. First, as noted in the NAEP report (2001), African American students lag far behind White and Asian students enrolled in the sciences. Second, these data indicate that fewer African American students take advance science courses in high school. Third, African American students are underrepresented in science majors at the college level (Fouad, 1995; Gates, 2001). Fourth, for most African American students, the only courses available are science classes designed to award credit without expectations to learn (DeBacker & Nelson, 2000; Fouad, 1995; Gates, 2001). This lack of achievement by African American middle school science students is further addressed in The Center for Research on Education Diversity and Excellence (CREDE, 2002) investigation involving middle school students to determine what and how students learn about scientific ideas. This investigation reports that classroom practices designed to engage diverse students (African American and Hispanic) in exploring the potential meaning and functions of their own ideas about scientific phenomena resulted in higher levels of participation and a deeper understanding of scientific phenomena. These ideas from CREDE suggest teaching science via scientific inquiry, which is at the heart of the National Science Education Standards (American Association for the Advancement in Science, 1995; Odom & Kelly, 1998), is a more effective methodology for these students. Since an important part of scientific inquiry involves students communicating an understanding of the evidence collected and an explanation about the evidence, students need to understand the relationships between explanation and evidence derived from investigations (Ruiz-Primo & Shavelson, 1996). Odom and Kelly (1998) stated that concept mapping, an inquiry-oriented and student-centered tool, may be an effective tool to enable African American students to learn science via scientific inquiry. A concept map is a diagram indicating a student's understanding of the interrelationship among concepts. Concepts are arranged in a hierarchy (from general to specific), connected with a label line. …

Mathematics Performance Assessment: Discovering Why Some Items or Rubrics Don’t Measure Up

This study examines the quality of mathematics performance assessment items and scoring rubrics at the middle school level. A total of 53 items on an assessment instrument aligned with key features of mathematics reform such as problem solving, reasoning, and communication were analyzed to identify features of the items and rubrics that tended to be related to how well an item functioned on the assessment instrument. A qualitative analysis of student responses to three items was undertaken to further explore the problematic features. Based on the results of both analyses, scoring rubrics were revised so as to produce a more valid assessment of students’ knowledge of mathematical concepts. The findings suggested five major issues important in the development and use of performance assessments in classrooms: the creation of well-defined criteria that accurately assess understanding of mathematical concepts, the consistency in applying rubric criteria across items, the range of points in the rubric, the mathematical richness of items, and the importance of having sample scored student responses that appropriately illustrate each score level in the rubric.

An Approach That Examines Sources of Misfit to Improve Performance Assessment Items and Rubrics

This article discusses an approach to analyzing performance assessments that identifies potential reasons for misfitting items and uses this information to improve on items and rubrics for these assessments. Specifically, the approach involves identifying psychometric features and qualitative features of items and rubrics that may possibly influence misfit; examining relations between these features and the fit statistic; conducting an analysis of student responses to a sample of misfitting items; and finally, based on the results of the previous analyses, modifying characteristics of the items or rubrics and reexamining fit. A mathematics performance assessment containing 53 constructed-response items scored on a holistic scale from 0 to 4 is used to illustrate the approach. The 2-parameter graded response model (Samejima, 1969) is used to calibrate the data. Implications of this method of data analysis for improving performance assessment items and rubrics are discussed as well as issues and limitations related to the use of the approach.

A SIMULATION STUDY OF THE EFFECT OF RATER DESIGNS ON ABILITY ESTIMATION

ABSTRACTAs more assessment programs move towards the use of constructed response and performance assessment items, the use of human raters to score these items necessarily will increase. Different designs can be created to assign raters to score examinee responses, and in this study some of these designs are evaluated in terms of their impact on the accuracy of examinee ability estimation. As expected, the optimum design where every rater judges every performance by every examinee results in ability estimates with minimum bias and small standard error. As this design is rarely, if ever, practical in real assessment situations, the results for nested and spiral rater designs are of more interest to practitioners. The nested rater designs result in biased ability estimates for the examinees judged by the most extreme raters, but the spiral rater designs examined prove surprisingly robust to rater tendencies.

Performance assessment in IEA studies

The International Association for the Evaluation of Educational Achievement (IEA) has been using performance assessment items in its multi-national studies of educational achievement for nearly thirty years. Performance assessment items have been used in studies in mathematics, science, written composition, French and English as foreign languages, computers in education and reading. The reason for using such items is that the specifications for the tests in each area called for constructed responses on the part of the examinee. This chapter furnishes a number of examples of such items and a discussion of the experience gained from using them. In some cases, the use of such items worked well while in other cases a number of problems were encountered. Some suggestions regarding the use of performance items are offered.

Procedures for Extending Item Bias Detection Techniques to Polytomously Scored Items

The recent increased emphasis on various types of performance assessments raises the question of what is the effect of such assessments on various population subgroups. This study was designed to suggest methodology for several statistical procedures to detect performance assessment items that function differently for two subgroups of examinees. Three different statistics are proposed. Each uses an external criterion (e.g., performance on a multiple-choice test) as a conditioning measure. All three methods being compared are alternative ways of combining data from 2 x k tables. Computer simulations were conducted to study the behavior of these statistics in the detection of items exhibiting varying degrees of differential item functioning. The t statistic-based methods appeared to be somewhat more powerful, and one of these two statistics also demonstrated better control of Type I error under certain conditions.

Peer mentorship program in scientific literacy for rural elementary teachers

Science education faculty at Western Illinois University worked with the local Educational Service Center to develop and deliver programs to regional teachers under the Illinois Science Literacy Project. Involved were 224 teachers and their principals from 40 school districts. Mentor teachers and teacher teams were identified in each school building and were given increasing levels of responsibility for product production (kits, performance assessment items, parent involvement plans, and community resource guides) as the project continued. Data indicated teacher attitudes toward science became more positive throughout the program, as did teacher confidence with respect to teaching via hands-on approaches.