Reliability Of Students Research Articles

Reliability of Students’ Teaching Practice Scores Using Generalizability Theory

Purpose: The aim was to demonstrate the potency of using Generalisability Theory (GT) over Classical Test Theory (CTT) in reliability estimation. The objectives are to find the major source of error in the Intern Teaching Evaluation Form (ITEF) scores and to identify the optimum number of occasions of rating of teaching practice that would give the most reliable scores. Design/ Methodology/ Approach: A random effects one-facet fully crossed design in which intern (p) was fully crossed with occasion (o) was adopted for the study. In all, 9,082 bachelor’s degree teaching practice triplicate scores for three academic years from 2015/2016 to 2017/2018 were analysed in this study. A univariate generalisability analysis with EduG was performed to analyse data. Findings: The finding for relative interpretation, the scores were strongly reliable. For absolute interpretation, the scores were moderate to strongly dependable. The major source of error in the ITEF scores was the p x o interaction combined with unidentified sources. Research Limitation/Implications: A firm conclusion on one major source of error being a single facet in the ITEF scores could not be made. Practical Implication: The level of reliability and the optimum scoring design of the ITEF established by this study will aid in selecting the ITEF by teacher training institutions in Ghana for evaluation of teaching practice to train pre-service teachers for effective curriculum implementation to achieve national educational aims. Social Implication: Reliable evaluation data can inform policy decisions related to teacher certification, professional development, and educational standards. Policymakers can use this data to develop evidence-based policies that support teacher quality and student achievement. Originality/ Value: It has unearthed the psychometric properties and the ideal scoring design of the ITEF which were hitherto unknown.

Eksplorasi Validitas dan Reliabilitas Soal Pemahaman Konsep Dalam Asesmen Pembelajaran

Students’ understanding of concept can be measured and assessed using mathematical concept understanding test instruments. The instrument to be used must be prepared to meet the criteria for a good instrument. Therefore, the aim of this research is to determine the validity and reliability of students' conceptual understanding questions in learning assessments. This type of research is qualitative research using descriptive methods. Data was obtained through tests and interviews with mathematics subject teachers. The subjects in this research were class IX students at MTsN 2 Kerinci, totaling 22 students in 1 class. Based on the results of validity calculations using the Pearson Product Moment formula and t test, as well as calculations of question reliability using the Cronbach's Alpha formula, it was found that each question item was valid and reliable, so this test instrument can be used to measure students' ability to understand mathematical concepts.

Instrumental Analysis of Student Perceptions of Chemistry Learning with the STEM Approach at the end of the Covid-19 Pandemic using Second Order Confirmatory Factor Analysis

This study aims to prove the construct validity and reliability of students' perceptions of chemistry learning instruments using the STEM approach using Second Order Confirmatory Factor Analysis. The sample in this study was 455 public high school students spread across South Bengkulu district, Bengkulu Province. The instrument developed is a questionnaire instrument using a Likert scale which consists of four dimensions, namely the scientific dimension (DS), the mathematical dimension (DM), the engineering and technology dimension (DTT) and the STEM dimension (DST). The results of the study show that the four dimensions have good validity and are still acceptable because the loading factor value is above 0.40. In proving reliability using Cronbach's Alpha, a value of > 0.6 (0.855) was obtained so that it was declared reliable and in further analysis proving construct reliability using Composite Reliability (CR) showed that all dimensions of student perceptions had good construct reliability as well

Analysis of Items in the Mid-Semester Assessment of Science Class VII SMP Yapis Manokwari

The purpose of this study was to analyze the test items in the midterm assessment using the Rasch model in terms of validity, reliability, level of difficulty, discriminating power, and the effectiveness of distractors in class VII science questions at SMP Yapis Manokwari. This research is a quantitative research that uses descriptive quantitative methods with data collection techniques using documentation. Respondent data were 60 students with purposive sampling. The items used in the Mid Semester Assessment are 40 questions. From the validity test, it was found that the suitability level of the items was 35 questions that were fit and 5 questions that were not fit. Student reliability 0.81 means good, item reliability item 0.92 means very good and reliability between students with item 0.83 means special. The level of difficulty of the questions is in the medium category, which means that the questions are good. The discriminating power of the items obtained is that 2 items (5%) are in the very good category, 30 items (75%) are in the good category, 4 items (10%) are in the sufficient category, and 4 items (10%) are bad category, from the items it has succeeded in identifying the distinguishing power of students divided into 3 groups in the category of above average ability, average ability, and below average ability. The effectiveness of the distractor of the 40 questions in the test generally functions well. The conclusion is that the analysis of the items using the Rasch model in the midterm assessment is stated to be valid, reliable, the level of difficulty of the items is very good, the discriminating power is very good, and the effectiveness of the distractors obtained is in the good category.

Весовой рост и убойные качества чистопородных и помесных баранчиков

Aim. Comparison of the productive qualities of purebred lambs of the Edilbay breed and crossbred animals obtained from crossing rams of the Dorper breed with ewes of the Edilbay breed. Materials and Methods. Scientific and economic experience was carried out in the conditions of the M.G. Shangeraev farm (Chernoyarsky district, Astrakhan region) on 60 heads of lambs of the Edilbay breed and Dorper-Edilbay crossbreds aged from birth to 4 months of age. Live weight of lambs was determined by individual weighing on electronic scales with an accuracy of 0.5 kg in the morning before feeding, pre-slaughter weight – after fasting for 24 hours. Meat productivity was determined by the results of the control slaughter of 3 typical lambs for each group according to the method of All-Russian Research Institute of Beef Cattle Breeding (1984). The digital material of the research was processed by the methods of variation statistics according to Plokhinsky N.A. (1969) on a PC using Microsoft Office software and determining Student's reliability criterion. Results. Under the identical growing conditions up to 4 months of age, the average live weight of crossbred lambs was 43.6 kg, which is 3.1 kg or 7.65% higher compared to purebred edilbays. Feed costs per 1 kg of live weight gain in crossbred animals were lower by 0.34 energy feed units and 36.7 g of digestible protein. The superiority of crossbred lambs over purebred animals was 2.33 kg or 11.34% in slaughter weight, and 1.68% in slaughter yield. Conclusion. The production of young lamb from purebred and crossbred animals is expedient. The best indicators of live weight and slaughter qualities had crossbred Dorper x Edilbayevsky lambs with relatively low costs of energy feed units and digestible protein per 1 kg of live weight gain.

Validity and reliability to instruments of student’s initial competence for development of physics learning games integrated with scientific literacy

Use of monotonous physics learning media makes online learning methods considered less effective in the 4.0 revolution era. Physics learning games developed must be in accordance with the needs and demand 4.0 revolution. Therefore, it is necessary to analyze the initial conditions of high school students in the city of Padang with one of the instruments being the initial competency instrument. The purpose of this study was to determine the validity and reliability of students’ initial competence instruments for the development of physics learning games containing scientific literacy. This type of research is descriptive research. Samples were taken using volunteer sampling technique. The number of instruments being tested is 35 items/questions. The instrument trial has been carried out in class X SMA Negeri 8 Padang with 44 students. The instrument trials were analyzed using SPSS and excel applications. From the results of the analysis that has been carried out, there are 16 valid and reliable questions, and 4 questions have been revised. Thus, some of the students’ initial competency instruments for the development of physics learning games containing scientific literacy are valid and moderate reliable so that they can be used to determine students’ initial competencies.

The reliability of students’ earnings expectations

The elicitation of subjective expectations in surveys has gained significant interest in economics. Students’ earnings expectations have been widely used to model school and occupational choices, but quantification of the reliability of this variable has not yet been done. To what extent are earnings expectations affected by random measurement error? To assess this issue, in this paper we use different waves of a survey carried out in a Spanish university eliciting the earnings expectations of economics students. A test-retest method is applied with different time spans between the first wave and subsequent repetitions of the survey: two weeks and two and a half months later. A significant number of students declared large differences in earnings expectations, even in the case where the time span between surveys was short. Apart from the reliability, we also provide an estimate of the measurement error variance. With this information we show how a sensitivity analysis can be performed in external earnings expectations data research.

Does Student-Submission Allocation Affect Peer Assessment Accuracy?

Peer assessment in education has pedagogical benefits and is a promising method for grading a large number of submissions. At the same time, student reliability has been regarded as a problem; consequently, various methods of estimating highly reliable grades from scores given by multiple students have been proposed. Under most of the existing methods, a nonadaptive allocation pattern, which performs allocation in advance, is assumed. In this study, we analyze the effect of student-submission allocation on score estimation in peer assessment under a nonadaptive allocation setting. We examine three types of nonadaptive allocation methods, random allocation, circular allocation and group allocation, which are considered the commonly used approaches among the existing nonadaptive peer assessment methods. Through simulation experiments, we show that circular allocation and group allocation tend to yield lower accuracy than random allocation. Then, we utilize this result to improve the existing adaptive allocation method, which performs allocation and assessment in parallel and tends to make similar allocation result to circular allocation. We propose the method to replace part of the allocation with random allocation, and show that the method is effective through experiments.

Training speech pathology students in the perceptual evaluation of speech in patients with cleft palate: Reliability results and the students' perspective

IntroductionListener training is necessary to achieve agreement between perceptual ratings. However, evidence is limited regarding the effect of a training in the perceptual evaluation of cleft palate speech on the reliability of students in speech pathology. The aim of this study was to evaluate the short-term and longer-term effect of a training using the Belgian Dutch outcome tool for cleft palate speech on reliability and the students' experiences. MethodsA 2-h training was provided during a specialist course for 31 first year master students in speech pathology. Information regarding listener characteristics, speech documentation and speech analysis (speech parameters and rating scales) was discussed. This last part focused on the Belgian Dutch outcome tool for speech in patients with cleft palate, including reference samples and consensus listening exercises. Pre training (T1) and post-training reliability (immediately following the training (T2) and one month later (T3)) were collected for the speech variables speech understandability, speech acceptability, hypernasality, hyponasality, nasal emission and nasal turbulence that were rated by means of ordinal scales. Inter-rater reliability was determined with interclass correlation coefficients whereas percentages of agreement were calculated for the intra-rater reliability. ResultsQualitative inter-rater ICC categories improved for hyponasality and speech acceptability at T2 and T3, whereas the inter-rater reliability of the other variables remained stable regarding their interpreted categories. All ICC values quantitatively improved at T2, except hypernasality and nasal emission. At T3, three of those variables showed a small drop one month later. Regarding intra-rater reliability, a significant decrease for hypernasality and nasal turbulence was found at T2, whereas a significant increase was observed for nasal turbulence and speech acceptability at T3. All variables, except hypernasality, had improved intra-rater reliability at T3 in comparison with T1. Students experienced fewer difficulties with most variables after training. Evaluation of nasal emission and nasal turbulence remained however difficult and required, in the students’ opinion, more attention. Discussion & conclusionOverall a positive training effect was noted, especially regarding the inter-rater reliability and the students’ confidence in perceptual ratings. However, it remains a challenge to achieve a positive training effect for all target variables that is maintained over-time.

Modular Classes during Covid-19 Pandemic: The Emerging Opportunities, Challenges and Difficulties

The study sought to find out the effects of the Modular classes from the parents, teachers and students. There are numerous effects of modular learning modality on students? learning acquisition and social behaviour. Some students became busy to other works instead of focusing on the modules. The study also brought out the emerging opportunities, challenges and difficulties of the modular classes. The result revealed that there are opportunities that people enjoyed during the modular classes. Some of the opportunities are (1) parent-children relationship became even closer; (2) more time to study and research; (3) ensures life-work balance and (4) parents are able to focus on their children?s studies. The respondents enumerated the challenges and difficulties they faced during modular classes. They emphasized the importance of cooperation among the students, parents and teachers. They pointed out that lack of direct supervision of the teachers is the greatest challenge. Validity and reliability of students? answers are questionable. Another challenge is the weakening students? learning outcomes and ? worse ?students became addict with computer games. Parents, teachers and students ? the primary stakeholders ? recommended the coming back to face-to-face classes because modular classes really affect students? learning. Key words: Modular Classes; challenges and difficulties; Direct Supervision; Primary Stakeholders; Emerging Opportunities

Validity and Reliability of Students’ Mathematical Process Rubric (Prom3) based on many-Facet Rasch Model (MRFM).

Validity and Reliability of Students’ Mathematical Process Rubric (Prom3) based on many-Facet Rasch Model (MRFM).

Mathematics Make Students Confused and Anxious: a Comparisons Between Australia, Indonesia, and Singapore in The 2015 Timss

The main purpose of this study is to find out the perceptions about Mathematics that confuse and anxious 2015 TIMSS participants in Australia, Indonesia, and Singapre. The data in this study is the 2015 TIMSS data that has been released to the public and has available for development purpose research. The research method used is quantitative method. The population of this study is students' perceptions of Mathematics subject that confuse and worry students aged 15 years in three countries who are studying in 2015, while the sample is 15-year-old students’ perceptions in these countries. The data in this study were in the form of questionnaires which were answered by 16599 students from these three countries. Based on the results of the analysis of variance, there is a significant difference (P <0,000) on students' perceptions about mathematics subject that confuse and worry students in these three countries (Sig. 0,000). The reliability of students' abilities is the same, Australia = 0.74, Indonesia = 0.75; and Singpura = 0.75. The conclusion is most students in these three countries stated "disagree a lot" about mathematics make students confuse and worry.

Online examination session: teachers’ and students’ satisfaction level

This paper is aimed at defining the satisfaction level of students and teachers with online examination session. To implement the study, we conducted online surveys for 2nd year bachelor students and 1st year master students (N=97) and online interviews with teachers (N=12). The analysis of received data indicated that students felt more satisfied with online examination session than teachers due to a number of reasons. Firstly, elder teachers faced technical problems on some steps of examination session. Secondly, teachers worried about the reliability of students’ testing results as they had access to any recourse or material and due to bad connection sometimes it was hard to hear or see students. Thus, some problems were revealed despite the general positive attitude to online examination session from students.

SECONDARY STUDENTS’ VARK SENSORY PREFERENCES IN SCIENCE LEARNING: ARE THEY RELIABLE?

Students’ sensory preferences (SP), that is, the ways or channels they prefer to receive the information, have been proposed as personal factors influencing science learning. Some studies relate improvement in learning to the adequacy of instructional resources to individual SP. A widely used instrument to determine SP is the VARK questionnaire. However, the individual SP has to be reliable to be considered useful to improve instruction. There are not enough studies on the reliability of this questionnaire. This study aimed to assess the reliability (temporal stability) of SP, as measured by the VARK instrument. Also, the reliability of the student’s subjective perception of his/her personal SP was evaluated. A mixed methodology was used, combining techniques for obtaining both quantitative and qualitative information. The sample was made of 582 male and female students, from 7th to 11th grades. The questionnaire was administered twice to a subsample of students, with a few months’ delay. Besides, a small group of students was interviewed. In the analysis of the results, using chi-square tests (X2) and a repeated measure MANCOVA, no massive permanent changes in the group distribution of SP were found along with secondary education. This would be useful for teachers to adapt learning materials to their students’ preferences. The individual SP was stable enough under a permissive criterion, but not when a stricter standard was used. Besides, the reliability of students’ self-perception of SP was low.

KESAHAN DAN KEBOLEHPERCAYAAN INSTRUMEN PENTAKSIRAN KEMAHIRAN INSANIAH MURID (IPKIM) MATA PELAJARAN REKACIPTA

This article discusses the validity and reliability of students’ soft skills assessment instrument of the invention subject. All items are built based on the adaptation of several theories and models of soft skills and literature studies that have been obtained locally or from abroad. The result of this study has submitted eight important constructs in implementing the assessment of secondary school student’s skills through communication skills, technology usage skills, skills in planning and organizing activities, skills of working with other people and in groups, problem-solving skills, management skills, choosing and analyzing information, skills of using mathematical ideas and techniques, and the skills of cultural understanding. Expert validity involves the lecturers of local universities, teacher education institutes, and teachers who have experienced in education for more than 10 years. A total of five experts were involved as a validity assessor for content and five panels as a practice expert to determine the item validity from the aspects of wordings, language register, and sentences in each item built. In addition, the quantitative measurement method through Content Validity Ratio (CVR) has been used to evaluate the items whether it is necessary to keep or to be modified. The findings found that all the items built have fulfilled the measurement features of an instrument and can be used as a tool for measuring the soft skills of students in secondary schools.

PENGEMBANGAN PERANGKAT PEMBELAJARAN MODEL GROUP INVESTIGATION DENGAN ADVANCE ORGANIZER UNTUK MENINGKATKAN HASIL BELAJAR DAN KETERAMPILAN PEMECAHAN MASALAH PADA SISWA SMP

This research aims to develop a learning material of group investigation model with advance organizer to improve learning output and student’s problem solving skills in SMPN 1 Tanjungbumi. This study would be expected to be applied using the several assessing of instruments. The experiment design of the material resources used one group pretest-posttest design. The assessing of instruments consist of lesson plan, BAS, worksheet, observing activity, test (product, process, and psychomotor), and questionnaire.
 The validity result showed that lesson plan, BAS, and LKS had in feasible category, LP product and LP process had in feasible category. The assessment of BAS and LKS showed that BAS and LKS had in understandable material by students. Learning feasibility of group investigation model with advance organizer is in excellent category. Reliability of students’ activity observation from the lesson plan with the percentage 92,32%. The completeness of product’s indicator with the percentage 83,33%, and the completeness of process’ indicator (problem solving skills) with the percentage 73,33%. The completeness of the individual output product with the percentage 93,33%, the completeness of the individual output process (problem solving skills) with the percentage 93,33%, the completeness of the individual output psikomotor with the percentage 100%, and the completeness of the individual output afektif with the percentage 100%. Group investigation model with advance organizer got positive respond from the students. The difficulties found during this study was group investigation model with advance organizer was a new thing for the students so the teacher should give information about group investigation model with advance organizer for the students.

Human-Machine Hybrid Peer Grading in SPOCs

Peer grading, allowing students/peers to evaluate others’ assignments, offers a promising solution for scaling evaluation and learning to massive open online courses (MOOCs) and small private online courses (SPOCs). In the environment of MOOCs, due to the varied skill levels and attitudes of online students, it is not easy for the students to present fair and accurate scores for their peers’ assignments. Recently, statistical models have been proposed to improve the fairness and accuracy of peer grading, and these models have achieved good performance in MOOCs. However, our experiments demonstrate that these models fail to deliver accurate inferences in the SPOC scenario because affinity among students may seriously affect the objectivity and reliability of students in the peer-assessment process. To address this problem in SPOCs, this paper proposes a human-machine hybrid peer-grading framework, in which an CNN(Convolutional Neural Networks)-based automated grader works as a filter to ensure reasonable peer scores before Bayesian models are utilized to infer the true scores. This framework can significantly eliminate the severely biased scores of undutiful students and, thus, improve the accuracy of the true-score estimation of the Bayesian peer-grading models. Both the simulated and actual peer-grading datasets in our experiments demonstrate the effectiveness of this new framework for SPOCs.

Negative emotions as predictors of behavioral outcomes in middle school physical education

Discrete emotions provide a lens for understanding students’ subjective experiences in physical education (PE). Enjoyment of PE is one discrete emotion that receives a great deal of attention because it promotes students’ interest and engagement. However, the implications of students’ negative emotions toward PE remain less clear. Therefore, the purpose of this study was to examine relationships between students’ negative emotions including shame, boredom, and anger in PE and self-reported disruptive behavior, leisure time moderate-to-vigorous physical activity (MVPA), and sedentary time. Middle school students ( N = 401) completed surveys on all study variables seven weeks apart. Confirmatory factor analysis supported the validity and reliability of students’ reports of negative emotions and behaviors across two time points. A cross-lagged design using structural equation modeling tested relationships between emotions and behavioral outcomes. Shame related to higher reports of future disruptive and sedentary behaviors and lower reports of leisure time MVPA. Boredom also related to lower reports of future leisure time MVPA. Lastly, anger related to higher reports of future disruptive behavior but lower reports of sedentary behavior. Each negative emotion produced unique relationship patterns that generally underscored undesirable student behaviors. Evaluating discrete emotions can highlight information regarding students’ action tendencies. Understanding relationships between negative emotions and their resulting behaviors may help inform interventions and best teacher practices in PE. Overall, physical education teachers need to be aware of students’ negative emotions and make pedagogical and content-related decisions to improve students’ subjective experiences.

Validity and Reliability of Students’ Science and Technology Culture Instrument (BST-M) using Rasch Measurement Model

Students’ Science and Technology Culture Instrument (Budaya Sains dan Teknologi Murid, BSTM) was developed to measure the level of Science and Technology Culture among the students. A total of 800 Form Two students from several schools in Seremban, Negeri Sembilan have been selected as a sample study. This study was conducted to determine the reliability and construct validity of the instrument using Rasch Model via Winstep 3.73 software. The constructs consist of Value and Perception on Science and Technology (B); Perception on Science and Technology (C); Scientific Attitude and Common Practice (D); Scientific Mind Habits and Environmental Concerns (E); and Personality Traits (F). From the analysis, it is found that PTMEA Corr has a positive value, in which items are able to differentiate the capabilities of the respondents. Besides, the results of infit and outfit mean square are ranged between 0.60 and 1.4. The quality of items is high because the reliability value is also high. In addition, the separation of item and person is at the acceptable range. However, statistical data shows that 8 out of 110 items need to be modified.

PERBEDAAN MOTIVASI BELAJAR SISWA BERJENIS KELAMIN PEREMPUAN DAN LAKI-LAKI SMK SWASTA BANDUNG

This Research is triggered by low of students’ learning motivation. The purpose of This Research are to: 1) describe students’ learning motivation of female gender, 2) describe students’ learning motivation of male gender, and 3) Differences students’ learning motivation of female gender and male gender. This research used a descriptive differencial quantitative method. The population of this research were X, XI, and XII grade of SMK Swasta Bandung 1 & 2 which totally 240 students. The sample of this research were 150 students, that selected by using proportional stratified random sampling technique. In this research, the researcher used scale as the instrument. The result of validity and reliability of students’ learning motivation state that the instrument of this research was valid and reliable. Data were analyzed with descriptive statistics and t-test. The finding of this research are: 1) on avarage students’ learning motivation of female gender are in high category, 2) on avarage students’ learning motivation of male gender are in high category, 3) there were significant differences between students’ learning motivation of female gender and male gender. Keyword : Learning Motivation, Male, Female