Learning Object Review Instrument Research Articles

Validation of a Learning Object Review Instrument: Relationship between Ratings of Learning Objects and Actual Learning Outcomes

Growing interest in learning objects (LOs) as a means of developing learning materials is leading to mainline LO evaluation methodologies using review instruments, such as evaluation rubrics, to suit various practical purposes. Such evaluation tools give evidence about the design and the value of the LOs, and studies performed with actual users can provide data against which these expectations of the effects of LOs on student achievements in practice can be set. This study presents a validation of a learning object review instrument (LORI) with student users (n=507) of twenty-four LOs used in K-12 environments. T he data collected through pre- and post-tests, teachers’ and students’ usability questionnaires, and the LORI revealed some interactions between those variables. However, the LORI ratings, and the usability assessments did not correlate with the learning gains of students. Some implications of these findings are discussed. Ke ywords: Learning objects, learning outcome, LORI, validation.

Pre-service Teachers' Learning Object Development: A Case Study in K-12 Setting

Teachers use variety of media to support students’ learning. Though there are many new tools and settlements in learning technologies and their specifications, there has been a fierce debate over learning objects and their development. This research aimed to examine the effect of preservice teachers’ experience in information and communication technology (ICT) use on their learning object development through (1) inspecting type, number and organization of their learning object (LO) components authored in a non-commercial system, (2) evaluating quality of their learning objects using the Learning Object Review Instrument (Nesbitt & Li, 2004), and (3) studying relationship between the components, quality and subject matter of their LOs. Seventysix pre-service teachers were asked to design and develop LOs in a non-commercial learning content development system. Analysis of their learning objects and a usability test data showed that the participants found the system easy to use in general. Novice and experienced information technology users were able to develop learning objects similar in size and features. The analysis showed meaningful correlation between the use of some of the learning object components. There seems to be significant relations between the quality of the LOs measured with the LORI items and some elements of the developed LOs, however, the novice and experienced groups’ LOs did differ neither in overall ratings nor at nine individual items of the LORI. The study suggested some further work for developing quality LOs through collaborative learning resource authoring.

Liberating learning object design from the learning style of student instructional designers

Learning objects are a new form of learning resource, and the design of these digital environments has many facets. To investigate senior instructional design students' use of reflection tools in designing learning objects, a series of studies was conducted using the Reflective Action Instructional Design and Learning Object Review Instrument tools. Analysis revealed that most participants found the reflection questions useful in design but also that intensive use of the tools is needed to free learning object design from the personal learning traits of the designers.

Learning Object Evaluation: Computer-Mediated Collaboration And Inter-Rater Reliability

Learning objects offer increased ability to share learning resources so that system-wide production costs can be reduced. But how can users select from a set of similar learning objects in a repository and be assured of quality? This article reviews recent developments in the establishment of learning object repositories and metadata standards, and presents a formative reliability analysis of an online, collaborative method for evaluating quality of learning objects. The method uses a 10-item Learning Object Review Instrument (LORI) within a Convergent Participation evaluation model that brings together instructional designers, media developers, and instructors. The inter-rater reliability analysis of 12 raters evaluating eight learning objects identified specific items in LORI that require further development. Overall, the collaborative process substantially increased the reliability and validity of aggregate learning object ratings. The study concludes with specific recommendations including changes to LORI items, a rater training process, and requirements for selecting an evaluation team.