Multiple Matrix Sampling Research Articles

Applicability of multiple matrix sampling to estimating effectiveness of educational programs

When evaluating the effectiveness of an educational program, a sampling strategy known as “multiple matrix sampling” is particularly effective and efficient when the goal of the evaluation is estimating group (as opposed to individual) performance. Although this technique involves sampling both program participants and items and, subsequently, different participants being assessed over different sets of items, the results may be used to estimate those which would have been obtained had all participants been administered all items. The technique is here described in some detail, with advantages and disadvantages discussed, and examples of its application are given.

The Effect of Item Stratification on the Estimation of the Mean and Variance of Universe Scores in Multiple Matrix Sampling

The effect of stratified sampling of items on the estimation of test score distribution parameters by multiple matrix sampling was studied. Item difficulty and/or interitem correlations were the bases of stratification. Various item universes were created by computer simulation and sampled according to several plans. The results indicate that stratification of items does not consistently improve the stability of parameter estimation. The results also show that the variance estimate used in many studies is biased for some item universes when difficulty stratification is used. A variance estimate developed in the current study removes this bias.

Developing a Multiple Matrix Sampling Assessment—a followup report

Developing a Multiple Matrix Sampling Assessment—a followup report

Comprehensive assessment of Spanish language proficiency using multiple matrix sampling techniques

Comprehensive assessment of Spanish language proficiency using multiple matrix sampling techniques

On Estimating Test Variance in Multiple Matrix Sampling

The purpose of this paper is to provide a rederivation of Lord's formula for estimating variance in multiple matrix sampling and to show how Cronbach's coefficient alpha and Spearman-Brown prophecy formula are related in this context.

Using Multiple Matrix Sampling to Assess Health Education Knowledge

Using Multiple Matrix Sampling to Assess Health Education Knowledge

The Use of Multiple Matrix Sampling for Drug Questionnaires

This paper presents a procedure called multiple matrix sampling which simplifies data collection for drug incidence surveys. The technique has been used in achievement testing where the research interest was related to group and not individual differences as with the evaluation of statewide assessment plans. Advantages for the use of multiple matrix sampling with drug surveys are discussed and a practical example is outlined presenting the guidelines for a sampling plan.

Estimating Means Via Multiple Matrix Sampling: a Note On the Effects of Selected Data Base Characteristics

In this paper, Shoemaker's (1972, 1973b) conclusions related to the influence of various data base characteristics (reliability, variability of item difficulty indices, and degree of skewness in the normative distribution) on the standard error of a mean estimated via Multiple Matrix Sampling (MMS) procedures are examined. It is noted that these conclusions are true under fairly restrictive conditions. Also, it is shown that even under these conditions it is more reasonable to attribute changes in the standard error to changes in examinee variability rather than to changes in these other data base characteristics.

Estimating Moments of Universe Scores and Associated Standard Errors in Multiple Matrix Sampling for All Item-Scoring Procedures

Described herein are formulas and computational procedures for estimating the mean and second through fourth central moments of universe scores through multiple matrix sampling. Additionally, procedures are given for approximating the standard error associated with each estimate. All procedures are applicable when items are scored either dichotomously or polychotomously.

The Use of Multiple Matrix Sampling to Approximate Norms Distributions: An Empirical Comparison of two Models

ABSTRACT Multiple matrix sampling (MMS) procedures were utilized to determine the necessary parameters of a Pearson Type I curve. Empirical norms distributions were approximated by both the Type I model and the negative hypergeometric model. Four existing ITED norms distributions, two subtests and two grades, were approximated by the MMS procedures. Two sampling designs for each test-grade combination were studied. Comparison of approximations obtained for the Type I curve and the negative hypergeometric curve supported the use of the Type I curve for determining test score distributions of large populations. (Author) 0

SCRAMBLING CONTENT IN ACHIEVEMENT TESTING: AN APPLICATION OF MULTIPLE MATRIX SAMPLING IN EXPERIMENTAL DESIGN

This study was designed to research the question of scrambling item content in the construction of achievement tests, so that very general implications could be drawn for both examinee and item populations. To achieve this generality, the methodology of multiple matrix sampling was combined with a simple two group experimental design: a random group of 8th graders responded to mathematics, science, social studies, reading, and language arts achievement items organized in a scrambled (random) test format, while another random group responded to the same items organized in a fixed (segregated by subject matter) test format. The results indicated that scrambling cognitive test items has minimal or no effect on mean examinee test performance or on any of the other parameters included in the analysis.

AN EXAMINATION OF THE CONTEXT EFFECT IN ITEM SAMPLING

Item sampling and/or multiple matrix sampling techniques have been recommended for a variety of purposes. For some of these purposes, it must be assumed that examinee performance on a set of items is unaffected by the conditions under which the items are taken (i.e., no context effect exists). In this paper factors that may lead to a context effect among high school students are discussed. The net effect of such factors on examinee scores for an English test and a mathematics test is investigated empirically. For the English test there was little support for the existence of a context effect, However, a definite context effect was found for the mathematics test.

Approximating Standardized Achievement Test Norms With a Theoretical Model

This study was undertaken to answer the following research question: Can standardized achievement test norms be better estimated with a theoretical probability model other than the negative hypergeometric model? Ninety norms distributions from two standardized achievement test batteries (ITBS and ITED) were fitted by six models. The results strongly supported the Pearson Type I model (four moments) as the best-fitting curve. It was concluded that this Type I model could provide a viable alternative to the negative hypergeometric model for use in approximating distributions when moment estimates are obtained from multiple matrix sampling or item sampling procedures.

A NOTE ON ALLOCATING ITEMS TO SUBTESTS IN MULTIPLE MATRIX SAMPLING AND APPROXIMATING STANDARD ERRORS OF ESTIMATE WITH THE JACKKNIFE

Investigated empirically through post mortem item‐examinee sampling were the relative merits of two alternative procedures for allocating items to subtests in multiple matrix sampling and the feasibility of using the jackknife in approximating standard errors of estimate. The results indicate clearly that a partially balanced incomplete block design is preferable to random sampling in allocating items to subtests. The jackknife was found to better approximate standard errors of estimate in the latter item allocation procedure than in the former.

An Analysis of Variance Framework for Matrix Sampling

This paper, a discussion of the methodology of matrix sampling, and the empirical and theoretical research on matrix sampling, attempts to demonstrate the f llowing points: 1. Matrix sampling can be viewed as a simple two factor, random model analysis of variance design, the matrix sampling formulas for estimating the mean and variance being simply the point estimate formulas for estimating components of the underlying linear model. 2. These formulas can be based on the weakest possible set of assumptions, viz., random and independent sampling of examinees and items. No assumptions about the statistical nature of the data need be made. 3. The literature is unclear about what effect the above sampling assumptions have upon matrix sampling in the estimation of the mean and, especially, the variance. 4. Of the three alternative procedures suggested for dealing with negative variance estimates in multiple matrix sampling--equating the negative estimate to zero, Winsorizing the distribution of estimates, or treating all estimates alike regardless of sign--the third procedure appears to be most promising. A simulation study is necessary to determine the shape of the distribution of variance component estimates for matrix sampling as well as the relative efficiency of the three methods for handling negative estimates.