The small effects of non-hierarchical complexity variables on performance.

Abstract

Even when the results show that most of the difficulty in solving problems is explained by the hierarchical complexity of the item, there are still variables that help in small ways in predicting how well items measure difficulty. One must understand what these variables are and take them into consideration when analyzing data from instruments designed to measure the impact of the order of hierarchical complexity of items. This study was designed to test the effect of small variables on task performance. The variables tested were hierarchical complexity, place in order, the number of calculations needs, the size of the numbers, and the causal variable position. Participants were asked to solve problems from task sequences from the logic/mathematics/physical science subdomains. The four instruments used were the algebra, balance beam, infinity and laundry instruments. These instruments were based on the model of hierarchical complexity (MHC). Participants were asked to first complete the laundry task sequence and move to the next task sequence. Items from each instrument were analyzed individually and as a group. A Rasch analysis was performed on all the items from all the instruments. The variables thought to have an effect were coded. The coded variables were then analyzed using stepwise regression. A stepwise regression was used and the small variables were tested with and without hierarchical complexity as a factor. The variables were regressed against the stage score of the items. For all four instruments stepwise regression with hierarchical complexity as one of the variable accounted for about 95% of the variance and the β was greater than 0.9. Stepwise regression with all the other variables except hierarchical complexity accounted for relatively lower variance and β. The results showed that order of hierarchical complexity has a very strong predictive role and accounts for most of the variance. The other variables only made very small contributions.