Mental Effort Ratings Research Articles

Cognitive load of critical thinking strategies

Argument–counterargument integration (Nussbaum, 2008) refers to the process of evaluating, refuting, and synthesizing arguments on two sides of an issue when creating justification for an overall conclusion. This study compared the cognitive load of two critical thinking strategies related to argument–counterargument integration: (a) constructing design claims that minimize disadvantages of an alternative, and (b) weighing refutations (which weaken an argument by arguing that there are more important values at stake). College students (N = 285) first completed the Need for Cognition (NFC) scale and were then presented with materials summarizing arguments and counterarguments on the topic of grading class participation. Participants completed a small, integrative essay justifying a stand on the issue, and completed the Mental Effort Rating Scale (Paas, 1992). Participants who generated complex weighing refutations reported more mental effort than those constructing complex design claims (and the control group), with a stronger relationship with those high in NFC. The need to coordinate disparate elements in working memory may explain the higher load associated with constructing weighing refutations. Students may need more (and different types of) scaffolding in using this strategy than when constructing a design claim, which is a more sequential process.

Measuring cognitive load with subjective rating scales during problem solving: differences between immediate and delayed ratings

Subjective cognitive load (CL) rating scales are widely used in educational research. However, there are still some open questions regarding the point of time at which such scales should be applied. Whereas some studies apply rating scales directly after each step or task and use an average of these ratings, others assess CL only once after the whole learning or problem-solving phase. To investigate if these two approaches are comparable indicators of experienced CL, two experiments were conducted, in which 168 and 107 teacher education university students, respectively, worked through a sequence of six problems. CL was assessed by means of subjective ratings of mental effort and perceived task difficulty after each problem and after the whole process. Results showed that the delayed ratings of both effort and difficulty were significantly higher than the average of the six ratings made during problem solving. In addition, the problems we assumed to be of higher complexity seemed to be the best predictors for the delayed ratings. Interestingly, for ratings of affective variables, such as interest and motivation, the delayed rating did not differ from the average of immediate ratings.

Cognitive Load of Learner Control: Extraneous or Germane Load?

Computer-based learning environments become more tailored when learners can exert control over one or more parts of the learning process. Learner control (LC) demands additional efforts of learners because, in addition to learning, they also have to monitor that learning. As a consequence, LC may cause additional cognitive load and even cognitive overload. The central question in this study is what type of cognitive load is induced by LC and whether the experienced load is related to learning outcomes. For this study, half of the students had control over task selection, while the other half had not. Within each condition, students were assigned to a single treatment, with the primary task to solely focus on the learning content, and a dual treatment, comprising a primary task and a secondary task. The results indicate that LC did not impose higher cognitive load as measured by secondary task scores and mental effort ratings.

Using a Personal Response System To Map Cognitive Efficiency and Gain Insight into a Proposed Learning Progression in Preparatory Chemistry

Personal response systems (“clickers”) have become an important means for instructors to gauge student learning in large lecture classes. In addition to measuring students’ performance on a particular question, requesting a measure of mental effort from students allows for richer data concerning student learning. This information can be provided to an individual student for a better gauge of his or her understanding of specific content, allowing that student to target his or her studies for subsequent assessments. Mental effort information also provides instructors with an opportunity to consider instructional interventions based on more than just performance. Lastly, using measures of both performance and reported mental effort can provide a better understanding of the learning progression. This paper describes using performance and mental effort data on review items given regularly in lecture with clickers in order to train the students to use the study reports and the mental effort rating system. The combination of high or low clicker performance and high or low mental effort was examined with exam performance and final grade as well as performance within specific content areas. A progression of high–low clicker performance and high–low mental effort and performance on assessments or in the course is discussed.

A Valid and Reliable Instrument for Cognitive Complexity Rating Assignment of Chemistry Exam Items

The design and use of a valid and reliable instrument for the assignment of cognitive complexity ratings to chemistry exam items is described in this paper. Use of such an instrument provides a simple method to quantify the cognitive demands of chemistry exam items. Instrument validity was established in two different ways: statistically significant correlations between expert-based cognitive complexity ratings and student performance (as measured through statistical difficulty of items), and statistically significant correlations between expert-based cognitive complexity ratings and student mental effort ratings. Key benefits associated with instrument use include an enhanced understanding of the cognitive complexity of chemistry assessment tasks and as a means for characterizing exam content for the measurement of cognitive development.

Mobile phone use while cycling: Incidence and effects on behaviour and safety

The effects of mobile phone use on cycling behaviour were studied. In study 1, the prevalence of mobile phone use while cycling was assessed. In Groningen 2.2% of cyclists were observed talking on their phone and 0.6% were text messaging or entering a phone number. In study 2, accident-involved cyclists responded to a questionnaire. Only 0.5% stated that they were using their phone at the time of the accident. In study 3, participants used a phone while cycling. The content of the conversation was manipulated and participants also had to enter a text message. Data were compared with just cycling and cycling while listening to music. Telephoning coincided with reduced speed, reduced peripheral vision performance and increased risk and mental effort ratings. Text messaging had the largest negative impact on cycling performance. Higher mental workload and lower speed may account for the relatively low number of people calling involved in accidents. Statement of Relevance: Although perhaps mainly restricted to flat countries with a large proportion of cyclists, mobile phone use while cycling has increased and may be a threat to traffic safety, similar to phone use while driving a car. In this study, the extent of the problem was assessed by observing the proportion of cyclists using mobile phones, sending questionnaires to accident-involved cyclists and an experimental study was conducted on the effects of mobile phone use while cycling.

The Effects of Problem Based Learning on Mathematics Performance and Affective Attributes in Learning Statistics at Form Four Secondary Level

The challenge of globalisation today requires students to acquire problem solving and communication skills besides good procedural and conceptual knowledge of mathematics. This study was conducted to explore the effects of Problem Based Learning (PBL) as an alternative instructional strategy in the teaching and learning of mathematics effectively in Malaysian secondary schools. Previous research had discovered that that PBL enhances student's team work, problem solving skills and communication skills. In addition, interest in the subject significantly improved. A quasi experimental study with non-equivalent control group posttest only design was conducted to investigate the effects of PBL on form four Malaysian student's mathematics performance, instructional efficiency and affective attributes. The experiment was carried out for six weeks involving 53 Form Four students randomly selected from the district of Port Dickson. The experimental (PBL) group (n=29) were exposed to the PBL instruction whereas the control (CT) group (n=24) were taught conventionally. There were five instruments used in this study namely, mathematical learning worksheets, a posttest, Paas Mental Effort Rating Scale, a questionnaire on perception towards group work, interest in mathematics and perception towards mathematics learning experience and a rubric evaluating student's effective use of Polya's problem solving procedures, mathematical communication and teamwork. The data were analysed using analysis of covariance (ANCOVA) and independent t-test. student's response to the questionnaire and researcher's observations based on the rubric were described using means, standard deviations and percentages. The findings of the study indicated that PBL is just as efficient as the conventional teaching strategy in enhancing Form Four student's mathematics performance. Even though both groups of students showed positive perception towards group work, interest in mathematics and perception towards the learning experience they went through, the PBL group used the Polya's problem solving procedures more effectively, displayed better mathematical communication skills and showed stronger teamwork compared to the CT group.

Influence of Text Type, Topic Familiarity, and Stuttering Frequency on Listener Recall, Comprehension, and Mental Effort

To determine how text type, topic familiarity, and stuttering frequency influence listener recall, comprehension, and perceived mental effort. Sixty adults listened to familiar and unfamiliar narrative and expository texts produced with 0%, 5%, 10%, and 15% stuttering. Participants listened to 4 experimental text samples at only 1 stuttering frequency. After hearing the text samples, each listener performed a free recall task, answered cued recall questions, answered story comprehension questions, and rated their perceived mental effort. Free and cued recall as well as story comprehension scores were higher for narrative than for expository texts. Free and cued recall scores were better for familiar than for unfamiliar stories, although topic familiarity did not affect story comprehension scores. Samples with all levels of stuttering resulted in higher mental effort ratings for both text types and topic familiarities. Stuttering has a greater influence on listener recall and comprehension for narrative than for expository texts. Topic familiarity affects free and cued recall but has no influence on story comprehension. Regardless of the amount of stuttering, mental effort was high for both text types and levels of familiarity.

A comparison of three measures of cognitive load: Evidence for separable measures of intrinsic, extraneous, and germane load.

Understanding how to measure cognitive load is a fundamental challenge for cognitive load theory. In 2 experiments, 155 college students (ages = 17 to 22; 49 men and 106 women) with low domain knowledge learned from a multimedia lesson on electric motors. At 8 points during learning, their cognitive load was measured via self-report scales (mental effort ratings) and response time to a secondary visual monitoring task, and they completed a difficulty rating scale at the end of the lesson. Correlations among the three measures were generally low. Analyses of variance indicated that the response time measure was most sensitive to manipulations of extraneous processing (created by adding redundant text), effort ratings were most sensitive to manipulations of intrinsic processing (created by sentence complexity), and difficulty ratings were most sensitive to indications of germane processing (reflected by transfer test performance). Results are consistent with a triarchic theory of cognitive load in which different aspects of cognitive load may be tapped by different measures of cognitive load.

Instructional Efficiency of Mathematical Learning Using Geometer’s Sketchpad and Graphing Calculator: Technological Tools versus Traditional Chalk and Talk

In this study, two technological tools in teaching and learning mathematics namely Geometer’s Sketchpad and graphing calculator were investigated. The purpose of this research was to compare instructional efficiency of utilizing Geometer’s Sketchpad and graphing calculator versus traditional approach for a secondary level topic ‘Quadratic Function’. A total of 134 students aged 16-17 years participated in this study. The students were randomly assigned to three groups and undergo three different treatments. Four phases were conducted: 1)Introduction to Software, 2) Induction to Quadratic Functions, 3) Integrated teaching and learning using Software, 4) Testing using Achievement Test and the Paas Mental Effort Rating Scale. The data were analyzed using ANOVA and post-hoc analyses. The findings from this study showed that there is no significant difference in performance between the technology-assisted instruction as compared to conventional chalk and talk instruction. However findings indicated that students in the geometer sketchpad group and the graphing calculator group performed better than students in traditional group in their procedural knowledge. Using Paas Mental Effort Rating Scale (2004) however indicated that instructional efficiency of using graphing calculator condition is significantly efficient [F (2, 131) = 9.888; p = 0.000< 0.05] compared to the geometer sketchpad and conventional chalk and talk condition. Graphing calculator condition thus far imposed relative low mental effort with high performance. Geometer sketchpad condition imposed high mental effort with low performance. Hence the use of technology within short period of time induce high mental effort in solving traditional assessment questions thus affects performance. This findings provide empirical evidence and confirmed earlier studies about the advantages of using graphic calculator as facilitative tool for improving students’ performance and provide explanation for the benefit of graphic calculator as a tool of learning from the cognitive load theory. However the use Geometer Sketchpad as facilitative technological tools has yet to be explored.

Problem format and presentation sequence: effects on learning and mental effort among US high school students

AbstractThe study involving 209 high school students in an instructional program about comma rules was an attempt to test the concepts of cognitive load theory in a real classroom setting. It investigated the effects of (a) cued versus conventional problems and (b) blocked versus random problem presentation sequence during practice on the performance and perceived mental effort of lower‐ and higher‐achieving students. The researchers anticipated that cued problems would reduce learner extraneous cognitive load while random problem presentation sequence would increase germane cognitive load. The researchers also predicted that expected effects may not hold true for higher‐achieving students. The treatments yielded a near‐significant interaction effect for mental effort ratings indicating that conventional problems may have appeared harder to lower‐achieving students while cued problems may have appeared harder to higher‐achieving students. There were no notable performance differences. Issues related to researching cognitive load theory effects in a real classroom setting are discussed. Copyright © 2006 John Wiley &amp; Sons, Ltd.

A Tactile Cockpit Instrument to Support Altitude Control

We tested the effect of a tactile torso display on keeping an instructed altitude during low level flight. The tactile instrument consisted of 64 vibrating elements attached to the torso, shoulders and thighs of the pilot. In a helicopter simulator, 12 student pilots flew under different conditions of vision (full vision and night vision) and the tactile instrument (none, a simple version and a complex version). The simple version presented the direction of the desired altitude. The complex version added the current motion direction. The participants performed an additional cognitive task during half of each scenario. We analyzed performance and subjective mental effort ratings. The results showed that the tactile instrument halved the altitude error without effecting the mental effort rating. This effect was present in full vision and in night vision conditions. There were no differences between both versions of the tactile instrument. We conclude that this emerging technology is a powerful support in a low-level flight without enlarging the pilot's mental effort.

A test of processing efficiency theory in a team sport context

In this study, we tested some key postulates of Eysenck and Calvo’s processing efficiency theory in a team sport. The participants were 12 elite male volleyball players who were followed throughout the course of a competitive season. Self-report measures of pre-match and in-game cognitive anxiety and mental effort were collected in groups of players high and low in dispositional anxiety. Player performance was determined from the statistical analysis of match-play. Sets were classified according to the point spread separating the two teams into one of three levels of criticality. Game momentum was also analysed to determine its influence on in-game state anxiety. Significant differences in in-game cognitive anxiety were apparent between high and low trait anxiety groups. An interaction between anxiety grouping and momentum condition was also evident in cognitive anxiety. Differences in set criticality were reflected in significant elevations in mental effort, an effect more pronounced in dispositionally high anxious performers. Consistent with the predictions of processing efficiency theory, mental effort ratings were higher in high trait-anxious players in settings where their performance was equivalent to that of low trait-anxious performers. The usefulness of processing efficiency theory as an explanatory framework in sport anxiety research is discussed in the light of these findings.

Subjective mental workload in Type A women

This study investigated the subjective mental workload in Type A women during a tracing task by analyzing ratings of six subscales (i.e., mental demand, physical demand, temporal demand, own performance, effort, and frustration level) of the National Aeronautics and Space Administration Task Load Index (NASA-TLX). Eight Type A and eight Type B women performed a tracing task for 30min. The task consisted of three 10min trials. Subjects rated their subjective workload on six subscales at the end of each trial. It was found that ratings of mental demand, physical demand, own performance, effort, and frustration level after the task were higher than those of Type B women. The ranks of ratings of frustration level in Type A women were quite high, whereas those of Type B women were quite low throughout the trials. These results suggest that Type A women felt a greater subjective mental workload for some specific aspects as compared with Type B women. In particular, it seems that higher ratings and ranks of frustration level in Type A women greatly contributed toward increasing overall workload in Type A women.Relevance to industryThe Type A behavior pattern has been considered to be one of the potential risk factors for coronary heart disease. A great number of Type A female workers are entering the workforce. Therefore, evaluation of the subjective mental workload of Type A women may be of importance in protecting their health.

Causal mechanisms of subjective cognitive dysfunction in schizophrenic and depressed patients.

We examined causal mechanisms of subjective cognitive (dis)abilities in schizophrenic and depressed patients, and in patient and normal control groups. This exploratory study included objective cognitive performance (Continuous Performance Task) as well as mood and mental effort ratings. Self-report of cognitive dysfunction in patients was not correlated with objective cognitive deficits. Correlational patterns differed between groups. Our findings in schizophrenic and depressed patients point to different causal mechanisms of their subjective cognitive malaise, despite similar subjective ratings. Depressed patients associate high mental effort during task performance with a reduced cognitive efficacy, whereas schizophrenic patients associate high effort with cognitive symptoms like distractibility and overload.

Measuring the dimensions of physician work.

Medicare has adopted a physician fee schedule that places emphasis on measuring physicians' work. We assessed the construct validity of self-reported work scores for 11 selected medical services performed by 44 internists in two large group practices. These physicians' work scores correlated highly with their ratings of time, physical and mental effort, and stress required to produce the services. Eighty-five percent of the variance in total work scores could be explained by the dimensions of work. Time was the most important input, but mental effort also was important for internists. We also found that physicians may reduce the total work required to produce two services for a patient if they provided those services in one visit, rather than in separate visits. Savings occurred for service pairs in which the physician could reduce his or her own time. Our findings imply that the Medicare fee schedule pays internists mainly for the time and mental effort required to produce medical services. They also underscore the importance to physicians of saving time by providing two services during the same patient visit.

Decrements in heart rate variability during memory search

Heart-rate variability (HRV) was recorded from subjects as they rested or performed two memory-search tasks with different memory-set sizes. Results indicated that all the estimates of HRV were considerably greater during resting periods than during the task but only the mid-frequency band (0.1 Hz component) reflected the changes in memory-set size by a more pronounced decrement with the higher load. Subjective mental-effort ratings collected for the task periods also reflected the different memory loadings but did not correlate with HRV measures.

A Physical Measure of Subjective Workload

This investigation compared a physical measure of subjective workload (i.e., hand dynamometer) with traditional verbal scaling techniques. There were four subjective rating groups. One group employed the Subjective Workload Assessment Technique (SWAT) which required three separate ratings of time stress, mental effort, and psychological stress. A second group used verbal magnitude estimation (ME). Two physical measure groups estimated the magnitude of workload by squeezing a dynamometer in accordance with the magnitude of workload experienced. The DYNA1 group made one overall rating of workload similar to the ME group. The DYNA3 group made three workload ratings along the same dimensions as SWAT. All groups rated the workload associated with the performance of a continuous memory task under twelve levels of task difficulty. The physical measure of subjective workload most closely corresponded to actual task performance differences. The results suggest future development of a physical measure of subjective workload which can be utilized on a continuous basis, thus avoiding a major shortcoming of typical verbal measures of subjective workload.