Concurrent Think-aloud Protocols Research Articles

Avoiding Unintended Consequences of Pediatric Blood Order Set Updates through In Situ Usability Testing.

Blood product ordering is a complex process and mistakes can harm patients and lead to poor outcomes. Orders and order sets can be designed to help mitigate errors, but major changes in design can unintentionally cause new errors. Our objective was as follows: (1) utilize formative in situ usability testing to iteratively improve the design of a redesigned blood product order set prior to go-live, (2) implement changes based on feedback derived from this testing, and (3) compare the error rate, system usability scale (SUS) score, time to task completion, and click counts between the prior order set in use at the time and the revised redesigned order set. A multidisciplinary project team convened to redesign blood product orders and order sets from scratch based on a review of the literature and benchmarking against four pediatric academic institutions with the goal of addressing prior ordering errors. The newly redesigned blood product order set was iteratively updated via in situ formative usability testing performed with available clinical users using a concurrent think-aloud protocol in real clinical environments. Errors, SUS scores, time to task completion, and click counts were assessed for the revised redesigned order set using summative testing. Formative usability testing with 20 participants led to seven design changes in the redesigned order set which reduced the error rate at go-live. Summative usability testing showed that even though the usability scores were only slightly improved for the revised redesigned order set, the error rates in blood orders were significantly decreased. Usability testing can identify design errors early in the process which can be rectified prior to implementation, thus avoiding unintended consequences of changes.

The Role of Technology in Undergraduate Bioscience Laboratory Learning: Bridging the Gap between Theory and Practice

Integration of technology is widespread in laboratory teaching, whose purpose includes building theoretical understanding and practical skills. How second-year bioscience undergraduate students at a UK university use technology to construct their understanding of laboratory-based topics was investigated using a concurrent think-aloud protocol in the laboratory, followed by semi-structured interviews. Analysis of think-aloud data used socially shared metacognitive coding since students may co-construct their understanding in these collaborative spaces. This analysis demonstrated that participants used technology within the laboratory either as a tool to conduct their experiment or, as a source of information to help them understand, apply or perform their experimental task. Semi-structured interviews demonstrated that students integrated technology into all aspects of their laboratory learning. Eight out of the ten participants described using technology to help them make connections between theory and practice as part of post-laboratory activities such as analysing or conducting further research on the topic. A survey of UK bioscience undergraduate modules found that 22% of modules did not use post-laboratory activities, suggesting that more scaffolding of post-laboratory activities could provide bioscience students with greater integration of practical and theoretical understanding and consequently meaningful laboratory learning.

Accounting for the knowledge gained during a web search: An empirical study on learning transfer indicators

Searches with learning intent typically require the users to interact with the searching environment and perform knowledge acquisition features such as scan, read, and process the online content to fulfill their information needs. To capture indicators from searching behaviors that could account for the knowledge gained during a Web search, a qualitative study was performed using the Concurrent Think-Aloud protocol to observe the mechanisms of transfer and map knowledge flows during 78 search sessions. Findings indicate evidence of transfer of learning in the form of sixteen online information searching strategy indicators. This research aids the understanding of how knowledge is gained during search sessions and how to identify behaviors that could indicate that learning has occurred, which could be used to represent knowledge gain on Web search engines. In this way, it can aid search engines to become not only better tools of searching, but also tools of learning.

STEM Integration through shared practices: examining secondary science and engineering technology students’ concurrent think-aloud protocols

To address the complete vision of the Next Generation Science Standards (NGSS Lead States. [2013]. Next Generation Science Standards: For States, by States. Washington, DC: The National Academies Press of the United States, the research focused on how scientific inquiry and engineering design shared practices promote design thinking and problem-solving skills. High school science and engineering technology students participated in the integrated STEM project, TRAILS, and learned engineering design-based integrated STEM lesson, D-BAIT. After each lesson implementation, triads of students from each class participated in a Concurrent Think-Aloud (CTA) protocol session. During the protocol session, the triads of students engaged in a design task as a team and discussed to solve a transfer problem, which required the application of STEM knowledge from the integrated STEM lesson. A total of 27 Think-Aloud datasets were collected and analyzed from the 2017–2019 school years. The results show that key features of TRAILS (scientific inquiry, biomimicry, and engineering design) emerged from the dialogue of both groups (engineering technology and Biology students). Additionally, patterns from the cognitive protocols (transitions between scientific inquiry, biomimicry, and engineering design) suggest that both groups used similar approaches to solving the engineering design problem.

Participatory Design of a Medication Module in an Electronic Medical Record for Paediatric Palliative Care: A Think-Aloud Approach with Nurses and Physicians.

Background: Electronic medical records (EMRs) play a key role in improving documentation and quality of care in paediatric palliative care (PPC). Inadequate EMR design can cause incorrect prescription and administration of medications. Due to the fact of complex diseases and the resulting high level of medical complexity, patients in PPC are vulnerable to medication errors. Consequently, involving users in the development process is important. Therefore, the aim of this study was to evaluate the acceptance of a medication module from the perspective of potential users in PPC and to involve them in the development process. Methods: A qualitative observational study was conducted with 10 nurses and four physicians using a concurrent think-aloud protocol and semi-structured qualitative interviews. A qualitative content analysis was applied based on a unified theory of acceptance and use of technology. Results: Requirements from the user’s perspective could be identified as possible influences on acceptance and actual use. Requirements were grouped into the categories “performance expectancies” and “effort expectancies”. Conclusions: The results serve as a basis for further development. Attention should be given to the reduction of display fragmentation, as it decreases cognitive load. Further approaches to evaluation should be taken.

Temporal Assessment of Self-Regulated Learning by Mining Students’ Think-Aloud Protocols

It has been widely theorized and empirically proven that self-regulated learning (SRL) is related to more desired learning outcomes, e.g., higher performance in transfer tests. Research has shifted to understanding the role of SRL during learning, such as the strategies and learning activities, learners employ and engage in the different SRL phases, which contribute to learning achievement. From a methodological perspective, measuring SRL using think-aloud data has been shown to be more insightful than self-report surveys as it helps better in determining the link between SRL activities and learning achievements. Educational process mining on the basis of think-aloud data enables a deeper understanding and more fine-grained analyses of SRL processes. Although students’ SRL is highly contextualized, there are consistent findings of the link between SRL activities and learning outcomes pointing to some consistency of the processes that support learning. However, past studies have utilized differing approaches which make generalization of findings between studies investigating the unfolding of SRL processes during learning a challenge. In the present study with 29 university students, we measured SRL via concurrent think-aloud protocols in a pre-post design using a similar approach from a previous study in an online learning environment during a 45-min learning session, where students learned about three topics and wrote an essay. Results revealed significant learning gain and replication of links between SRL activities and transfer performance, similar to past research. Additionally, temporal structures of successful and less successful students indicated meaningful differences associated with both theoretical assumptions and past research findings. In conclusion, extending prior research by exploring SRL patterns in an online learning setting provides insights to the replicability of previous findings from online learning settings and new findings show that it is important not only to focus on the repertoire of SRL strategies but also on how and when they are used.

Processing of grid-based design representations: a qualitative analysis of concurrent think-aloud protocols

Processing of grid-based design representations: a qualitative analysis of concurrent think-aloud protocols

A framework of artificial intelligence augmented design support

ABSTRACT Recent advances in Artificial Intelligence raise interest in its participation in design activity, which is commonly considered to be complex and human-dominated. In this work, we aim to examine AI roles in early design stages. The human ideation components and design tools related to AI are discussed in a framework of AI-augmented design support. The framework develops a hierarchy of design cognition (basis), approaches and principles. The cognitive models are constructed in an empirical study of 30 designers (26 for analysis, 4 for pilot study) by concurrent Think-Aloud protocol and behavior analysis. The process of producing new design ideas is explained by a transparent analysis of designers’ language and behaviors. Three strategies to organize cognitive activities in design ideation are summarized: develop structured consideration, relate to a scenario, and stick-to designing. These strategies suggest AI could act as (1) representation creation, (2) empathy trigger and (3) engagement, in principles of “knowledge-driven” and “decompose-and-integrate”. The design support with AI provides new perspectives on computer-based design tools that limit to well-defined design variables. The framework is built on a generic notion of design activity and “mimic” human design rationales, expected to benefit research of domain-independent computational design supports and cognitive supports.

Concurrent Think-Aloud Verbalizations and Usability Problems

The concurrent think-aloud protocol—in which participants verbalize their thoughts when performing tasks—is a widely employed approach in usability testing. Despite its value, analyzing think-aloud sessions can be onerous because it often entails assessing all of a user's verbalizations. This has motivated previous research on developing categories to segment verbalizations into manageable units of analysis. However, the way in which a category might relate to usability problems is currently unclear. In this research, we sought to address this gap in our understanding. We also studied how speech features might relate to usability problems. Through two studies, this research demonstrates that certain patterns of verbalizations are more telling of usability problems than others and that these patterns are robust to different types of test products (i.e., physical devices and digital systems), access to different types of information (i.e., video and audio modality), and the presence or absence of a visualization of verbalizations. The implication is that the verbalization and speech patterns can potentially reduce the time and effort required for analysis by enabling evaluators to focus more on the important aspects of a user's verbalizations. The patterns could also potentially be used to inform the design of systems to automatically detect when in the recorded think-aloud sessions users experience problems.

The Effectiveness of Learners’ Preferred and Unpreferred Written Corrective Feedback : A Think-Aloud Study

In early written corrective feedback studies, there were a lot of attempts to test if the feedback types are effective. This study was based on the assumption prevailing in the new era of studies of written corrective feedback, that feedback is effective. Firstly, it aims to compare the think-aloud protocols of the preferred and unpreferred feedback types which were chosen by 20 English learners (10 in preferred, and 10 in unpreferred feedback). To this aim, learners were asked to write diaries for six weeks. In each session, concurrent think-aloud protocols were employed and learners’ verbal reports were extracted, and their mental processes were coded through developing a table of mental operations. In the second part of the study, the relative effectiveness of the preferred and unpreferred feedback was measured. Using the Chi-square goodness-of-fit test, the findings of the first part showed a significant difference between the mental operations for each feedback type. The results for the second part suggested that there is a significant difference between the effectiveness of the preferred and unpreferred feedback types. This latter finding indicates that the preferred feedback can facilitate the development of the accuracy in L2 learners’ writing more than the unpreferred feedback.

Supporting children with burns: Developing a UK parent-focused peer-informed website to support families of burn-injured children

BackgroundChildren’s burn injuries can have a significant psychosocial impact on parents. However, the stress involved in caring for a child following a burn can often go unrecognized and does not necessarily prompt help seeking by parents. ObjectiveIt is common for adults to seek health-related support and information via the Internet. Many benefit from immediate and easy access to online psychological interventions. A prototype burn-specific, parent-focused, peer-informed, supportive website, designed to provide easy access to information and psychoeducation, was created and tested for acceptability. Patient involvementUsing a partnership-based method of website development, parents and professionals (clinical, academic and support organizations) were recruited and their particular expertise was acknowledged and valued. A participatory action approach was adopted to determine the acceptability of the website for parents/carers. Methods31 participants (9 parents, 22 professionals) tested a prototype version of the website. Data was collected using the eHealth Impact Questionnaire and the concurrent think-aloud protocol. ResultsParents and professionals had favorable opinions of the website. Parents’ ratings tended to be more favorable than professionals’, which was significant for the information and presentation. Participants’ thoughts were categorized into seven topics: need, structure/navigation, trust/relevance, language/comprehension, therapeutic content, mode of delivery, and suggested improvements. DiscussionMany practical and psychological barriers can prevent parents of burn-injured children accessing psychosocial support and contribute to a feeling of isolation. Participants felt that the website would be a valuable addition to UK pediatric burn care. The existence of an accessible resource could help to normalize parents’ experience of their child’s injury and reduce their perceived isolation, although peer interaction is not provided by the website. Practical valueThis online resource, hosting information and peers’ personal experiences, offers promising and exciting opportunities to empower parents whilst providing accessible supportive advice to encourage self-care and formal/informal support seeking when necessary.

Advancing Cardiac Surgery Case Planning and Case Review Conferences Using Virtual Reality in Medical Libraries: Evaluation of the Usability of Two Virtual Reality Apps.

BackgroundCare providers and surgeons prepare for cardiac surgery using case conferences to review, discuss, and run through the surgical procedure. Surgeons visualize a patient’s anatomy to decide the right surgical approach using magnetic resonance imaging and echocardiograms in a presurgical case planning session. Previous studies have shown that surgical errors can be reduced through the effective use of immersive virtual reality (VR) to visualize patient anatomy. However, inconsistent user interfaces, delegation of view control, and insufficient depth information cause user disorientation and interaction difficulties in using VR apps for case planning.ObjectiveThe objective of the study was to evaluate and compare the usability of 2 commercially available VR apps—Bosc (Pyrus Medical systems) and Medical Holodeck (Nooon Web & IT GmbH)—using the Vive VR headset (HTC Corporation) to evaluate ease of use, physician attitudes toward VR technology, and viability for presurgical case planning. The role of medical libraries in advancing case planning is also explored.MethodsAfter screening a convenience sample of surgeons, fellows, and residents, ethnographic interviews were conducted to understand physician attitudes and experience with VR. Gaps in current case planning methods were also examined. We ran a usability study, employing a concurrent think-aloud protocol. To evaluate user satisfaction, we used the system usability scale (SUS) and the National Aeronautics and Space Administration-Task Load Index (NASA-TLX). A poststudy questionnaire was used to evaluate the VR experience and explore the role of medical libraries in advancing presurgical case planning. Semistructured interview data were analyzed using content analysis with feedback categorization.ResultsParticipants were residents, fellows, and surgeons from the University of Washington with a mean age of 41.5 (SD 11.67) years. A total of 8 surgeons participated in the usability study, 3 of whom had prior exposure to VR. Users found Medical Holodeck easier to use than Bosc. Mean adjusted NASA-TLX score for Medical Holodeck was 62.71 (SD 18.25) versus Bosc’s 40.87 (SD 13.90). Neither app passed the mean SUS score of 68 for an app to be considered usable, though Medical Holodeck (66.25 [SD 12.87]) scored a higher mean SUS than Bosc (37.19 [SD 22.41]). One user rated the Bosc usable, whereas 3 users rated Medical Holodeck usable.ConclusionsInterviews highlighted the importance of precise anatomical conceptualization in presurgical case planning and teaching, identifying it as the top reason for modifying a surgical procedure. The importance of standardized user interaction features such as labeling is justified. The study also sheds light on the new roles medical librarians can play in curating VR content and promoting interdisciplinary collaboration.

An Investigation of Engineering Design Cognition and Achievement in Primary School

This study examined the design cognition and achievement results of both kindergarten and fourth grade students engaged in engineering design-based instructional activities. Relationships between design cognition and student grade level, as well as quality of student work, were investigated. 30 concurrent think-aloud protocols were collected from individual primary students as they worked in groups to design and make a solution to a design task. The concurrent think-aloud protocols were examined and coded to determine the duration of time the participants devoted to a pre-established set of mental processes for technological problem solving. Significant differences between kindergarten and fourth grade participants were found in the amount of time various cognitive processes were employed. Fourth grade students dedicated significantly more time to the mental processes of Creating, Defining Problems, Measuring, and Testing than kindergarten students. In addition, when examining the think-aloud protocols along with the evaluations of the participant’s design work, it was found that more time devoted to the cognitive process of Managing could be a significant predictor of lower design achievement. These findings can highlight potential areas for improving educational practice based on the cognitive abilities of students at different grade levels and the quality of their design work. As engineering design-based activities become more prevalent for the teaching of STEM-related content and practices, the results of this research, and the employed methodology, may demonstrate a promising practice for better understanding and assessing such education efforts.

A comparison of the problem-solving strategies between high school engineering and science students in integrated STEM education

우리 사회가 빠르게 발달함에 따라 통합적 문제 해결 능력을 갖춘 인재의 중요성이 커지고 있다. 이러한 사회의 요구를 뒷받침하기 위해 STEM/STEAM 교육이 등장하여 학생들의 통합적 문제해결 능력을 향상시키는데 기여하고 있다. 하지만 STEM/STEAM 수업에서 각 교과의 특성은 학생들은 사고 과정에 영향을 미치고, 교과 고유의 사고 과정은 문제 해결 과정에서 다른 특성을 보인다. 특히 연구자는 공학과 과학의 문제해결 접근법의 차이를 규명하기 위해 고등학교 공학(기술)과 과학 수업에서 각각 3명의 학생들을 대상으로 사례 연구를 실시했다. 이 연구에서 공학과 과학 교과 학생들은 STEM/STEAM 수업의 도입 부분을 함께 수강한 후에 모형 제작에서는 공학 학생들이 주도를 하고 과학 학생들은 피드백을 주는 활동을 실시했다. 연구자는 이 중에 6명의 학생들을 대상으로 수업내용과 유사한 공학 문제를 제시한 후 문제해결 과정을 분석했다. 문제 해결 과정은 소리내어 말하기 (think-aloud) 기법을 통해 음성과 영상이 비디오로 녹화되었다. 연구자는 학생들의 문제해결 전략을 분석하기 위해 Halfin이 제시한 7가지 인지전략을 이용했다. 연구 결과에서 공학 학생들은 다양한 문제해결 전략을 사용하려는 특성을 보였고, 과학 학생들은 해결 중심적인 몇 가지 인지전략에 집중적으로 사용했다. 또한 공학 학생들은 문제와 해결책 사이에서 순환적인 인지사용을 보인 반면, 과학 학생들은 디자인이나 모델링과 같은 몇 가지 전략을 집중적으로 사용한 것으로 드러났다. 또한 해결책의 비교에서는 과학 학생들은 창의적인 아이디어를 제시했지만 구체적인 실현 계획을 제시하지 못하였고, 공학 학생들은 혁신적인 아이디어 보다는 구체성과 현실적인 문제 등을 고려한 해결책을 제시했다. 연구자는 과학과 공학의 실천 방법이 학생들의 인지에 영향을 미친다는 것을 확인했으며, STEM/STEAM 수업을 실시할 때 학생들의 문제 해결 전략의 이해를 바탕으로 효과적인 지도 방법을 마련해야 한다는 것을 제언하였다.Given our rapid societal change, the ability to solve problems using interdisciplinary knowledge and skills is becoming increasingly important. To support these educational needs, many international countries have adopted STEM/STEAM education as a core learning platform. However, the fundamental differences of problem-solving strategies within the STEM/STEAM disciplines influence students’ cognitive structures and problem-solving strategies. To investigate the various problem-solving strategies used by science and engineering (technology) students, the researcher recruited six high school students, three from engineering courses and three from science courses. The participant students were taught a STEM/STEAM unit, but the engineering group had only participated in the physical modeling process. After completing the STEM/STEAM lessons, the two groups were given a design task with a structure similar to the integrated lessons. To assess the mental strategies of students, the researcher used the Concurrent-Think Aloud protocol and the seven of Halfin’s codes. The results showed that the engineering students tended to use a variety of problem-solving strategies, while the science students concentrated on solution-oriented strategies. The researcher also confirmed that the engineering students showed various design iterations by revisiting the problem identification process throughout the design process. However, the science students tended to show a limited number of iterations. The design outcomes also showed that the science students yielded many innovative design ideas but failed to provide details for their design solutions. When instructing STEM/STEAM lessons, educators need to utilize authentic instructional strategies to improve students’ integrative problem-solving abilities.

Identifying design process patterns: a sequential analysis study of design thinking

Design is a key element of both the teaching and learning of engineering and technology. However, the process of engineering design has yielded limited research results. This study explored the iterative design process by searching for sequential design thinking patterns. The researchers collected nine concurrent think-aloud protocols from fourth-grade elementary students. The study identified that idea generation plays a central role in design that features the dominant use of time. In addition, the researchers revealed significant pathways in design thinking and built a design pattern model. The results will not only help engineering and technology educators the understanding of design behavior, but also support the harmonious matching of learning and teaching styles in engineering and technology education.

Are L1 and L2 strategies transferable? An exploration of the L1 and L2 writing strategies of Chinese graduate students

ABSTRACT Over the past three decades, second language writing has witnessed a growth of studies on learners’ writing strategies, as well as a shift from product-oriented to process-oriented research. Although the field has achieved a greater understanding of strategies used by English-as-an-additional-language (EAL) writers, limited research has examined the strategies used by learners in completing the same writing task in their L1 (first language) versus L2 (second language). Such an exploration would shed light on the transferability of writing strategies between L1 and L2 in different writing tasks. The present study used a questionnaire, concurrent think-aloud protocol and immediate post-task retrospective interviews to examine the strategies used by 35 Chinese EAL learners in completing L1 and L2 writing tasks and how these strategies related to writing performance. Results pertaining to the identified L1 and L2 writing strategies, similarities and differences between L1 and L2 writing strategy use, the potential transferability of L1 and L2 strategy use in writing, and strategy use in relation to writing performance are reported and discussed in relation to previous research. Also considered are implications, limitations and future research directions to move the field forward.

Automated comprehensive evaluation approach for user interface satisfaction based on concurrent think-aloud method

The concurrent think-aloud protocol (CTA) is an effective method for collecting abundant product comments related to user satisfaction during the execution of evaluation tasks. However, manual analysis of these audio comments is time-consuming and labor-intensive. This paper aims to propose an approach for automated comprehensive evaluation of user interface (UI) satisfaction. It takes advantage of text mining and sentiment analysis (SA) techniques instead of manual analysis in order to assess user comments collected by the CTA. Based on the results of the SA, the proposed approach makes use of the analytic hierarchy process (AHP) method to evaluate the overall satisfaction and support developers for UI design improvements. In order to enhance the objectivity of evaluation, a sentiment matrix originating from text mining and SA on user comments is used to replace the criteria and the relative weights of the AHP method which were previously defined by experts. A comparison between the questionnaire survey method and the proposed approach in the empirical study suggested that the latter can efficiently evaluate UI satisfaction with high accuracy and provide designers abundant and specific information directly related to defects in design. It is argued that the proposed approach could be used as an automated framework for handling any type of comments.

Exploring rater conceptions of academic stance and engagement during group tutorial discussion assessment

The present study uses concurrent think-aloud verbal protocols alongside post-hoc interviews to explore how six teacher-raters determine a students' ability to explain academic concepts and argue for an academic stance supported by sources during a 25-minute group tertiary academic tutorial oral assessment. We explored how the raters arrived at decisions regarding the quality of students' academic stance and engagement in light of difficulties with rater attention in real-time, L2 language concerns, assessing engagement in a group oral setting, and the use of spoken citation to support speakers’ claims. Substantial differences in rater practice, beliefs and interpretation of assessment criteria were all found during the assessment of student performance, confirming a number of difficulties faced by raters assessing the academic ability of multiple participants over lengthy extended, interactional discourse. The findings shed real-time conceptions of (un)successful academic stance and engagement in group oral contexts, as well as confirm the usefulness of verbal protocols in revealing previously hidden complications for group oral assessments in an academic context, with accompanying suggestions for resolving such complications.

Examining Elementary School Students’ Transfer of Learning Through Engineering Design Using Think-Aloud Protocol Analysis

The introduction of engineering practices within the Next Generation Science Standards provides technology educators with opportunities to help STEM educators infuse engineering design within a core curriculum. The introduction of teaching engineering design in early elementary grades also provides opportunities to conduct research investigating how young students use engineering design as a way to solve problems. There is a need for research to assess how students experience engineering design as a pedagogical approach to learning science. This article will feature research on elementary students’ cognitive strategies used during engineering-design science activities. We adopted the concurrent think-aloud (CTA) protocol analysis method to capture how students conceptualize design and enhance science learning. During the 2012–2013 school year, we video recorded 66 CTA sessions, and this study examines six of those sessions. NVivo (Version 10) was used to code each video using common cognitive strategies categorized by Halfin (1973). Research findings indicate that participants increased the amount of time spent on mathematical thinking by 34% when given a math-specific design task. Pre- and post-tests showed that participants gained significant science content knowledge. However, we also confirmed that participants struggled with applying accurate mathematical and scientific knowledge to solving the given design problem.

Diagnosing Students’ Understanding of the Nature of Models

Students’ understanding of models in science has been subject to a number of investigations. The instruments the researchers used are suitable for educational research but, due to their complexity, cannot be employed directly by teachers. This article presents forced choice (FC) tasks, which, assembled as a diagnostic instrument, are supposed to measure students’ understanding of the nature of models efficiently, while being sensitive enough to detect differences between individuals. In order to evaluate if the diagnostic instrument is suitable for its intended use, we propose an approach that complies with the demand to integrate students’ responses to the tasks into the validation process. Evidence for validity was gathered based on relations to other variables and on students’ response processes. Students’ understanding of the nature of models was assessed using three methods: FC tasks, open-ended tasks and interviews (N = 448). Furthermore, concurrent think-aloud protocols (N = 30) were performed. The results suggest that the method and the age of the students have an effect on their understanding of the nature of models. A good understanding of the FC tasks as well as a convergence in the findings across the three methods was documented for grades eleven and twelve. This indicates that teachers can use the diagnostic instrument for an efficient and, at the same time, valid diagnosis for this group. Finally, the findings of this article may provide a possible explanation for alternative findings from previous studies as a result of specific methods that were used.