Collaborative Assessment Process Research Articles

Evaluating the Impact of an AI Critical Friend on Student Research Proposals, Critical AI Literacy, and Transparent Collaborative Assessment Practices

This paper presents a work-in-progress report on an innovative approach to integrating generative AI capabilities into the assessment process, focusing on research proposal development. Specifically, the study explores the impact of incorporating an AI-powered "critical friend" that provides targeted feedback and guidance to students as they craft their research proposals. The aim is to evaluate how this AI-powered support system affects students' critical AI literacy, the quality and transparency of the collaborative assessment process, and, ultimately, the strength of the final research proposals. Through a mixed-methods approach combining surveys, focus groups, and analysis of student work and interactions with generative AI, the researchers seek to uncover key insights on the benefits, challenges, and ethical considerations of leveraging AI to enhance the research proposal development experience. The findings from this study will contribute to the growing body of literature on the role of AI in academic assessments and provide practical recommendations for educators and institutions looking to responsibly integrate these emerging technologies.

Collaborative Assessment: Using Self-assessment and Reflection for Student Learning and Program Development

As program-level assessment increasingly becomes an integral part of the higher-education landscape, so does the debate regarding the efficacy of current assessment methods. Traditionally, students do not participate in assessment—neither of their own learning nor of institutional or program efficacy. Our assessment process presents an alternative to traditional program-level assessment and is meant to improve student learning in two ways: (1) by asking students to reflect on their achievement of learning outcomes using evidence-based methods; (2) by providing assessment practitioners with authentic, contextualized data on which to make claims about curricula. This collaborative assessment process was designed to address the complex needs of a cross-curricular rhetoric program but responds to many general concerns about traditional assessment methods.

Attentional bias and the Suicide Status Form: Behavioral perseveration of written responses

Given the vast public health problem of suicide, the need for more effective assessment of suicidal risk is clear. The major approaches applied to this challenge include various direct approaches (e.g., suicide-focused interviews) and indirect approaches (e.g., implicit methodologies or “occult” assessments) that tend to assess an attentional bias for suicidal risk, the latter of which the present investigation sought to study. Using the Suicide Status Form (SSF)—the central multi-purpose tool that is used within a collaborative assessment process with suicidal patients who are engaged in the “Collaborative Assessment and Management of Suicidality” (CAMS; Jobes, 2016)—we aimed to investigate the influence of perseverative, hand-written content responses pertaining to potential suicidal risk. Specifically, we explored whether repeating certain topic content might reflect a perseverative response style; we thus compared written content results of first session SSFs taken from a sample of suicidal U.S. Army Soldiers (Study 1) and a sample of suicidal college students (Study 2). Across the two studies, patients who repeated the same content (“1-Topic Repeaters”) had significantly higher ratings related to suicidal ideation in comparison to those with more heterogeneous response styles. This replicated finding perhaps reveals a form of behavioral perseveration that is potentially related to increased suicidal risk with possible implications for successful treatment.

A workflow-based solution to support the assessment of collaborative activities in e-learning

PurposeThe purpose of this paper is to focus on assessing individuals’ problems in learning groups/teams and should lead to the assessment of the group/team itself as a learning entity.Design/methodology/approachIn this paper, an extension of the IMS-Learning Design (IMS-LD) meta-model is proposed in order to support the assessment of collaborative activities in e-learning. Besides, the software architecture which consists of a set of components forming a web wizard to create, track and assess the collaborative assessment processes is described as to support that extension of the IMS-LD meta-model.FindingsWith the proposed solution we can: make assessment fairer using individual and collective assessment indicators to assign final scores to learners; make an assessment step by step for better individual and collective monitoring activities; and divide the assessment into lighter phases for the correctors. Consequently, the evaluator will have more detailed information about his/her students and the quality of judgment will be better. This could also be useful for the evaluator in order to plan further examinations.Research limitations/implicationsFurther experimentations are necessary to test the effectiveness of the proposed system in order to analyze its performances under a massive usage. In addition, the authors plan to use a survey to collect learners’ opinions to know the effectiveness of the proposal in terms of fairness in the assessment of collaborative activities in an online community.Originality/valueThis paper addresses important issues in the educational area, especially assessment of collaborative activities. In fact, to reduce subjectivity and increase fairness in assessing learners in collaborative work, for example, using the peer assessment, in order to try reducing subjectivity and fairly assessing learners. However, while assessing group work, the same mark is attributed for all group members and authors have concluded that it is not the right approach to make a fair and more objective assessment.

Building a sustained climate assessment process

The leaders and authors of the Third US National Climate Assessment (NCA3) developed new modes of engaging academia, the private sector, government agencies and civil society to support their needs for usable, rigorous, and timely information and better connect science and decision-making. A strategic vision for assessment activities into the future was built during the NCA3 process, including recommendations on how to establish a sustained assessment process that would integrate evolving scientific understanding into decision making to manage the risks of climate change over time. This vision includes a collaborative assessment process that involves partnerships across a diverse and widely distributed set of non-governmental and governmental entities. The new approach to assessments would produce timely, scientifically sound climate information products and processes, rather than focusing on the production of single quadrennial synthesis reports. If properly implemented, a sustained assessment would be more efficient and cost-effective, avoiding the painful and time-consuming process of beginning the assessment process anew every 4 years. This ongoing assessment would also encourage scientific and social innovations and explore new insights and opportunities, building the capacity to advance the development and delivery of climate information to meet societal requirements and benefit from scientific opportunities.

Contributory Role of Collaborative Assessment in Improving Critical Thinking and Writing

Instilling critical thinking skills in language learners’ mind and enhancing second language writing are gaining momentum in the field of English language teaching. Though some approaches to meeting these objectives have been proposed, review of the literature revealed that the contributory role of collaborative assessment in fostering critical thinking and second language writing has been overlooked to date. Thus, this study was conducted to delve into this issue. To this aim, two intact intermediate EFL classes in Iran, each of which contains 18 learners, were included in the study; they were randomly assigned to Student-Student (S-S) collaborative assessment and Teacher-Student (T-S) collaborative assessment groups. After receiving six sessions of treatment, both groups wrote an essay on an IELTS topic. Results of the study indicated that collaborative assessment, regardless of its type, has the potential to foster critical thinking and writing proficiency. Further, it came to light that S-S collaborative assessment group significantly outperformed the T-S collaborative assessment group in terms of gains in critical thinking and writing proficiency. In light of the findings, language teachers are suggested to involve learners in collaborative assessment processes; further, some suggestions are offered at the end to show some fruitful avenues for further research.

Therapeutic Assessment “on the Front Lines”: Comment on Articles From WestCoast Children's Clinic

I comment on the articles by Mercer (2011/this issue); Guerrero, Lipkind, and Rosenberg (2011/this issue); and Haydel, Mercer, and Rosenblatt (2011/this issue), which describe their practice of collaborative and therapeutic psychological assessment in a community mental health setting. These articles demonstrate that collaborative and Therapeutic Assessment can be used successfully with clients from underprivileged, high-risk backgrounds. Such assessments are rigorous for both clients and assessors, and their success depends on the ability of clinicians to form trusting relationships with clients. In fact, empathic disruptions by assessors are almost inevitable during the collaborative assessment process, especially if assessors’ backgrounds and contexts are quite different from those of their clients. However, if assessors receive enough support to “take off their white coats” and reach into their clients’ worlds, such disruptions can be repaired, leading to positive changes in both clients and assessors. This kind of assessment work is not for the faint of heart, but it is immensely rewarding.

Examining the Role of Collaborative Assessment in a Didactic Dental Hygiene Course

The purpose of this study was to evaluate a technique known as collaborative assessment considering students' and faculty members' perceptions of the value and utility of this approach. Twenty-eight dental hygiene students took eight individual assessments (quizzes) immediately followed by completion of the same assessment in a five-member peer group as part of a didactic dental hygiene course. At the end of the semester, student perceptions and course performance were examined. Faculty perceptions were also collected. A paired samples t-test was conducted to compare student performance on individual and group assessments. Students reported that this collaborative assessment approach enhanced their learning, provided them with experience in defending their answers, and decreased stress. Concerns for fairness and individual accountability were noted by both students and faculty. The mean group score for all quizzes combined (M=91 percent, SD=2 percent) was significantly higher by 16 percentage points (91 percent versus 75 percent) than the mean individual score for all quizzes combined (M=75 percent, SD=8 percent), t(27)=11.61, alpha =.05, p<.0005. This study suggests that additional measures are needed to ensure individual preparation and accountability when using the collaborative assessment process as a learning strategy with dental hygiene students.

“Am I Going Crazy, Doc?”: A Self Psychology Approach to Therapeutic Assessment

In this case study, we explore the effectiveness of Therapeutic Assessment with a severely disturbed 25-year-old man, referred by his therapist, following Finn's (2007; Finn & Tonsager, 1992, 1997) model. This patient–therapist pair had been working together for approximately 2 months, but the therapy had ceased to progress. The therapist requested a clearer picture of his patient's affective functioning, interpersonal functioning, and self-functioning that might facilitate more effective treatment. Through a collaborative assessment process informed by the principles of Kohutian self psychology, the evaluator and patient slowly formed a working alliance that proved useful for the eventual communication to the patient of his psychologically tenuous reality. This case illustrates the utility of a collaborative, multimethod Therapeutic Assessment with a severely ill patient and the use of Therapeutic Assessment by a less experienced clinician.

Symposium 4: Networked Learning and Interculturality: Perspectives on Working in Intercultural Contexts

Increasingly, higher education is becoming the setting for intercultural education, where those involved are often from different cultures and may therefore hold different beliefs, values and expectations about the nature of learning, teaching, scholarship and research. Higher education institutions in many "westernised" countries now recruit large numbers of students from overseas: these students are a significant source of funds for these universities. Accommodating international students into 'local' higher education settings is therefore a challenge to many HE teachers. Requiring overseas students to merely 'integrate' into the local HE culture is unlikely to be the answer. Moving to forms of interculturally sensitive learning and teaching practices is more likely to prove acceptable to the overseas students as well as being practically and perhaps also ethically acceptable to the teachers and universities involved. Teachers and students alike have to develop new forms of cultural communication. This is the focus of this symposium. This symposium will look at Networked Learning in the emerging and important context of intercultural learning where teachers and students from different cultural settings are working in geographically dispersed contexts. The six papers making up this symposium consider the nature of culture and interculturality from a variety of different but inter-related perspectives: Banks, McConnell and Bowskill consider a recent online intercultural course where students and tutors came from the UK and China, and where the course design explicitly tried to provide an integrated and purposeful intercultural experience for all involved (see http://csalt.lancs.ac.uk/echina). They consider the possibility of teachers and students developing what some call 'intercultural' competence in these educational settings, and ask the question is that possible? Bowskill, McConnell and Banks examine the same course but from the perspective of tutor development and integration. Against a literature background of little if any practical and theoretical understanding of how to develop online course tutors for participating in a virtual, intercultural course they suggest that a reflective 'space' in which the online tutors can communicate and share practical and theoretical experiences is a good way forward. McConnell, Bowskill and Banks (again in the same context) consider higher education teachers conceptions of e-learning in the UK and in China. Their interest here stems from their experience of working over several years in the Sino-UK eChina e-learning programme during which they developed Masters level online courses. During this period they came to realise that intercultural exchanges in course development between the UK and China were paramount, and that it was necessary for all those involved to understand each other's conceptions of learning and teaching in order to ensure high levels of authentic course development. Reynolds considers the case of international students, the application of participative methods in teaching and the possibility of greater cultural differences at play. The paper examines the experience of teaching multinational students in cooperative learning contexts, and considers the consequences of different research approaches in interpreting the meaning of these experiences. Trehan's paper focuses on the adoption of a critical psychodynamic approach to our understanding of collaborative assessment processes in networked learning which have the aim of supporting diversity and difference. Her aim is to illuminate the dynamics of these processes, and by doing so show how little these processes do in reality support diversity and difference. In Zhang's paper we return to the context of the Sino-UK e-learning programme. Here, the experiences of Chinese students taking the course are examined. Despite a prior belief that the technology of the virtual learning environment and the need to communicate in English might impede Chinese students' capacity to take part in the course and learn, Zhang shows that in fact this was not the case. She says that he major factors impeding these students ability to participate in the course was their wish to save "face" in their online communications, and their belief in the authority of the text and the teacher. The implications of this for the design and implementation of intercultural online courses is considered by the author.

Examining a collaborative assessment process in networked lifelong learning

This case study paper is concerned with the development of professional practice in networked learning environments. Through the presentation of one case (a) an assessment process in a networked learning environment which involves the learner, their peers and a tutor in making judgments about the learner's formal coursework; and (b) the role of the tutor/ teacher in these processes are examined. From an analysis of the case it is suggested that assessment should be a learning event and that networked learning course participants should be involved in making judgments about their own and their peers’ learning. The benefits to learners and tutors of these collaborative learning processes are examined and discussed.

Academic, Employer and Student Collaborative Assessment in a Work‐based Cooperative Education Course

ABSTRACT This paper describes an assessment method that was adopted for a work‐based co‐operative education course that forms part of a BBS degree programme. An innovative feature of the course is the collaborative process used for assessing student achievement. The student's on‐site project work is assessed jointly by the student, employer and academic supervisor. At this stage of the course's implementation feedback from the three parties indicates that there is general support for this collaborative assessment process. However, analysis of the grading practices of the three parties has produced some interesting preliminary findings, particularly in relation to gender differences.