Novelty Evaluation using Sentence Embedding Models in Open-ended Cocreative Problem-solving

Abstract

AbstractCollaborative creativity (cocreativity) is essential to generate original solutions for complex challenges faced in organisations. Effective cocreativity requires the orchestration of cognitive and social processes at a high level. Artificial Intelligence (AI) techniques, specifically deep learning sentence embedding models, have emerged as valuable tools for evaluating creativity and providing feedback to improve the cocreation process. This paper examines the implications of sentence embedding models for evaluating the novelty of open-ended ideas generated within the context of real-life project-based learning. We report a case study research design involving twenty-five secondary students, where a cocreative process was developed to solve a complex, open-ended problem. The novelty of the co-generated ideas was evaluated using eight pre-trained sentence embedding models and compared with experts’ evaluations. Correlation and regression analyses were performed to examine the reliability of the sentence embedding models in comparison to the experts’ scoring. Our findings disclose that sentence embedding models can solve the challenge of evaluating open-ended ideas generated during the cocreative process. Moreover, the results show that two-sentence embedding models significantly correlate better with experts- Universal Sentence Encoder Transformer (USE-T) and USE Deep Averaging Network (USE-DAN). These findings have a high pedagogical value as they successfully evaluate the novelty generated in a real problem-based environment that uses technology to promote key cocreative processes. Furthermore, the real-time evaluation facilitated by these models can have a strong pedagogical impact because it can provide valuable feedback to teachers and students, thereby optimising collaborative ideation processes and promoting effective cocreative teaching and learning methodologies.