From Metrics to Meaning: Time to Rethink Evaluation in Human–AI Collaborative Design

Every software in the universe requires maintenance and management during its life cycle. The manual management of software is costly and sometimes error-prone. The other solution is autonomic computing that induces self-management capabilities, “self-*services”, in software systems with the help of autonomic managers. The design quality of a self-management capability affects the computing infrastructure regarding processing load, the memory requirement, data channel demand and performance of perturbation restore. It is critical to assess the design quality of a self-management capability to determine its effect over the computing infrastructure when it gets invoke against some anomaly or perturbation. Moreover, there are two possible host environments for an autonomic manager to offer a self-management capability as a self-* service: the local environment and the cloud environment. A criterion is needed to decide which environment is more suitable and cost-effective to run the service. However, the literature lacks in the assessment of the design quality metrics on self-management capabilities and the suitability and cost-effectiveness of the execution environment. In this work, we have proposed a suite of design quality metrics to determine the design quality of self-management capabilities. We validate the proposed metrics with a stock trade & forecasting system that was designed as an autonomic computing system with self-management capabilities. The proposed metrics were applied to define functions that identify the suitable and cost-effective execution environment for the self-* service. The results proved that these metrics are useful in determining the design quality, suitability, and cost-effectiveness of a self-* capability for an autonomic computing system. The proposed metrics can be used to compare differently designed autonomic solutions for complexity, efficiency, performance, understandability, and maintainability.

Purpose This study examines how innovators and AI work together during idea generation for product innovation. It examines how varying levels of reliance on AI impact cognitive engagement and, in turn, influence the quantity, originality and feasibility of ideas as well as innovators’ overconfidence. The study highlights AI’s role as a cognitive amplifier, showing how human intuition and AI's analytical power interact to support creativity and innovation. Design/methodology/approach A controlled experiment was conducted with 123 product innovators, testing three conditions: no AI, moderate AI assistance and high AI assistance, to measure cognitive engagement, number of ideas generated, originality, feasibility and innovator overconfidence. ANOVA, polynomial regression and mediation tests were performed to determine the effects of AI assistance on innovative idea generation. Findings The results reveal an inverted U relationship between AI assistance, cognitive engagement and the generation of ideas for product innovation. Moderate AI assistance optimally enhances cognitive engagement, producing the highest number of original and feasible ideas. In contrast, excessive AI assistance may foster automation bias, reducing originality and increasing overconfidence. At the same time, the absence of AI constrains idea generation due to cognitive limitations in relying only on human abilities. Practical implications The findings show that moderate AI use maximizes the quantity, originality and feasibility of ideas while minimizing overconfidence. Innovation managers should structure ideation sessions to cap AI interactions, promote critical evaluation of AI outputs and combine them with human insight. This balanced approach enables firms to optimize cognitive engagement and generate higher-quality product innovations. Originality/value This research uniquely contributes to product innovation literature by explicitly focusing on human–AI synergy, highlighting AI’s optimal role as a cognitive enhancer rather than a substitute. It elucidates conditions that maximize innovative outcomes through balanced human–AI collaboration, providing actionable managerial guidelines for structuring AI integration to amplify creativity and mitigate biases in idea generation for product innovation.

Self-regulated learning (SRL) is essential in promoting students’ learning performance, especially in technology-rich environments where learning can be disorienting. Student engagement is closely associated with SRL, although the regulation of engagement in SRL is still underexplored. In this study, we aimed to compare the relative importance of cognitive and behavioral engagement in the three SRL phases (i.e., forethought, performance, self-reflection) to learning performance in the context of clinical reasoning. Specifically, students were tasked to solve two virtual patients in BioWorld, an intelligent tutoring system. We measured student behavioral engagement as their time spent on diagnostic behaviors. Students’ cognitive engagement was extracted from their think-aloud protocols as they verbalized their thinking and reasoning process during the tasks. We analyzed the relative importance of cognitive and behavioral engagement in the three SRL phases to diagnostic efficacy. Results suggested that the effects of engagement on student performance depend on task complexity. In the complex task, the six predictors (i.e., two types of engagement in the three SRL phases) explained 36.81% of the overall variances in learner performance. Cognitive engagement in SRL played a more significant role than behavioral engagement in predicting students’ performance in clinical reasoning.

Purpose Brands are investing heavily in content marketing within digital communication channels, yet there is limited understanding of the effectiveness of this content on consumer engagement. This paper aims to examine how consumer engagement with branded content is created through consumer-initiated online brand communities (OBCs) and brand-initiated digital content marketing (DCM) communications. Self-brand connections are examined as an important antecedent to the cognitive, affective, behavioural and social dimensions of consumer engagement and the subsequent impact of engagement on loyalty is explored across these two channels. Design/methodology/approach A survey approach was used with two consumer samples for one focal retail brand, namely, a consumer-initiated OBC (Facebook) and email subscribers of the retail brand’s DCM communications. A multi-group analysis of structural invariance procedure was used to comparatively examine the formation of engagement for consumers within the OBC and DCM channels. Findings This study demonstrates the different ways in which engagement forms across different digital communication channels. Self-brand connection (SBC) was found to strongly drive behavioural, cognitive, affective and social engagement. The cognitive, affective and behavioural engagement was found to mediate the self-brand connection and consumer loyalty relationship. Overall, this relationship was most strongly and significantly mediated by affective and cognitive engagement within the OBC channel when compared to the DCM channel. Research limitations/implications The findings of this study should be interpreted with several limitations in mind. First, the research was conducted within the confines of one OBC, within one social networking site platform characterised by self-selected membership based on a passion and immersion with the brand. This means that consumers within the OBC were highly connected to one another and the retail brand and highly socialised in-group norms and mores. This type and intensity of connection may not be the case for all forms of OBCs. Second, this study was limited to one retail brand, from one brand category. Future research should examine OBCs across a range of utilitarian and hedonic brands to comprehensively contextualise the dimensions of engagement. Third, the data for this study was cross-sectional. The use of netnographic analysis and qualitative interviews across a range of OBCs would support the triangulation of the findings of this research, especially with regard to the narrative that consumers’ express when discussing how their SBC manifests through the dimensions of engagement. Fourth, this study explored a single antecedent of engagement, namely, self-brand connections. Future research may consider how SBC operates in conjunction with other complementary factors to enhance consumers’ affective, cognitive, social and behavioural engagement such as brand awareness, satisfaction and participation/interactivity. In addition, future research could examine an expanded array of engagement outcomes such as purchase intention, the share of wallet and reputation. Finally, future research should examine the operationalisation and validation of the dimensions of engagement using multiple competing scales to assess the suitability of these engagement scales across multiple brand categories and contexts. Practical implications Given the increasing investment in branding within social media and the fragmentation of brand communications across multiple communications platforms, the management of effective brand communications remains a significant challenge. This study found that the relationship between self-brand connections, affective, social, behavioural and cognitive engagement and loyalty was context-specific and moderated by a digital communication channel (OBC vs DCM email marketing), thus providing insights as to the effectiveness of OBCs and DCMs as two tools for enhancing consumer loyalty. Originality/value This study makes a novel contribution to the engagement literature by examining the antecedent role of self-brand connections in predicting consumers’ engagement; the moderating role of digital communication platforms (OBC vs DCM) on the formation of cognitive, affective, behavioural and social engagement; and the mediating effect of these dimensions on loyalty.

Data warehouses have been developed that stores information enabling the knowledge worker to make better and faster decisions. As a decision support information system, a data warehouse must provide high level quality of data and quality of service. Various metrics have been defined and theoretical validated to measure the quality of the data warehouse in a consistent and objective manner and if quality measured, it can be managed and improved. Now, in this paper we will use these design quality metrics and empirically validated these metrics by conducting an experiment using regression analysis and deriving the conclusions according to the analysis so that they can be used by researchers and users.KeywordsObject Oriented Conceptual ModelingMetricsRegression AnalysisEmpirical Validation

This article reports on original research investigating the pivotal role that teachers play in student engagement, using a tri-dimensional framework. This framework identifies how teachers’ pedagogical choices impact student engagement in ways that influence students’ external behaviours, internal emotions and internal cognitions. A questionnaire was developed to explore secondary teachers’ (n = 223) perceptions of pedagogies that support students’ behavioural, emotional and cognitive engagement in the classroom. Findings revealed that female participants placed higher importance on pedagogies that support students’ cognitive and behavioural engagement, and participants with leadership roles placed higher importance on pedagogies that support students’ cognitive and emotional engagement. Also emerging from the research was a negative correlation between the importance teachers placed on pedagogies that support cognitive and behavioural engagement and their school’s ICSEA value (the measure of socio-educational advantage in Australian schools). Overall, results support the tri-dimensional framework of student engagement utilised in this study and provide a robust framework for future research to further explore teachers’ pedagogical choices and how these choices impact student engagement.

<p>The phenomenon of school engagement, considered as a stable, directed and active participation of students in educational activities and in the school life in general, is of considerable interest to researchers in the field of educational psychology. According to modern scientific concepts, engagement can be assessed through behavioral, cognitive, emotional and social manifestations. The research had its purpose to study the dynamics of school engagement in adolescents, as well as to reveal the relationship of conscious self-regulation with behavioral and cognitive components of engagement based on the longitudinal data obtained on the sample of 6-8 grade students (N=80). A separate task was to find an answer to the question of whether the conscious self-regulation can be considered as a significant predictor of changes in the behavioral and cognitive engagement of students during their study in the secondary school. Methods: "Multidimensional Scale of School Engagement"(Wang et al., 2019; Fomina, Morosanova, 2020); "The Self-Regulation Profile of Learning Activity Questionnaire — SRPLAQ" (Morosanova, Bondarenko, 2017). Statistical processing of longitudinal data (including the latent growth curve modeling) made it possible to reveal the negative dynamics of the students’ behavioral and cognitive engagement during their study in the secondary school. The data analysis allowed to describe the effects of relationship between behavioral and cognitive engagement: a higher level of cognitive engagement contributes to a less pronounced decrease in behavioral engagement. The study established positive correlations of conscious self-regulation with both cognitive and behavioral engagement. The decrease in engagement is less pronounced in adolescents with a higher level of development of conscious self-regulation.</p>

<p>The aim of this research is to examine the relationship between employees’ self-efficacy components (behavioural, cognitive and motivational engagement) and their job performance in the F&B manufacturing sector in Klang Valley, Malaysia. In this study, researchers explored how cognitive engagement, defined as a worker’s intention to engage in their work, motivational engagement or the drive to exceed their tasks’ requirements and achieve excellence and behavioural engagement, which refers to an employee's positive attitude towards their work, were predicted from self-efficacy. Overall, self-efficacy is an individual’s firm belief that (s)he can succeed in performing a specific task. The quantitative survey method was applied, and the sample included 183 employees who work in the food and beverage manufacturing sector in Klang Valley. The results illustrate that while behavioural engagement has not been proven to have a substantial influence on the job performance of an employee, both cognitive and motivational engagement have greatly impacted job performance with a significant positive relationship. Cognitive engagement can be seen as an individual’s enthusiasm and willingness to put in effort to be able to accomplish any specific task. As such, it has demonstrated a significant relationship with job performance. Motivational engagement, or the desire to outdo specific task requirements and avail high-quality performance, was also significant and had a positive impact on job performance. On the other hand, behavioural engagement, which refers to an employee's positive attitude towards their work, which enhances motivation and performance in organizational activities has revealed an insignificant influence on job performance. Thus, this study has proven the need for any organization to enhance both cognitive and motivational engagement in order to improve the performance of all employees, along with the success of such organizations.</p><p> </p><p><strong> Article visualizations:</strong></p><p><img src="/-counters-/soc/0719/a.php" alt="Hit counter" /></p>

BackgroundThis research explores the relationships between the educational environment, student engagement, and academic achievement in Health Professions Education (HPE) , specifically examining the mediating role of engagement.MethodsThe study used cross-sectional design, and data were collected from 554 HPE students via self-report questionnaires. The Dundee Ready Education Environment Measure (DREEM) assessed the educational environment while the University Student Engagement Inventory measured learning engagement across the behavioral, emotional, and cognitive dimensions. Academic achievement was measured using cumulative GPA. Relationships between study variables were analyzed using path analysis.ResultsPath analysis demonstrated that four educational environment subscales directly affected emotional engagement (48% variance explained). Students’ perception of learning and academic self-perceptions influenced behavioral engagement (28% variance explained), while cognitive engagement was influenced by academic self-perceptions (39% variance explained). GPA was positively influenced by behavioral and cognitive engagement but negatively by emotional engagement. Cognitive and behavioral engagement mediated the relationship between students’ academic self-perceptions and academic achievement.ConclusionsStudents’ perceptions of the educational environment significantly influenced emotional engagement, followed by cognitive and behavioral engagement. Cognitive and behavioral engagement directly affected academic achievement and mediated the relationship between the educational environment and academic achievement.

적극적인 수업참여는 학생들의 학업성취와 수업 및 학습의 질을 예측하는 중요한 변인이다. 수업참여는 단순히 수업에 출석한다는 의미를 넘어서 행동, 정서, 인지적 차원을 포함하는 복합적인 개념이다. 본 연구에서는 수업참여를 행동, 정서, 인지적 참여 등 다차원으로 정의하고 수업참여 하위요인과 학업성취와의 종단적 상호관계와 변화 양상을 탐색하고자 하였다. 이를 위해 국가수준 학업성취도 평가의 2012-2014년(중3-고2) 두 시점 종단 자료를 활용하였으며 자기회귀교차지연모형을 활용하였다. 그 결과 첫째, 수업참여 하위요인인 행동, 정서, 인지적 참여는 학업성취에 모두 유의한 정적 영향을 미쳤다. 뿐만 아니라 학업성취 또한 수업참여에 유의한 정적 영향을 미치고 있어, 수업참여와 학업성취는 상호 간 촉진시키는 변인임을 확인할 수 있었다. 둘째, 수업참여 하위요인 간 종단적 상호관계는 서로 유의하여 세 하위요인 간의 상보적 인과관계가 파악되었다. 특히, 인지와 행동적 참여 간 상호효과 크기가 가장 크게 나타났고, 정서적 참여는 상대적 효과크기가 작았으며 교과별로 차이를 나타냈다. 셋째, 수업참여 각 하위요인의 상대적 안정성은 중간 수준이었고, 잠재평균분석 결과 중학생에 비해 고등학생은 행동적 참여가 낮고 인지적 참여는 높은 것으로 나타났다. 이러한 연구결과를 바탕으로 교육적 시사점 및 제언을 추가하였다.Student engagement is one of the important factors predicting student achievement and quality of teaching and learning. In the present study, student engagement is defined as behavioral, emotional, and cognitive engagement. The purpose of this study is to investigate the longitudinal interplay between student engagement and achievement, and their stability across two years from 9 th to 11 th grades using an autoregressive cross-lagged (ARCL) model. The main results are as follows. First earlier behavioral, emotional and cognitive engagement had the significant effects on later achievement. Also, achievement was a significant predictor for three kinds of student engagement. It was confirmed that student engagement and achievement are factors that would promote reciprocal development. Second, we found the reciprocal causal relationships among the three kinds of student engagement. Especially, our findings showed that the longitudinal reciprocal effects between cognitive and behavioral engagement were the largest. The effect size of emotional engagement was relatively small and inconsistent across subjects. Third, the relative stability of student engagement was found to be moderate. According to the latent mean analyses where 9 th grade was used as a reference group, 11 th graders showed lower mean values in behavioral engagement but higher ones in cognitive engagement. Finally, implications of the study and recommendations for future research were discussed.

Context: Investing on Open Source Software (OSS) as a "code reuser", involves certain risks, such as the difficulty in understanding the level of OSS design quality Aim: We investigate the levels of design quality of OSS projects, across different application domains. Method: We conducted a case study, which is the most fitting research method for observing a phenomenon in its real context, which is active for a long period of time, and for which variables cannot be controlled. Results: We present the values for seven design quality metrics of 546 OSS projects, as well as the statistically significant differences across application domains. Conclusions: The results of the study suggest that OSS application domains correlate with several design quality characteristics, in the sense that projects within one application domain appear to have similar levels of design quality. In addition to that, the results reveal application domains with high and low levels of design quality.

The aspect-oriented programming (AOP) approach is supposed to enhance a system's features such as modularity, readability and simplicity. Owing to a better modularisation of crosscutting concerns, the developed system implementation would be less complex, and more readable. Thus, software development efficiency would increase, so the system would be created faster than its object-oriented programming (OOP) equivalent. An empirical study of a web-based system development is carried out to examine AOP against OOP approach with regard to software development efficiency and design quality. The study reveals that the AOP approach appears to be a fullfledged alternative to the pure OOP approach. Nevertheless, the impact of AOP on software development efficiency and design quality was not confirmed. In particular, it appeared that design quality metrics were not significantly associated with using AOP, instead of OOP. It is possible that the benefits of AOP will exceed the results obtained in the present study for experiments with larger number of subjects.

Artificial Intelligence (ai) systems are precious support for decision-making, with many applications also in the medical domain. The interaction between mds and ai enjoys a renewed interest following the increased possibilities of deep learning devices. However, we still have limited evidence-based knowledge of the context, design, and psychological mechanisms that craft an optimal human–ai collaboration. In this multicentric study, 21 endoscopists reviewed 504 videos of lesions prospectively acquired from real colonoscopies. They were asked to provide an optical diagnosis with and without the assistance of an ai support system. Endoscopists were influenced by ai (textsc {or}=3.05), but not erratically: they followed the ai advice more when it was correct (textsc {or}=3.48) than incorrect (textsc {or}=1.85). Endoscopists achieved this outcome through a weighted integration of their and the ai opinions, considering the case-by-case estimations of the two reliabilities. This Bayesian-like rational behavior allowed the human–ai hybrid team to outperform both agents taken alone. We discuss the features of the human–ai interaction that determined this favorable outcome.

This study aimed to investigate the relationships between college students’ behavioral and cognitive engagements while performing an online collective information searching (CIS) activity. The activity aimed to assist the students in utilizing a social bookmarking application to exploit the Internet in a collective manner. A group of 101 college students in Taiwan participated in the research procedure, and performed the CIS activity to glean quality online resources for the given search assignment. The actions taken and annotations and comments made during the activity were recorded as log data, and used as the main resource for later analyses of behavioral and cognitive engagements in the activity. Through cluster analysis of the students’ contributions to the CIS activity, four categories of behavioral engagement were identified, namely “Hitchhiker,” “Individualist,” “Active” and “Commentator,” to represent the students’ investments in performing the activity. Furthermore, to explore the students’ cognitive engagement in the activity, content analysis of the verbal transcripts of their annotations and comments was conducted based on the refined coding framework of the present study. The results of further cluster analysis revealed that the students’ cognitive engagement levels could be identified as “Deep” and “Surface.” Through comparison of their behavioral and cognitive engagements, the findings revealed that the students with “Active” behavioral engagement tended to exhibit a “Deep” level of cognitive engagement. It is therefore suggested that both behavioral and cognitive engagements are critical to participatory learning with practice in CIS activities.

Efficient Computation of Argumentation Semantics