Human-in-the-loop Research Articles

Distributed intelligence in industrial and automotive cyber-physical systems: a review.

Cyber-physical systems (CPSs) are evolving from individual systems to collectives of systems that collaborate to achieve highly complex goals, realizing a cyber-physical system of systems (CPSoSs) approach. They are heterogeneous systems comprising various autonomous CPSs, each with unique performance capabilities, priorities, and pursued goals. In practice, there are significant challenges in the applicability and usability of CPSoSs that need to be addressed. The decentralization of CPSoSs assigns tasks to individual CPSs within the system of systems. All CPSs should harmonically pursue system-based achievements and collaborate to make system-of-system-based decisions and implement the CPSoS functionality. The automotive domain is transitioning to the system of systems approach, aiming to provide a series of emergent functionalities like traffic management, collaborative car fleet management, or large-scale automotive adaptation to the physical environment, thus providing significant environmental benefits and achieving significant societal impact. Similarly, large infrastructure domains are evolving into global, highly integrated cyber-physical systems of systems, covering all parts of the value chain. This survey provides a comprehensive review of current best practices in connected cyber-physical systems and investigates a dual-layer architecture entailing perception and behavioral components. The presented perception layer entails object detection, cooperative scene analysis, cooperative localization and path planning, and human-centric perception. The behavioral layer focuses on human-in-the-loop (HITL)-centric decision making and control, where the output of the perception layer assists the human operator in making decisions while monitoring the operator's state. Finally, an extended overview of digital twin (DT) paradigms is provided so as to simulate, realize, and optimize large-scale CPSoS ecosystems.

A new AI-assisted data standard accelerates interoperability in biomedical research.

In this paper, we leveraged Large Language Models(LLMs) to accelerate data wrangling and automate labor-intensive aspects of data discovery and harmonization. This work promotes interoperability standards and enhances data discovery, facilitating AI-readiness in biomedical science with the generation of Common Data Elements (CDEs) as key to harmonizing multiple datasets. Thirty-one studies, various ontologies, and medical coding systems served as source material to create CDEs from which available metadata and context was sent as an API request to 4th-generation OpenAI GPT models to populate each metadata field. A human-in-the-loop (HITL) approach was used to assess quality and accuracy of the generated CDEs. To regulate CDE generation, we employed ElasticSearch and HITL to avoid duplicate CDEs and instead, added them as potential aliases for existing CDEs. The generated CDEs are foundational to assess the interoperability potential of datasets by determining how many data set column headers can be correctly mapped to CDEs as well as quantifying compliance with permissible values and data types. Subject matter experts reviewed generated CDEs and determined that 94.0% of generated metadata fields did not require manual revisions. Data tables from the Alzheimer's Disease Neuroimaging Initiative (ADNI) and the Global Parkinson's Genetic Program (GP2) were used as test cases for interoperability assessments. Column headers from all test cases were successfully mapped to generated CDEs at a rate of 32.4% via elastic search.The interoperability score, a metric for dataset compatibility to CDEs and other connected datasets, based on relevant criteria such as data field completeness and compliance with common harmonization standards averaged 53.8 out of 100 for test cases. With this project, we aim to automate the most tedious aspects of data harmonization, enhancing efficiency and scalability in biomedical research while decreasing activation energy for federated research.

Looping In: Exploring Feedback Strategies to Motivate Human Engagement in Interactive Machine Learning

This study investigates effective feedback mechanisms to maintain human engagement in interactive machine learning (IML) systems, focusing on social media platforms. We developed “Loop,” an IML system based on human-in-the-loop (HITL) principles that recommends content while encouraging users to report inaccuracies for model refinement. Loop implements three types of artificial intelligence (AI) feedback on user reports: (a) machine learning (ML)-centric, (b) personal-centric, and (c) community-centric feedback. In addition, we evaluated the relative effectiveness of these feedback types under two different task criticality scenarios: high and low. A user study with 30 participants was conducted to evaluate Loop through questionnaires and interviews. Results showed that participants preferred algorithmic improvements for personal benefit over altruistic contributions to the community, especially for low-criticality tasks. Furthermore, personal-centric feedback had a significant impact on user engagement and satisfaction. Our findings provide insights into the effectiveness of machine feedback in HITL-ML systems, contributing to the design of more engaging and effective IML interfaces. We discuss implications and strategies for encouraging proactive user engagement in HITL-ML-based systems, emphasizing the importance of tailored feedback mechanisms.

Implementation and Evaluations of Visual Scaffolding to Support Attentional Management Training

One approach to training is using scaffolding to assist the novice trainee in performing tasks while they develop fundamental skills. This paper describes the implementation and testing of visual scaffolding to support training of attentional management skills. The training tool was developed specifically for sensor operators of unmanned aerial systems (UAS) to learn attentional skills associated with managing the competing demands of UAS operational tasks. A key part of the training included attentional scaffolding—visual highlighting that was used to guide the trainee’s attention. Three different evaluations were conducted to assess performance while leveraging scaffolding: Two were performed with U.S. Navy and U.S. Coast Guard (USCG) sensor operators, and one was performed in a human-in-the-loop (HITL) study. In this paper, we detail the Navy and USCG studies, summarize the HITL study, and synthesize findings across the evaluations.

Human-in-the-Loop-A Deep Learning Strategy in Combination with a Patient-Specific Gaussian Mixture Model Leads to the Fast Characterization of Volumetric Ground-Glass Opacity and Consolidation in the Computed Tomography Scans of COVID-19 Patients.

Background/Objectives: The accurate quantification of ground-glass opacities (GGOs) and consolidation volumes has prognostic value in COVID-19 patients. Nevertheless, the accurate manual quantification of the corresponding volumes remains a time-consuming task. Deep learning (DL) has demonstrated good performance in the segmentation of normal lung parenchyma and COVID-19 pneumonia. We introduce a Human-in-the-Loop (HITL) strategy for the segmentation of normal lung parenchyma and COVID-19 pneumonia that is both time efficient and quality effective. Furthermore, we propose a Gaussian Mixture Model (GMM) to classify GGO and consolidation based on a probabilistic characterization and case-sensitive thresholds. Methods: A total of 65 Computed Tomography (CT) scans from 64 patients, acquired between March 2020 and June 2021, were randomly selected. We pretrained a 3D-UNet with an international dataset and implemented a HITL strategy to refine the local dataset with delineations by teams of medical interns, radiology residents, and radiologists. Following each HITL cycle, 3D-UNet was re-trained until the Dice Similarity Coefficients (DSCs) reached the quality criteria set by radiologists (DSC = 0.95/0.8 for the normal lung parenchyma/COVID-19 pneumonia). For the probabilistic characterization, a Gaussian Mixture Model (GMM) was fitted to the Hounsfield Units (HUs) of voxels from the CT scans of patients with COVID-19 pneumonia on the assumption that two distinct populations were superimposed: one for GGO and one for consolidation. Results: Manual delineation of the normal lung parenchyma and COVID-19 pneumonia was performed by seven teams on 65 CT scans from 64 patients (56 ± 16 years old (μ ± σ), 46 males, 62 with reported symptoms). Automated lung/COVID-19 pneumonia segmentation with a DSC > 0.96/0.81 was achieved after three HITL cycles. The HITL strategy improved the DSC by 0.2 and 0.5 for the normal lung parenchyma and COVID-19 pneumonia segmentation, respectively. The distribution of the patient-specific thresholds derived from the GMM yielded a mean of -528.4 ± 99.5 HU (μ ± σ), which is below most of the reported fixed HU thresholds. Conclusions: The HITL strategy allowed for fast and effective annotations, thereby enhancing the quality of segmentation for a local CT dataset. Probabilistic characterization of COVID-19 pneumonia by the GMM enabled patient-specific segmentation of GGO and consolidation. The combination of both approaches is essential to gain confidence in DL approaches in our local environment. The patient-specific probabilistic approach, when combined with the automatic quantification of COVID-19 imaging findings, enhances the understanding of GGO and consolidation during the course of the disease, with the potential to improve the accuracy of clinical predictions.

Human-in-the-Loop Trajectory Optimization Based on sEMG Biofeedback for Lower-Limb Exoskeleton.

Lower-limb exoskeletons (LLEs) can provide rehabilitation training and walking assistance for individuals with lower-limb dysfunction or those in need of functionality enhancement. Adapting and personalizing the LLEs is crucial for them to form an intelligent human-machine system (HMS). However, numerous LLEs lack thorough consideration of individual differences in motion planning, leading to subpar human performance. Prioritizing human physiological response is a critical objective of trajectory optimization for the HMS. This paper proposes a human-in-the-loop (HITL) motion planning method that utilizes surface electromyography signals as biofeedback for the HITL optimization. The proposed method combines offline trajectory optimization with HITL trajectory selection. Based on the derived hybrid dynamical model of the HMS, the offline trajectory is optimized using a direct collocation method, while HITL trajectory selection is based on Thompson sampling. The direct collocation method optimizes various gait trajectories and constructs a gait library according to the energy optimality law, taking into consideration dynamics and walking constraints. Subsequently, an optimal gait trajectory is selected for the wearer using Thompson sampling. The selected gait trajectory is then implemented on the LLE under a hybrid zero dynamics control strategy. Through the HITL optimization and control experiments, the effectiveness and superiority of the proposed method are verified.

A Novel Personalized Strategy for Hip Joint Flexion Assistance Based on Human Physiological State.

Soft exosuits have emerged as potent assistive tools for walking support and rehabilitation training. However, most existing soft exosuit systems rely on preset assistance modes, which may not accurately align with individual physiological states and movement requirements, leading to variable user experiences and efficacy. While existing human-in-the-loop (HIL) research predominantly focuses on optimizing metabolic cost and torque difference parameters, there is a notable absence of real-time monitoring methods that closely reflect the human body's physiological state and strategies that dynamically indicate walking efficiency. Motivated by this, we developed a novel personalized power-assist system. This system optimizes the power-assist output of the hip joint by monitoring the user's physiological and motion signals in real time, including heart rate (HR), blood oxygen saturation (SpO2), and inertial measurement unit (IMU) data, to assist hip flexion based on feedback. The findings from a metabolic expenditure trial demonstrate that the innovative soft exosuit, which is based on a Physiological State Monitoring Control (PSMC) system, achieves a reduction of 7.81% in metabolic expenditure during treadmill walking at a speed of 3.5 km/h compared to walking without the assistance of the exosuit. Additionally, during continuous exercise with varying intensities, the metabolic consumption level is reduced by 5.1%, 5.8%, and 8.2% at speeds of 2, 4, and 6 km per hour, respectively. These results support the design of a novel hip flexion-assisting soft exosuit, demonstrating that applying different assistance forces in consideration of different physiological states is a reasonable approach to reducing metabolic consumption.

Using Dynamic Bayesian Optimization to Induce Desired Effects in the Presence of Motor Learning: a Simulation Study.

Human-in-the-loop (HIL) optimization is a control paradigm used for tuning the control parameters of human-interacting devices while accounting for variability among individuals. A limitation of state-of-the-art HIL optimization algorithms such as Bayesian Optimization (BO) is that they assume that the relationship between control parameters and user response does not change over time. BO can be modified to account for the dynamics of the user response by implementing time into the kernel function, a method known as Dynamic Bayesian Optimization (DBO). However, it is unknown if DBO outperforms BO when the human response is characterized by models of human motor learning. In this work, we simulated runs of HIL optimization using BO and DBO towards establishing if DBO is a suitable paradigm for HIL optimization in the presence of motor learning. Simulations were conducted assuming either purely time-dependent participant responses, or assuming that responses would arise from state-space models of motor learning capable of describing both adaptation and use-dependent learning behavior. Statistical comparisons indicated that DBO was never inferior to BO, and, after a certain number of iterations, generally outperformed BO in convergence to optimal inputs and outputs. The number of iterations beyond which DBO was superior to BO occurred earlier when the input-output relationship of the simulated responses was more dynamic. Our results suggest that DBO may improve the performance of HIL optimization over BO when a sufficient number of iterations can be evaluated to accurately distinguish between unstructured variability (noise) and learning.

Evaluating local open-source large language models for data extraction from unstructured reports on mechanical thrombectomy in patients with ischemic stroke

BackgroundA study was undertaken to assess the effectiveness of open-source large language models (LLMs) in extracting clinical data from unstructured mechanical thrombectomy reports in patients with ischemic stroke caused by...

Human-in-the-Loop Behavior Modeling via an Integral Concurrent Adaptive Inverse Reinforcement Learning.

One goal of artificial intelligence (AI) research is to teach machines how to learn from humans, such that they can perform a certain task in a natural human-like way. In this article, an online adaptive inverse reinforcement learning (IRL) approach to human behavior modeling is proposed to enhance machine intelligence for a class of linear human-in-the-loop (HiTL) systems using the state data only, where the human behavior is described by a linear quadratic optimal control model with an unknown weighting matrix for the quadratic cost function. First, an integral concurrent adaptive law is developed to learn the human feedback gain matrix online using the demonstrated state data only, which removes the persistent excitation (PE) conditions required by traditional adaptive estimation approaches and thus is more in line with real applications. Then, with the learned feedback gain matrix, the IRL problem is formulated as a linear matrix inequality (LMI) optimization problem, which can be efficiently solved to retrieve the weighting matrix of the human cost function. Finally, a simulation example is provided to illustrate the effectiveness of the proposed approach.

HEX: Human-in-the-loop explainability via deep reinforcement learning

The use of machine learning (ML) models in decision-making contexts, particularly those used in high-stakes decision-making, are fraught with issue and peril since a person – not a machine – must ultimately be held accountable for the consequences of decisions made using such systems. Machine learning explainability (MLX) promises to provide decision-makers with prediction-specific rationale, assuring them that the model-elicited predictions are made for the right reasons and are thus reliable. Few works explicitly consider this key human-in-the-loop (HITL) component, however. In this work we propose HEX, a human-in-the-loop deep reinforcement learning approach to MLX. HEX incorporates 0-distrust projection to synthesize decider-specific explainers that produce explanations strictly in terms of a decider’s preferred explanatory features using any classification model. Our formulation explicitly considers the decision boundary of the ML model in question using a proposed explanatory point mode of explanation, thus ensuring explanations are specific to the ML model in question. We empirically evaluate HEX against other competing methods, finding that HEX is competitive with the state-of-the-art and outperforms other methods in human-in-the-loop scenarios. We conduct a randomized, controlled laboratory experiment utilizing actual explanations elicited from both HEX and competing methods. We causally establish that our method increases decider’s trust and tendency to rely on trusted features.

Revolutionizing defect recognition in hard metal industry through AI explainability, human-in-the-loop approaches and cognitive mechanisms

Defect detection is one of the main areas that Industry 4.0 concepts like automation, IoT, digitization and AI aimed to provide solutions. In this work, a platform that extends the aforementioned concepts with ones coming from Industry 5.0 like reconciliation and collaboration between humans and machines is introduced. The proposed platform provides defect detection and localization services for hard metal industry by extending current AI solutions with exponentially growing technologies such as interpretable and explainable AI (XAI), human-in-the-loop (HITL) approaches and cognitive retraining mechanisms. In particular, the platform, that is built on a micro-service architecture to enable its software sustainability, is powered by machine and deep learning models for defect detection and localization. These models are able to recognize the defects, locate them with high precision and present them to end users so to maximize resiliency in quality inspection processes. Alongside the state-of-the-art AI algorithms that are utilized by the platform, the human in the loop mechanisms and interfaces that enable human experts to inject their knowledge for AI models training are considered as key innovative aspects of this work. To enhance further the human and AI collaboration, XAI mechanisms are included in the platform,to enable the experts to gain useful insights regarding the operation of the AI models in order to be able to provide further feedback that can trigger on-the-fly platform’s cognition mechanisms to improve models’ accuracy and performance.

A versatile framework for analyzing galaxy image data by incorporating Human-in-the-loop in a large vision model* *The authors acknowledge the support from the National Natural Science Foundation of China (Grant Nos. 12173027, 12303105, 12173062), the National Key R & D Program of China (Grant Nos. 2023YFF0725300, 2022YFF0503402), the Science Research Grants from the Square Kilometre Array (SKA) (2020SKA0110100), the Science

The exponential growth of astronomical datasets provides an unprecedented opportunity for humans to gain insight into the Universe. However, effectively analyzing this vast amount of data poses a significant challenge. In response, astronomers are turning to deep learning techniques, but these methods are limited by their specific training sets, leading to considerable duplicate workloads. To overcome this issue, we built a framework for the general analysis of galaxy images based on a large vision model (LVM) plus downstream tasks (DST), including galaxy morphological classification, image restoration, object detection, parameter extraction, and more. Considering the low signal-to-noise ratios of galaxy images and the imbalanced distribution of galaxy categories, we designed our LVM to incorporate a Human-in-the-loop (HITL) module, which leverages human knowledge to enhance the reliability and interpretability of processing galaxy images interactively. The proposed framework exhibits notable few-shot learning capabilities and versatile adaptability for all the abovementioned tasks on galaxy images in the DESI Legacy Imaging Surveys. In particular, for the object detection task, which was trained using 1000 data points, our DST in the LVM achieved an accuracy of 96.7%, while ResNet50 plus Mask R-CNN reached an accuracy of 93.1%. For morphological classification, to obtain an area under the curve (AUC) of ~0.9, LVM plus DST and HITL only requested 1/50 of the training sets that ResNet18 requested. In addition, multimodal data can be integrated, which creates possibilities for conducting joint analyses with datasets spanning diverse domains in the era of multi-messenger astronomy.

Human-in-the-Loop Optimization of Knee Exoskeleton Assistance for Minimizing User's Metabolic and Muscular Effort.

Lower limb exoskeletons have the potential to mitigate work-related musculoskeletal disorders; however, they often lack user-oriented control strategies. Human-in-the-loop (HITL) controls adapt an exoskeleton's assistance in real time, to optimize the user-exoskeleton interaction. This study presents a HITL control for a knee exoskeleton using a CMA-ES algorithm to minimize the users' physical effort, a parameter innovatively evaluated using the interaction torque with the exoskeleton (a muscular effort indicator) and metabolic cost. This work innovates by estimating the user's metabolic cost within the HITL control through a machine-learning model. The regression model estimated the metabolic cost, in real time, with a root mean squared error of 0.66 W/kg and mean absolute percentage error of 26% (n = 5), making faster (10 s) and less noisy estimations than a respirometer (K5, Cosmed). The HITL reduced the user's metabolic cost by 7.3% and 5.9% compared to the zero-torque and no-device conditions, respectively, and reduced the interaction torque by 32.3% compared to a zero-torque control (n = 1). The developed HITL control surpassed a non-exoskeleton and zero-torque condition regarding the user's physical effort, even for a task such as slow walking. Furthermore, the user-specific control had a lower metabolic cost than the non-user-specific assistance. This proof-of-concept demonstrated the potential of HITL controls in assisted walking.

Embodying Human-Like Modes of Balance Control Through Human-In-the-Loop Dyadic Learning

In this paper, we explore how humans and AIs trained to perform a virtual inverted pendulum (VIP) balancing task converge and differ in their learning and performance strategies. We create a visual analogue of disoriented IP balancing, as may be experienced by pilots suffering from spatial disorientation, and train AI models on data from human subjects performing a real-world disoriented balancing task. We then place the trained AI models in a dyadic human-in-the-loop (HITL) training setting. Episodes in which human subjects disagreed with AI actions were logged and used to fine-tune the AI model. Human subjects then performed the task while being given guidance from pretrained and dyadically fine-tuned versions of an AI model. We examine the effects of HITL training on AI performance, AI guidance on human performance, and the behavior patterns of human subjects and AI models during task performance. We find that in many cases, HITL training improves AI performance, AI guidance improves human performance, and after dyadic training the two converge on similar behavior patterns.

Understanding and Addressing AI Hallucinations in Healthcare and Life Sciences

Purpose: This paper investigates the phenomenon of "AI hallucinations" in healthcare and life sciences, where large language models (LLMs) produce outputs that, while coherent, are factually incorrect, irrelevant, or misleading. Understanding and mitigating such errors is critical given the high stakes of accurate and reliable information in healthcare and life sciences. We classify hallucinations into three types input-conflicting, context-conflicting, and fact-conflicting and examine their implications through real-world cases. Methodology: Our methodology combines the Fact Score, Med-HALT, and adversarial testing to evaluate the fidelity of AI outputs. We propose several mitigation strategies, including Retrieval-Augmented Generation (RAG), Chain-of-Verification (CoVe), and Human-in-the-Loop (HITL) systems, to enhance model reliability. Findings: As artificial intelligence continues to permeate various sectors of society, the issue of hallucinations in AI-generated text poses significant challenges, especially in contexts where precision and reliability are paramount. This paper has delineated the types of hallucinations commonly observed in AI systems input-conflicting, context-conflicting, and fact-conflicting and highlighted their potential to undermine trust and efficacy in critical domains such as healthcare and legal proceedings. Unique contribution to theory, policy and practice: This study's unique contribution lies in its comprehensive analysis of AI hallucinations' types and impacts and the development of robust controls that advance theoretical understanding, practical application, and policy formulation in AI deployment. These efforts aim to foster safer, more effective AI integration across healthcare and life sciences sectors

Advancing human–robot collaboration in handcrafted manufacturing: cobot-assisted polishing design boosted by virtual reality and human-in-the-loop

Industry 5.0 envisions a future where seamless collaboration between humans and robots enhances efficiency, innovation, and coevolution. While collaborative robots have found widespread applications in manufacturing, particularly in tasks like pick-and-place and assembly, their integration into handcrafted manufacturing processes presents unique challenges. This article focuses on advancing technology in the less-explored field of cobot-assisted handcrafted manufacturing, specifically in the fashion industry, with a priority on reducing work-related risks. In handcrafted processes, which often involve intricate and artistic work, cobots face challenges related to nuanced decision-making, adaptability to customizations, and the need for precise manual dexterity. The study delves into the cobot-assisted polishing of leather shoes, addressing issues associated with product delicacy, process and knowledge formalization, versatility, and integration into existing manufacturing processes. To overcome these challenges, the research proposes the application of cobots in the initial polishing phase, which is the most physically demanding, allowing artisans to focus on finalization, quality control, and process supervision. The study also applies the concept of human-in-the-loop (HITL) and virtual reality simulation to optimize collaboration, ensuring safety, ergonomics, and efficiency. The article contributes to the scientific and industrial communities by pioneering the study of collaborative robotics in craftsmanship, successfully implementing human–robot collaboration (HRC) in an industrial setting, demonstrating the effectiveness of virtual simulation and HITL, and prioritizing human factors throughout the design and development of HRC. The insights gained from this research are crucial for achieving practical solutions in industrial environments while aligning with the performance objectives of companies and workers’ well-being.

Development of data anomaly classification for structural health monitoring based on iterative trimmed loss minimization and human-in-the-loop learning

Huge amounts of data can be generated during long-term monitoring performed by structural health monitoring (SHM) and structural integrity management applications. Monitoring data can be corrupted, and the presence of abnormal data can distort information during signal processing, extract incorrect characteristics during system identification, produce false conclusions during damage detection, and ultimately lead to misjudgment of structural conditions during diagnosis and prognosis. Therefore, developing effective techniques to autonomously detect and classify anomalies becomes necessary and significant. Generally, conventional physics-based strategies can be straightforward, but their performance highly depends on prior knowledge of measurement. Recently, data-driven methods leveraging machine learning (ML) have been used to directly handle the task. This study proposes an ML-based classifier and improves it by incorporating the human-in-the-loop (HITL) learning. The classifier is built on a shallow neural network with high performance to address potential online or real-time applications for long-term monitoring. First, a field monitoring dataset is introduced, and various anomalies are defined to investigate the effectiveness. To further enhance the performance of the proposed classifier, the mislabels in the monitoring dataset are examined, and a correction technique is performed. Additionally, HITL ML is developed to overcome the disadvantages of the conventional correction technique. As a result, the proposed procedure can improve both the classifier and the field dataset, and the proposed classifier can now function as a fundamental component in achieving a continuous and autonomous SHM system.

Towards interactive reinforcement learning with intrinsic feedback

Reinforcement learning (RL) and brain–computer interfaces (BCI) have experienced significant growth over the past decade. With rising interest in human-in-the-loop (HITL), incorporating human input with RL algorithms has given rise to the sub-field of interactive RL. Adjacently, the field of BCI has long been interested in extracting informative brain signals from neural activity for use in human–computer interactions. A key link between these fields lies in the interpretation of neural activity as feedback such that interactive RL approaches can be employed. We denote this new and emerging medium of feedback as intrinsic feedback. Despite intrinsic feedback’s ability to be conveyed automatically and even unconsciously, proper exploration surrounding this key link has largely gone unaddressed by both communities. Thus, to help facilitate a deeper understanding and a more effective utilization, we provide a tutorial-style review covering the motivations, approaches, and open problems of intrinsic feedback and its foundational concepts.

Explainable Deep Learning: A Visual Analytics Approach with Transition Matrices

The non-transparency of artificial intelligence (AI) systems, particularly in deep learning (DL), poses significant challenges to their comprehensibility and trustworthiness. This study aims to enhance the explainability of DL models through visual analytics (VA) and human-in-the-loop (HITL) principles, making these systems more transparent and understandable to end users. In this work, we propose a novel approach that utilizes a transition matrix to interpret results from DL models through more comprehensible machine learning (ML) models. The methodology involves constructing a transition matrix between the feature spaces of DL and ML models as formal and mental models, respectively, improving the explainability for classification tasks. We validated our approach with computational experiments on the MNIST, FNC-1, and Iris datasets using a qualitative and quantitative comparison criterion, that is, how different the results obtained by our approach are from the ground truth of the training and testing samples. The proposed approach significantly enhanced model clarity and understanding in the MNIST dataset, with SSIM and PSNR values of 0.697 and 17.94, respectively, showcasing high-fidelity reconstructions. Moreover, achieving an F1m score of 77.76% and a weighted accuracy of 89.38%, our approach proved its effectiveness in stance detection with the FNC-1 dataset, complemented by its ability to explain key textual nuances. For the Iris dataset, the separating hyperplane constructed based on the proposed approach allowed for enhancing classification accuracy. Overall, using VA, HITL principles, and a transition matrix, our approach significantly improves the explainability of DL models without compromising their performance, marking a step forward in developing more transparent and trustworthy AI systems.