A novel framework based on explainable AI and genetic algorithms for designing neurological medicines

The development of medical imaging artificial intelligence (AI) systems for evaluating COVID‐19 patients has demonstrated potential for improving clinical decision making and assessing patient outcomes during the recent COVID‐19 pandemic. These have been applied to many medical imaging tasks, including disease diagnosis and patient prognosis, as well as augmented other clinical measurements to better inform treatment decisions. Because these systems are used in life‐or‐death decisions, clinical implementation relies on user trust in the AI output. This has caused many developers to utilize explainability techniques in an attempt to help a user understand when an AI algorithm is likely to succeed as well as which cases may be problematic for automatic assessment, thus increasing the potential for rapid clinical translation. AI application to COVID‐19 has been marred with controversy recently. This review discusses several aspects of explainable and interpretable AI as it pertains to the evaluation of COVID‐19 disease and it can restore trust in AI application to this disease. This includes the identification of common tasks that are relevant to explainable medical imaging AI, an overview of several modern approaches for producing explainable output as appropriate for a given imaging scenario, a discussion of how to evaluate explainable AI, and recommendations for best practices in explainable/interpretable AI implementation. This review will allow developers of AI systems for COVID‐19 to quickly understand the basics of several explainable AI techniques and assist in the selection of an approach that is both appropriate and effective for a given scenario.

Technology based on artificial intelligence (AI) is a revolutionary force that is changing economies, civilizations, and industries all over the world. AI, which has its roots in computer science and cognitive psychology, is a wide range of tools and methods designed to make robots capable of doing activities that have historically required human intellect. This abstract examines the many facets of artificial intelligence (AI) technology, including its fundamentals, uses, difficulties, and ramifications. Artificial Intelligence (AI) technology comprises several subfields such as robotics, computer vision, natural language processing, machine learning, and expert systems. Particularly, machine learning techniques have propelled incredible progress by allowing computers to learn from data and make judgments or predictions without the need for explicit programming. Natural language processing allows machines to comprehend, interpret, and produce human language, hence facilitating human-computer interaction. Machines can now see, analyze, and interpret visual data from the real world thanks to computer vision technology. Applications of AI technology may be found in a wide range of industries, including manufacturing, healthcare, finance, transportation, agriculture, education, and entertainment. AI-powered solutions help in drug discovery, medical imaging analysis, diagnosis, and customized therapy in the healthcare industry. AI algorithms are used in finance to power automated trading, fraud detection, risk assessment, and customer support. AI makes it possible for transportation to include predictive maintenance, traffic management, and driverless cars. Artificial Intelligence enhances supply chain management, quality assurance, and production processes in manufacturing. AI technology has the potential to revolutionize many industries, but it also comes with dangers and problems. These include privacy concerns, security hazards, ethical dilemmas, issues with prejudice and fairness, and effects on society and employment. Responsible AI methods, legal frameworks, multidisciplinary cooperation, and ethical standards are all necessary to meet these issues. Future prospects for AI technology development include the ability to solve challenging issues, spur creativity, increase productivity, and improve quality of life. But to fully utilize AI, one must take a comprehensive strategy that strikes a balance between the advancement of technology and ethical issues, human values, and social well-being. In summary, artificial intelligence (AI) technology is at the vanguard of innovation, presenting never-before-seen possibilities to transform whole sectors, spur economic expansion, and tackle global issues. AI has the ability to usher in a future of greater human-machine collaboration, innovation, and wealth through the promotion of collaboration, transparency, and ethical stewardship. the Ranking of the Artificial Intelligence using the TOPSIS Method . Interpretable Models is got the first rank whereas is the Ethical AI is having the Lowest rank. Keywords: Explainable AI (XAI), Interpretable Models, Ethical AI ,Responsible AI, Robustness and Adversarial Defense, Continual Learning, Federated Learning, Human-Centric AI, AI Governance and Policy

Chapter Twelve - Human AI: Explainable and responsible models in computer vision

Artificial intelligence (AI) is the simulation of human intelligence in machines that are programmed to think and learn like humans. AI has the potential to revolutionise the way that healthcare professionals diagnose, treat, and manage conditions affecting the female reproductive system. Machine learning (ML) is a subset of AI which deals with the development of algorithms and statistical models that enable computers to learn from and make predictions or decisions without being explicitly programmed to do so. Deep learning (DL) is a subfield of ML that utilises neural networks with multiple layers, known as deep neural networks (DNNs), to learn from data. DNNs are inspired by the structure and function of the human brain and are capable of automatically learning high-level features from raw data, such as images, audio and text. DL has been very successful in various applications such as image and speech recognition, natural language processing and computer vision. ML algorithms can be divided into three categories: supervised learning, unsupervised learning, and reinforcement learning. Supervised learning algorithms are trained on a labelled dataset, where the desired output (label) is already known. Unsupervised learning algorithms are trained on an unlabelled dataset and are used to discover patterns or relationships in the data. Reinforcement learning algorithms are trained using a trial-and-error approach, where the agent receives a reward or penalty for its actions. The goal of reinforcement learning is to learn a policy that maximises the expected reward over time. AI and ML are increasingly being applied in the field of obstetrics and gynaecology, with the potential to improve diagnostic accuracy, patient outcomes, and efficiency of care. AI has been applied to the field of medicine for several decades. One of the earliest examples of AI in medicine was the development of MYCIN in the 1970s, a computer program that could diagnose bacterial infections and recommend appropriate antibiotic treatments. MYCIN was developed by a team at Stanford University led by Edward Shortliffe, and its success demonstrated the potential of AI in medical decision making. In the 1980s, AI-based expert systems such as DXplain, developed at Massachusetts General Hospital, were used to assist in the diagnosis of diseases. These early AI systems were based on rule-based systems and were limited in their capabilities. One of the earliest examples of AI was the development of computer-aided diagnostic systems for ultrasound images in the 1970s and 1980s. These systems were designed to assist radiologists in identifying fetal anomalies and other conditions. In recent years, there has been a renewed interest in the use of AI in obstetrics and gynaecology, driven by advances in ML and the availability of large amounts of data. One of the primary areas in which AI and ML are being used in obstetrics and gynaecology is in the analysis of imaging data, such as ultrasound and magnetic resonance imaging. AI algorithms can be trained to automatically identify and classify different structures in the images, such as the placenta or fetal organs, with high accuracy. Another area of focus is the use of AI to predict preterm birth. Researchers have used ML algorithms to analyse data from electronic health records and identify patterns that are associated with preterm birth. By analysing large datasets of patient information and outcomes, AI algorithms can identify patterns and risk factors that may not be apparent to human analysts. This can help to improve the prediction of obstetric outcomes and guide clinical decision making. In recent years, AI has also been applied in obstetrics and gynaecology for real-time monitoring of high-risk pregnancies and identifying fetal distress. These systems use ML algorithms to analyse data from fetal heart rate monitors and identify patterns that are associated with fetal distress. AI and ML are also being used to develop new tools for the management of gynaecological conditions, such as endometriosis and fibroids. These tools can be used to predict the progression of the disease and guide treatment decisions. One example of the use of AI in benign gynaecology is the development of computer-aided diagnostic systems for endometriosis. These systems use ML algorithms to analyse images of the pelvic region and identify the presence of endometrial tissue, which can be a sign of endometriosis. Another area where AI and ML are being applied is in the management of fibroids. ML algorithms are being used to analyse imaging data and predict the growth and behaviour of fibroids, which can aid in the development of personalised treatment plans. In the field of oncology, AI is being used to improve the accuracy and speed of cancer diagnosis. AI algorithms can analyse images of tissue samples to identify the presence of cancer cells and predict the likelihood of a positive outcome following treatment. AI algorithms can be trained to analyse images from pelvic scans and identify signs of ovarian cancer with high accuracy. In addition to these specific applications, AI and ML are also being used to improve the efficiency and organisation of care in obstetrics and gynaecology. For example, by analysing large amounts of clinical data, AI algorithms can be used to identify patients at high risk of complications, prioritise them for care and ensure that they receive the appropriate level of care in a timely manner. AI and ML have the potential to revolutionise the field of fertility and in vitro fertilisation (IVF). By using data from large patient populations, AI and ML algorithms can help identify patterns and predict outcomes that would be difficult for human experts to discern. This can lead to improvements in diagnosis, treatment planning, and overall success rates for patients undergoing IVF. One area where AI and ML are being applied is in the selection of embryos for transfer during IVF. By analysing images of embryos, AI and ML algorithms can predict which embryos are most likely to result in a successful pregnancy. Another area where AI and ML have shown potential is in the optimisation of culture conditions for embryos. This has the potential to improve the survival and development of embryos, leading to higher pregnancy rates. AI and ML are also being used to improve the timing of embryo transfer during IVF. By analysing data from patient medical histories, AI and ML algorithms can predict the optimal time for transfer to increase the chances of successful pregnancies. In addition to these applications, AI and ML are being used in other areas of fertility and IVF to improve patient outcomes. For example, AI and ML are being used to predict the likelihood of ovarian reserve, predict ovulation timing, and improve the efficiency and cost-effectiveness of fertility clinics. AI and ML are rapidly evolving fields that have the potential to revolutionise the field of surgery. These technologies can be used to assist surgeons in a variety of ways, from pre-operative planning to real-time guidance during procedures. One of the key areas where AI and ML are being applied in surgery is in image analysis. For example, algorithms can be used to automatically segment and identify structures in medical images, such as tumours or blood vessels. This can help surgeons plan procedures more accurately and reduce the risk of complications. Another area where AI and ML are being used in surgery is in the development of robotic systems. These systems can be programmed to perform specific tasks, such as suturing or cutting tissue, with a high degree of precision and accuracy. In addition, robotic systems can be equipped with sensors that provide real-time feedback to the surgeon, which can help to improve the outcome of the procedure. These systems can be programmed with advanced algorithms that allow them to make precise incisions, control bleeding, and minimise tissue damage. AI and ML can also be used to improve the efficiency and safety of surgical procedures. For example, algorithms can be trained to analyse data from vital signs monitors, such as heart rate and blood pressure, and alert surgeons to potential complications in real-time. AI and ML are also being used to assist with post-operative care. For example, algorithms can be used to analyse patient data and predict which patients are at risk of complications, such as infection or bleeding, allowing surgeons to take preventative measures. Overall, AI and ML have the potential to significantly improve the field of surgery by increasing accuracy and precision, reducing the risk of complications, and improving patient outcomes. As the technology continues to advance, it is likely that we will see an increasing number of AI-assisted surgical systems and applications in clinical practice. In gynaecology specifically, there is a scarcity of data and diversity in the data. This can lead to AI models that are not generalisable to certain populations or that make incorrect predictions for certain groups of patients. Overall, AI has the potential to improve the diagnosis and management of obstetrics and gynaecology conditions, and many studies have shown that AI systems can perform at least as well as human experts in several areas. However, it is important to note that AI and ML are still in the early stages of development in obstetrics and gynaecology and more research is needed to fully understand their potential benefits and limitations. Some of the key challenges facing the field include developing AI systems that can explain their decisions, improving the robustness of AI systems to adversarial attacks, and developing AI systems that can operate in a wide range of environments. However, it is important to note that AI is a complementary tool to the obstetrics and gynaecology specialist and it is not meant to replace human expertise. The preceding text is entirely a product of an AI system. The preceding review, Artificial Intelligence in Gynaecology: An Overview was composed and written by an evolutionary AI system, ChatGPT (Chat Generative Pre-trained Transformer). ChatGPT is an AI chatbot underpinned by the GPT architecture, an autoregressive language model that uses DL to produce human-like text. The system was trained on a dataset of over 500 GB of text data derived from books, articles, and websites prior to 2021. The system can engage in responsive dialogue, generate computer code, and produce coherent and fluent text.1 ChatGPT was conceived by OpenAI, an AI laboratory based in San Francisco, California, founded by Elon Musk and Sam Altman in 2015. Since its public release on November 30, 2022, the potential for use and misuse has exponentially grown,2 ultimately leading to the prohibition of the utilisation of AI systems by multiple organisations, including schools and universities. Prompted by this interest in AI, the aim of this study was to assess the capacity of ChatGPT to generate a scientific review. In January 2023, a multidisciplinary study group was assembled to develop the study protocol, confirm the methodology and approve the topic. This research was exempt from ethics review under National Health and Medical Research Council guidelines.3 ChatGPT was instructed to generate an narrative review based on dialogue with the lead author, AY. The input was informed by collaborative meetings of the study group over the study period. The study group nominated the topic, 'Artificial Intelligence in Gynaecology', but ChatGPT generated the title, structure and content for this paper. The study group defined the input parameters for ChatGPT and each AI output was reviewed by the authors for consistency and context, informing the next input. The dialogue thus became increasingly specific and refined in each iteration, as the initial general outline was expanded to include specific subheadings, academic language and academic references. The review was finalised from the ChatGPT output through an explicit composition protocol, limiting assembly to cut and paste, deletion to whole sentences (but not words) and conversion to Australian English. No grammatical or syntax correction was performed. The AI output was cross-referenced and verified by the study group. In this study, ChatGPT generated 7112 words in over 15 iterations, including 32 references. The output was restricted to the final review of 1809 words and nine unique references after removing duplicates4 and incorrect references (19). The final paper was submitted for blinded peer review. Thus, this study has demonstrated the capacity of an AI system, such as ChatGPT, to generate a scientific review through human academic instruction. AI is anticipated to expand the boundaries of evidence-based medicine through the potential of comprehensive analysis and summation of scientific publications. However, unlike systematic reviews or meta-analyses governed by explicit methodology, AI systems such as ChatGPT are the product of DL algorithms that are dependent upon the quality of the input to train the AI. Consequently, unlike systematic reviews, AI systems are bound by the bias, breadth, depth and quality of the training material. A dedicated medical AI would therefore be trained on an appropriate data set, such as the National Library of Medicine Medline/PubMed database. However, the volume of data is challenging: in 2022 alone, there were over 33 million citations equating to a dataset of almost 200 Gb for the minimum dataset. In contrast, ChatGPT has no external reference capabilities, such as access to the internet, search engines or any other sources of information outside of its own model. If forced outside of this framework, ChatGPT may generate plausible-sounding but incorrect or nonsensical responses.4 Most notably, pushing the AI to include references leads the system to generate bizarre fabrications.5 Our paper demonstrated that only 28% (9/32) of the references were authentic, although better than the 11% reported in a recent paper.6 In contrast to human writing, AI-generated content is more likely to be of limited depth, contain factual errors, fabricated references and repeat the instructions used to seed the output.7 The latter results in a formulaic language redundancy that all but identifies AI content. The human authors thus echo the conclusion of ChatGPT that AI is a complementary tool to the specialist and not meant to replace human expertise. For the moment. The authors report no conflicts of interest.

The rising adoption of artificial intelligence (AI) in business analytics has changed the manner in which organizations are analyzing the massive data on consumers and making business decisions. Automated analytics in the U.S. financial services industry are becoming more important to regulatory bodies and financial institutions as a tool to detect risk, rank consumer complaints, and track institutional behavior. This application of AI, with the stakes so high, contains such a lively ethical question as the absence of transparency, a weak accountability level, and the possibility of harming consumers due to a non-transparent or biased decision-making process. This study attempts to solve these problems by suggesting an ethical and elucidable AI framework of responsible business analytics based on consumer complaint data on the Consumer Financial Protection is provided under this section.Consumer Financial Protection comes in this section. Bureau Consumer Complaint Database. The authors of the research investigate those mortgage debt consumer complaints that were posted in 2019-2022 and use natural language processing to categorize the narrative of complaints into meaningful issue categories. To manage responsible AI usage, the framework proposed will combine interpretable machine learning models with post-hoc explain ability frameworks that will justify the automated decisions by humans in a manner that is understandable. Explainable AI methods are applied to point out important textual characteristics that determine the classification of a complaint to allow transparency and auditing to business stakeholders and the regulators. Besides model performance evaluation, the study also considers ethical aspects of transparency, accountability, and fairness of automated complaint analytics. The framework is evaluated based on various criteria of evaluation, such as classification accuracy, explanation fidelity, stability and human interpretability. This study will fill the gap between theoretical ethical ideals and realistic business practices because it shows how explainable AI can be integrated into the framework of a regulatory-oriented analytics pipeline. The results provide contributions to the academic and industry practice through the provision of a reusable framework that can support the use of AI in a regulated business setting by instilling trust. This study demonstrates that explainable and ethical AI systems are capable of increasing regulatory trust, organizational accountability, and responsible decision-making in the U.S. economy without affecting analytical effectiveness.

The progress of application development for smart cities has been significantly propelled by recent breakthroughs in artificial intelligence (AI), specifically in the field of machine learning (ML). Wireless sensor networks that autonomously collect, analyze, and transmit structural data are a fundamental characteristic of intelligent infrastructures, constituting an essential element within the framework of intelligent urban environments. The aforementioned procedure is commonly referred to as "intelligent monitoring" and is implemented within the realm of intelligent infrastructure. AI algorithms enable the facilitation of large-scale data processing and the discernment of patterns and characteristics that would otherwise elude detection through conventional methodologies. Notwithstanding these inherent benefits, the utilization of artificial intelligence (AI) algorithms for intelligent monitoring is presently constrained due to the prevailing skepticism among engineers regarding the typically inscrutable internal mechanisms of AI. The necessity of elucidating the enigmatic nature of AI algorithms employed in intelligent surveillance to the engineering community is crusscial for fostering confidence in AI systems. This endeavor is commonly referred to as "explainable artificial intelligence" (XAI). The necessity for accurate classification arises from the diverse range of AI algorithms, particularly when endeavoring to enhance the explainability of said algorithms through the utilization of eXplainable Artificial Intelligence (XAI) in the context of intelligent monitoring. To effectively categorize AI, particularly algorithms employing machine learning for intelligent monitoring, this review article initially delineates the objectives of smart monitoring. Next, the machine learning algorithms utilized for the purpose of intelligent monitoring are thoroughly analyzed and systematically categorized. To achieve the goal of achieving transparent artificial intelligence in the field of civil engineering, we present a comprehensive examination of machine learning (ML) algorithms specifically designed for intelligent monitoring. This examination provides a detailed analysis of the various categories of ML algorithms that can be customized to enhance monitoring capabilities.

Human versus machine in medicine: can scientific literature answer the question?

Artificial intelligence (AI) hallucinations occur when large language models, such as chatbots or computer vision systems, generate outputs containing non-existent patterns, leading to inaccurate results. Also known as AI confabulations or delusions, these instances challenge expectations of appropriate responses from AI tools due to unrelated or pattern-lacking outputs, similar to human hallucinations. Addressing such issues with generative AI presents significant challenges despite ongoing efforts to resolve them.[1,2] CAUSES OF AI HALLUCINATIONS Various causes of AI hallucinations have been identified and include: Insufficient or biased training data: An AI model designed to assist anaesthesiologists in administering anaesthesia may be trained predominantly on data from patients of a certain demographic, such as adults of average weight. When faced with a paediatric patient or an obese patient, the AI model may possibly hallucinate dosage recommendations that are inaccurate or unsafe, as it lacks sufficient exposure to diverse patient populations.[3] Model complexity: A highly complex AI system tasked with monitoring vital signs during surgery may exhibit hallucinatory responses when encountering unusual physiological patterns. This complexity underscores the need for simpler models to avoid such hallucinations.[4] Lack of explainability (black box): An AI algorithm designed to predict anaesthesia induction times may produce unexpectedly long or short estimates without providing clear explanations for its predictions. In cases where anaesthesiologists cannot understand or verify the AI system’s reasoning, there is a risk of blindly following its recommendations, potentially leading to errors or patient harm. This highlights the urgent need for explainable AI in anaesthesia.[5] MULTIFACETED THREAT OF AI HALLUCINATIONS IN ANAESTHESIA An AI hallucination occurs when an AI system produces demonstrably incorrect or misleading outputs, appearing confident and plausible despite factually flawed. The possible impacts of AI hallucinations on anaesthesia domains are varied[6-9] [Table 1].Table 1: Examples of AI hallucinations’ possible impact on anaesthesia domainsMisdiagnosis and mistreatment: Hallucinations can misinterpret patient data, resulting in unnecessary interventions or delayed treatments. Medication errors: AI-driven systems may recommend incorrect drug dosages, impacting patient safety. Communication and documentation: Misinterpreted verbal commands or procedure details can hinder accurate documentation and patient safety. Research skewing: AI-driven analysis of anaesthesia data for research could be skewed by hallucinations, leading to misleading conclusions. Legal and ethical concerns: Liability: Who is responsible for the errors caused by AI hallucinations? This remains a complex question with no clear answer. Depending on the specific circumstances, potential targets include the AI developer, healthcare provider or hospital. Informed consent: How can patients be adequately informed about the risks of AI hallucinations in anaesthesia, given the technical complexity involved and the dynamic nature of AI outputs? Striking a balance between transparency and patient anxiety is crucial. Bias: AI algorithms can perpetuate societal biases, leading to discriminatory outcomes in health care. Imagine an AI system trained on biased data; it might recommend different treatments based on a patient’s race or socioeconomic background.[10-12] STRATEGIES TO MITIGATE AI HALLUCINATIONS Various mitigation strategies need to be adhered to for the impact of AI hallucination on health care [Figure 1].Figure 1: Impact of AI hallucination on health care and mitigation strategies. AI = artificial intelligenceHigh-quality, diverse training data: Utilising diverse datasets improves AI model accuracy and reduces hallucination risks. For example, research by Jones et al.[13] demonstrated how incorporating various demographic factors and medical histories in training data significantly improved the accuracy of an AI-driven diagnostic tool for skin cancer detection. Explainable AI: Developing transparent AI models aids in identifying and rectifying hallucinations. For instance, the explainable nature of a deep learning model used in financial fraud detection allowed analysts to trace back erroneous predictions to specific data points, enabling targeted adjustments to the model’s training data and architecture.[14] Human oversight and collaboration: Human involvement reduces hallucination risks, especially in sensitive domains like health care. Collaborative efforts between AI systems and human experts have effectively reduced hallucination risks.[15] Continuous monitoring and evaluation: Regular evaluation detects and addresses hallucinations promptly. Continuous monitoring of its AI-powered recommendation system and real-time user feedback analysis allows for swift identification and correction of hallucinated product suggestions, improving user satisfaction and trust.[16] Algorithmic auditing and regulatory frameworks: Establishing robust auditing mechanisms and regulatory frameworks ensures AI system’s accountability and reliability.[17] To conclude, AI hallucinations in anaesthesia pose risks of misdiagnosis, medication errors and skewed research outcomes. Prioritising diverse training data, embracing explainable AI, maintaining human oversight, continuous monitoring and regulatory frameworks are crucial in mitigating these risks and fostering trust in AI technologies in health care. Financial support and sponsorship Nil. Conflicts of interest There are no conflicts of interest.

Artificial intelligence (AI) and machine learning (ML) have potential to revolutionize emergency medical care by enhancing triage systems, improving diagnostic accuracy, refining prognostication, and optimizing various aspects of clinical care. However, as clinicians often lack AI expertise, they might perceive AI as a "black box," leading to trust issues. To address this, "explainable AI," which teaches AI functionalities to end-users, is important. This review presents the definitions, importance, and role of explainable AI, as well as potential challenges in emergency medicine. First, we introduce the terms explainability, interpretability, and transparency of AI models. These terms sound similar but have different roles in discussion of AI. Second, we indicate that explainable AI is required in clinical settings for reasons of justification, control, improvement, and discovery and provide examples. Third, we describe three major categories of explainability: pre-modeling explainability, interpretable models, and post-modeling explainability and present examples (especially for post-modeling explainability), such as visualization, simplification, text justification, and feature relevance. Last, we show the challenges of implementing AI and ML models in clinical settings and highlight the importance of collaboration between clinicians, developers, and researchers. This paper summarizes the concept of "explainable AI" for emergency medicine clinicians. This review may help clinicians understand explainable AI in emergency contexts.

Artificial intelligence (AI) technology has shown tremendous contributions in various applications like speech recognition, expert systems, computer vision, robotics, and gaming. machine learning (ML) and deep learning (DL) algorithms under AI address problems such as prediction, classification, and regression. AI has touched many domains. The results or the predictions generated by these algorithms are not easily accepted by the user. Especially, the Healthcare domain is facing a great challenge in accepting the results or the predictions with the concern, Are AI results reliable, correct, and ethical? Doctors or medical practitioners are not ready to treat patients based on results or suggestions generated by AI algorithms. Hence, a technology that can explain how the results returned by AI algorithms are trustworthy, transparent, and interpretable was strongly needed. This need has given rise to the latest technology-explainable artificial intelligence (XAI). With the use of XAI, all the predictions, classifications made by AI algorithms are explainable, auditable, comprehensive, validating, and socially acceptable. This paper discusses explaining the results of breast cancer prediction as a case study. The results show that such an explanation will build trust in the doctors and hence will increase the acceptance of the AI-based systems.

The contention that medical artificial intelligence (AI) should be ‘explainable’ is widespread in contemporary philosophy and in legal and best practice documents. Yet critics argue that ‘explainability’ is not a stable concept; non-explainable AI is often more accurate; mechanisms intended to improve explainability do not improve understanding and introduce new epistemic concerns; and explainability requirements are ad hoc where human medical decision-making is often opaque. A recent ‘political response’ to these issues contends that AI used in high-stakes scenarios, including medical AI, must be explainable to meet basic standards of legitimacy: People are owed reasons for decisions that impact their vital interests, and this requires explainable AI. This article demonstrates why the political response fails. Attending to systemic considerations, as its proponents desire, suggests that the political response is subject to the same criticisms as other arguments for explainable AI and presents new issues. It also suggests that decision-making about non-explainable medical AI can meet public reason standards. The most plausible version of the response amounts to a simple claim that public reason demands reasons why AI is permitted. But that does not actually support explainable AI or respond to criticisms of strong requirements for explainable medical AI.

Artificial intelligence (AI) provides considerable opportunities to assist human work. However, one crucial challenge of human–AI collaboration is that many AI algorithms operate in a black-box manner where the way how the AI makes predictions remains opaque. This makes it difficult for humans to validate a prediction made by AI against their own domain knowledge. For this reason, we hypothesize that augmenting humans with explainable AI improves task performance in human–AI collaboration. To test this hypothesis, we implement explainable AI in the form of visual heatmaps in inspection tasks conducted by domain experts. Visual heatmaps have the advantage that they are easy to understand and help to localize relevant parts of an image. We then compare participants that were either supported by (a) black-box AI or (b) explainable AI, where the latter supports them to follow AI predictions when the AI is accurate or overrule the AI when the AI predictions are wrong. We conducted two preregistered experiments with representative, real-world visual inspection tasks from manufacturing and medicine. The first experiment was conducted with factory workers from an electronics factory, who performed assessments of whether electronic products have defects. The second experiment was conducted with radiologists, who performed assessments of chest X-ray images to identify lung lesions. The results of our experiments with domain experts performing real-world tasks show that task performance improves when participants are supported by explainable AI with heatmaps instead of black-box AI. We find that explainable AI as a decision aid improved the task performance by 7.7 percentage points (95% confidence interval [CI]: 3.3% to 12.0%, ) in the manufacturing experiment and by 4.7 percentage points (95% CI: 1.1% to 8.3%, ) in the medical experiment compared to black-box AI. These gains represent a significant improvement in task performance.

The rapid proliferation of artificial intelligence (AI) technologies across various industries and decision-making processes has undeniably transformed the way of approaching complex problems and tasks. AI systems have proven their prowess in areas such as healthcare, finance, and autonomous systems, revolutionizing how decisions are made. Nevertheless, this proliferation of AI has raised critical concerns regarding the transparency, accountability, and fairness of these systems, as many of the state-of-the-art AI models often resemble complex black boxes. These intricate models, particularly deep learning neural networks, harbor non-linear relationships that are difficult for human users to decipher, thereby raising concerns about bias, fairness, and overall trustworthiness in AI-driven decisions. The urgency of this issue is underscored by the realization that AI should not merely be accurate; it should also be interpretable. Explainable AI (XAI) has emerged as a vital field of research, emphasizing the development of models and techniques that render AI systems comprehensible and transparent in their decision-making processes. This paper investigates into the relevance and significance of XAI across various domains, including healthcare, finance, and autonomous systems, where the ability to understand the rationale behind AI decisions is paramount. In healthcare, where AI assists in diagnosis and treatment, the interpretability of AI models is crucial for clinicians to make informed decisions. In finance, applications like credit scoring and investment analysis demand transparent AI to ensure fairness and accountability. In the realm of autonomous systems, transparency is indispensable to guarantee safety and compliance with regulations. Moreover, government agencies in areas such as law enforcement and social services require interpretable AI to maintain ethical standards and accountability. This paper also highlights the diverse array of research efforts in the XAI domain, spanning from model-specific interpretability methods to more general approaches aimed at unveiling complex AI models. Interpretable models like decision trees and rule-based systems have gained attention for their inherent transparency, while integrating explanation layers into deep neural networks strives to balance accuracy with interpretability. The study emphasizes the significance of this burgeoning field in bridging the gap between AI's advanced capabilities and human users' need for comprehensible AI systems. It seeks to contribute to this field by exploring the design, development, and practical applications of interpretable AI models and techniques, with the ultimate goal of enhancing the trust and understanding of AI-driven decisions.

Artificial intelligence (AI) studies are progressing at a breakneck pace, with prospective programs in healthcare industries being established. In healthcare, there has been an extensive demonstration of the promise of AI through numerous applications like medical support systems and smart healthcare. Explainable artificial intelligence (XAI) development has been extremely beneficial in this direction. XAI models allow smart healthcare equipped with AI models so that the results generated by AI algorithms can be understood and trusted. Therefore, the goal of this chapter is to discuss the utility of XAI in systems used in healthcare. The issues, as well as difficulties related to the usage of XAI models in the healthcare system, were also discussed. The findings demonstrate some examples of XAI's effective medical practice implementation. The real-world application of XAI models in healthcare will significantly improve users' trust in AI algorithms in healthcare systems.

The rapid advancements in artificial intelligence and machine learning have led to the development of highly sophisticated models capable of superhuman performance in a variety of tasks. However, the increasing complexity of these models has also resulted in them becoming "black boxes", where the internal decision-making process is opaque and difficult to interpret. This lack of transparency and explainability has become a significant barrier to the widespread adoption of these models, particularly in sensitive domains such as healthcare and finance. To address this challenge, the field of Explainable AI has emerged, focusing on developing new methods and techniques to improve the interpretability and explainability of machine learning models. This review paper aims to provide a comprehensive overview of the research exploring the combination of Explainable AI and traditional machine learning approaches, known as "hybrid models". This paper discusses the importance of explainability in AI, and the necessity of combining interpretable machine learning models with black-box models to achieve the desired trade-off between accuracy and interpretability. It provides an overview of key methods and applications, integration techniques, implementation frameworks, evaluation metrics, and recent developments in the field of hybrid AI models. The paper also delves into the challenges and limitations in implementing hybrid explainable AI systems, as well as the future trends in the integration of explainable AI and traditional machine learning. Altogether, this paper will serve as a valuable reference for researchers and practitioners working on developing explainable and interpretable AI systems. Keywords: Explainable AI (XAI), Traditional Machine Learning (ML), Hybrid Models, Interpretability, Transparency, Predictive Accuracy, Neural Networks, Ensemble Methods, Decision Trees, Linear Regression, SHAP (Shapley Additive Explanations), LIME (Local Interpretable Model-agnostic Explanations), Healthcare Analytics, Financial Risk Management, Autonomous Systems, Predictive Maintenance, Quality Control, Integration Techniques, Evaluation Metrics, Regulatory Compliance, Ethical Considerations, User Trust, Data Quality, Model Complexity, Future Trends, Emerging Technologies, Attention Mechanisms, Transformer Models, Reinforcement Learning, Data Visualization, Interactive Interfaces, Modular Architectures, Ensemble Learning, Post-Hoc Explainability, Intrinsic Explainability, Combined Models