Transparency In Machine Learning Research Articles

Explainable artificial intelligence to improve the resilience of perishable product supply chains by leveraging customer characteristics

AbstractOptimizing costs and profits while meeting customer demand is a critical challenge in the development of perishable supply chains. Customer-centric demand forecasting addresses this challenge by considering customer characteristics when determining inventory levels. This study proposes a solution framework comprising two steps: (a) segmentation using customer characteristics and (b) demand forecasting for each segment using transparent and responsible artificial intelligence techniques. We employed k-means, hierarchical clustering, and explainable AI (XAI) to segment, model, and compare several machine-learning techniques for demand forecasting. The results showed that support vector regression outperformed the autoregressive models. The results also showed that the two-step segmentation and demand forecasting process using hierarchical clustering and LSTM outperforms (Weighted average RMSE across segments = 61.57) the conventional single-step unsegmented forecasting process (RMSE overall data = 238.18). The main implication of this study is the demonstration of XAI in enhancing transparency in machine learning and an improved method for reducing forecasting errors in practice, which can strengthen the supply chain resilience for perishable products.

Interpretable Machine Learning: A Case Study on Predicting Fuel Consumption in VLGC Ship Propulsion

The integration of machine learning (ML) in marine engineering has been increasingly subjected to stringent regulatory scrutiny. While environmental regulations aim to reduce harmful emissions and energy consumption, there is also a growing demand for the interpretability of ML models to ensure their reliability and adherence to safety standards. This research highlights the need to develop models that are both transparent and comprehensible to domain experts and regulatory bodies. This paper underscores the importance of transparency in machine learning through a use case involving a VLGC ship two-stroke propulsion engine. By adhering to the CRISP-DM standard, we fostered close collaboration between marine engineers and machine learning experts to circumvent the common pitfalls of automated ML. The methodology included comprehensive data exploration, cleaning, and verification, followed by feature selection and training of linear regression and decision tree models that are not only transparent but also highly interpretable. The linear model achieved an RMSE of 23.16 and an MRAE of 14.7%, while the accuracy of decision trees ranged between 96.4% and 97.69%. This study demonstrates that machine learning models for predicting propulsion engine fuel consumption can be interpretable, adhering to regulatory requirements, while still achieving adequate predictive performance.

Algorithmic Accountability for Record Linkage

This session will examine record linkage approaches through the lens of principles for algorithmic accountability: responsibility, explainability, accuracy, auditability, and fairness.[1] It is easy to hypothesize cases where linkage algorithms could have disparate impacts, such as linking names more or less easily depending on their cultural background. How can researchers use careful linkage design and sensitivity testing to identify and, where necessary, mediate these impacts? What does it look like for record linkage processes to be explainable and auditable—and how can researchers invite engagement from broader groups of stakeholders in considering linkage methods? We will present on our experiences identifying and addressing these challenges in the context of linking state and local public sector administrative data sources. These data sources, which generally include populations engaging with particular programs or systems, have especially variable data quality and are often marred by structural biases. Our approach to linking these datasets emphasizes integrating policy and data context, interrogating linkage decisions for iterative improvement, and collaborating with a wide range of stakeholders for equitable, accountable data use. We will demonstrate examples of how linkage decisions and results have varied across contexts. [1] Diakopoulos, N., Friedler, S., Arenas, M., Barocas, S., Hay, M., Howe, B., Jagadish, H.V., Unsworth, K., Sahuguet, A., Venkatasubramanian, S., Wilson, C., Yu, C., and Zevenbergen, B. (2016). Principles for Accountable Algorithms and a Social Impact Statement for Algorithms. Fairness, Accountability, and Transparency in Machine Learning. https://www.fatml.org/resources/principles-for-accountable-algorithms.

Enhancing Transparency in Healthcare Machine Learning Models Using Shap and Deeplift a Methodological Approach

This paper intends to provide a better understanding of how these models produce predictions, particularly in complex medical diagnoses, and at the same time bridging the gap between technical model outputs and clinical applications. This study addresses the critical problem of transparency of machine learning (ML) models in health care where interpretability is an essential aspect for ethical decision making and trust building.The goal of this paper is to present a clearer understanding of how these models generate predictions, especially in important fields like complex medical diagnoses, thereby bridging the gap between technical model outputs and clinical applications. The crucial issue of transparency in machine learning (ML) models within healthcare is addressed in this study where interpretability plays a vital role in ethical decision-making and fostering trust. The focus of the research is enhancing model transparency by using SHapley Additive exPlanations (SHAP) and Deep Learning Important FeaTures (DeepLIFT), two crucial methods that are designed to elucidate the decision-making processes of ML models.This mode of approach helps to have more distributed comprehension of the decision pathways by models thus aiding in knowing how each feature contributed to the last prediction. It is this method that has been employed to showcase the efficiency of predicting melanoma and also diabetic retinopathy which are two vital medical diagnostic areas. In healthcare, SHAP along with DeepLIFT has improved the models’ explainability and trustworthiness significantly and hence making them easy for those in the field. The advanced interpretability methods presented in this document enhances ML model transparency especially when dealing with health issues. As a result, interpretability becomes an even bigger issue and they are supposed to be able to use these tools for reliable and open decisions when it comes to medical specialists.

Interpretability and Transparency of Machine Learning in File Fragment Analysis with Explainable Artificial Intelligence

Interpretability and Transparency of Machine Learning in File Fragment Analysis with Explainable Artificial Intelligence

Prediction of occupant thermal state via infrared thermography and explainable AI

Accurate and real-time assessment of occupant thermal comfort can provide a solid foundation for efficient air conditioning operations. Existing studies already show the feasibility of using contactless technologies for thermal comfort prediction assisted by machine learning algorithms. However, the lack of transparency in machine learning often weakens user trust. This study performs explainable AI analysis to explore the potential of infrared imaging in thermal comfort evaluation. Specifically, an investigation was carried out in a climatic chamber, and infrared cameras were used to collect facial temperature data. Five popular ensemble tree models were employed to construct prediction models, and explainable AI analysis was performed using SHAP (SHapley Additive exPlanations) theory. Results show that combining additional facial information can significantly improve the overall model performance, and certain facial attributes present high contributions based on SHAP values. Combining facial features with explainable AI provides a convincing basis for thermal comfort assessment. The high SHAP values of facial features can also contribute to finding selective occupants with low neutral voting rates, providing evidence for customized cooling or heating from building systems.

Increasing transparency in machine learning through bootstrap simulation and shapely additive explanations.

Machine learning methods are widely used within the medical field. However, the reliability and efficacy of these models is difficult to assess, making it difficult for researchers to identify which machine-learning model to apply to their dataset. We assessed whether variance calculations of model metrics (e.g., AUROC, Sensitivity, Specificity) through bootstrap simulation and SHapely Additive exPlanations (SHAP) could increase model transparency and improve model selection. Data from the England National Health Services Heart Disease Prediction Cohort was used. After comparison of model metrics for XGBoost, Random Forest, Artificial Neural Network, and Adaptive Boosting, XGBoost was used as the machine-learning model of choice in this study. Boost-strap simulation (N = 10,000) was used to empirically derive the distribution of model metrics and covariate Gain statistics. SHapely Additive exPlanations (SHAP) to provide explanations to machine-learning output and simulation to evaluate the variance of model accuracy metrics. For the XGBoost modeling method, we observed (through 10,000 completed simulations) that the AUROC ranged from 0.771 to 0.947, a difference of 0.176, the balanced accuracy ranged from 0.688 to 0.894, a 0.205 difference, the sensitivity ranged from 0.632 to 0.939, a 0.307 difference, and the specificity ranged from 0.595 to 0.944, a 0.394 difference. Among 10,000 simulations completed, we observed that the gain for Angina ranged from 0.225 to 0.456, a difference of 0.231, for Cholesterol ranged from 0.148 to 0.326, a difference of 0.178, for maximum heart rate (MaxHR) ranged from 0.081 to 0.200, a range of 0.119, and for Age ranged from 0.059 to 0.157, difference of 0.098. Use of simulations to empirically evaluate the variability of model metrics and explanatory algorithms to observe if covariates match the literature are necessary for increased transparency, reliability, and utility of machine learning methods. These variance statistics, combined with model accuracy statistics can help researchers identify the best model for a given dataset.

Bridging the Gap: Enhancing Trust and Transparency in Machine Learning with Explainable AI

Explainable Artificial Intelligence (XAI) aims to address the complexity and opacity of AI systems, often referred to as "black boxes." It seeks to provide transparency and build trust in AI, particularly in domains where decisions impact safety, security, and ethical considerations. XAI approaches fall into three categories: opaque systems that offer no explanation for their predictions, interpretable systems that provide some level of justification, and comprehensible systems that enable users to reason about and interact with the AI system. Automated reasoning plays a crucial role in achieving truly explainable AI. This paper presents current methodologies, challenges, and the importance of integrating automated reasoning for XAI. It is grounded in a thorough literature review and case studies, providing insights into the practical applications and future directions for XAI.

The Limits of Value Transparency in Machine Learning

AbstractTransparency has been proposed as a way of handling value-ladenness in machine learning (ML). This article highlights limits to this strategy. I distinguish three kinds of transparency: epistemic transparency, retrospective value transparency, and prospective value transparency. This corresponds to different approaches to transparency in ML, including so-called explainable artificial intelligence and governance based on disclosing information about the design process. I discuss three sources of value-ladenness in ML—problem formulation, inductive risk, and specification gaming—and argue that retrospective value transparency is only well-suited for dealing with the first, while the third raises serious challenges even for prospective value transparency.

Fish AI

Sustainable Commercial Fishing is the second challenge at the Nordic AI Meet following the successful MedAI, which had a focus on medical image segmentation and transparency in machine learning (ML)-based systems. FishAI focuses on a new domain, namely, commercial fishing and how to make it more sustainable with the help of machine learning. A range of public available datasets is used to tackle three specific tasks. The first one is to predict fishing coordinates to optimize catching of specific fish, the second one is to create a report that can be used by experienced fishermen, and the third task is to make a sustainable fishing plan that provides a route for a week. The second and third task require to some extend explainable and interpretable models that can provide explanations. A development dataset is provided and all methods will be tested on a concealed test dataset and assessed by an expert jury.
 
 artificial intelligence; machine learning; segmentation; transparency; medicine

MedAI: Transparency in Medical Image Segmentation

MedAI: Transparency in Medical Image Segmentation is a challenge held for the first time at the Nordic AI Meet that focuses on medical image segmentation and transparency in machine learning (ML)-based systems. We propose three tasks to meet specific gastrointestinal image segmentation challenges collected from experts within the field, including two separate segmentation scenarios and one scenario on transparent ML systems. The latter emphasizes the need for explainable and interpretable ML algorithms. We provide a development dataset for the participants to train their ML models, tested on a concealed test dataset.

Transparency of machine-learning in healthcare: The GDPR & European health law

Machine-learning (‘ML’) models are powerful tools which can support personalised clinical judgments, as well as patients’ choices about their healthcare. Concern has been raised, however, as to their ‘black box’ nature, in which calculations are so complex they are difficult to understand and independently verify. In considering the use of ML in healthcare, we divide the question of transparency into three different scenarios:1)Solely automated decisions. We suggest these will be unusual in healthcare, as Article 22(4) of the General Data Protection Regulation presents a high bar. However, if solely automatic decisions are made (e.g. for inpatient triage), data subjects will have a right to ‘meaningful information’ about the logic involved.2)Clinical decisions. These are decisions made ultimately by clinicians—such as diagnosis—and the standard of transparency under the GDPR is lower due to this human mediation.3)Patient decisions. Decisions about treatment are ultimately taken by the patient or their representative, albeit in dialogue with clinicians. Here, the patient will require a personalised level of medical information, depending on the severity of the risk, and how much they wish to know.In the final category of decisions made by patients, we suggest European healthcare law sets a more personalised standard of information requirement than the GDPR. Clinical information must be tailored to the individual patient according to their needs and priorities; there is no monolithic ‘explanation’ of risk under healthcare law. When giving advice based (even partly) on a ML model, clinicians must have a sufficient grasp of the medically-relevant factors involved in the model output to offer patients this personalised level of medical information. We use the UK, Ireland, Denmark, Norway and Sweden as examples of European health law jurisdictions which require this personalised transparency to support patients’ rights to make informed choices. This adds to the argument for post-hoc, rationale explanations of ML to support healthcare decisions in all three scenarios.

Combining genetic algorithm with machine learning strategies for designing potent antimicrobial peptides

BackgroundCurrent methods in machine learning provide approaches for solving challenging, multiple constraint design problems. While deep learning and related neural networking methods have state-of-the-art performance, their vulnerability in decision making processes leading to irrational outcomes is a major concern for their implementation. With the rising antibiotic resistance, antimicrobial peptides (AMPs) have increasingly gained attention as novel therapeutic agents. This challenging design problem requires peptides which meet the multiple constraints of limiting drug-resistance in bacteria, preventing secondary infections from imbalanced microbial flora, and avoiding immune system suppression. AMPs offer a promising, bioinspired design space to targeting antimicrobial activity, but their versatility also requires the curated selection from a combinatorial sequence space. This space is too large for brute-force methods or currently known rational design approaches outside of machine learning. While there has been progress in using the design space to more effectively target AMP activity, a widely applicable approach has been elusive. The lack of transparency in machine learning has limited the advancement of scientific knowledge of how AMPs are related among each other, and the lack of general applicability for fully rational approaches has limited a broader understanding of the design space.MethodsHere we combined an evolutionary method with rough set theory, a transparent machine learning approach, for designing antimicrobial peptides (AMPs). Our method achieves the customization of AMPs using supervised learning boundaries. Our system employs in vitro bacterial assays to measure fitness, codon-representation of peptides to gain flexibility of sequence selection in DNA-space with a genetic algorithm and machine learning to further accelerate the process.ResultsWe use supervised machine learning and a genetic algorithm to find a peptide active against S. epidermidis, a common bacterial strain for implant infections, with an improved aggregation propensity average for an improved ease of synthesis.ConclusionsOur results demonstrate that AMP design can be customized to maintain activity and simplify production. To our knowledge, this is the first time when codon-based genetic algorithms combined with rough set theory methods is used for computational search on peptide sequences.

Deep Automation Bias: How to Tackle a Wicked Problem of AI?

The increasing use of AI in different societal contexts intensified the debate on risks, ethical problems and bias. Accordingly, promising research activities focus on debiasing to strengthen fairness, accountability and transparency in machine learning. There is, though, a tendency to fix societal and ethical issues with technical solutions that may cause additional, wicked problems. Alternative analytical approaches are thus needed to avoid this and to comprehend how societal and ethical issues occur in AI systems. Despite various forms of bias, ultimately, risks result from eventual rule conflicts between the AI system behavior due to feature complexity and user practices with limited options for scrutiny. Hence, although different forms of bias can occur, automation is their common ground. The paper highlights the role of automation and explains why deep automation bias (DAB) is a metarisk of AI. Based on former work it elaborates the main influencing factors and develops a heuristic model for assessing DAB-related risks in AI systems. This model aims at raising problem awareness and training on the sociotechnical risks resulting from AI-based automation and contributes to improving the general explicability of AI systems beyond technical issues.

Artificial cognition: How experimental psychology can help generate explainable artificial intelligence.

Artificial intelligence powered by deep neural networks has reached a level of complexity where it can be difficult or impossible to express how a model makes its decisions. This black-box problem is especially concerning when the model makes decisions with consequences for human well-being. In response, an emerging field called explainable artificial intelligence (XAI) aims to increase the interpretability, fairness, and transparency of machine learning. In this paper, we describe how cognitive psychologists can make contributions to XAI. The human mind is also a black box, and cognitive psychologists have over 150 years of experience modeling it through experimentation. We ought to translate the methods and rigor of cognitive psychology to the study of artificial black boxes in the service of explainability. We provide a review of XAI for psychologists, arguing that current methods possess a blind spot that can be complemented by the experimental cognitive tradition. We also provide a framework for research in XAI, highlight exemplary cases of experimentation within XAI inspired by psychological science, and provide a tutorial on experimenting with machines. We end by noting the advantages of an experimental approach and invite other psychologists to conduct research in this exciting new field.

‘Computer Says No’: Was Your Mortgage Application Rejected Unfairly?

`Fairness, Accountability and Transparency in Machine Learning' is a new field of AI science and what sounds like its examination of conscience. Are the machines tainted by the sins of their imperfect creators? In this article I review a selection of FAT/ML research aimed at ensuring fairness in some of the critical aspects of human existence. On the example of AI algorithms advising the US justice system, I demonstrate a simple statistical procedure of assessing bias in decision making, highlighting the importance of careful understanding of data and statistical concepts. Finally, I describe how FAT/ML tries to reconcile the machine transgressions in the best of possible, counterfactual reality. Should we absolve the machines and let them go and make the world a better place? I hope you will be able to answer this question yourself.

Regulation of AI and AI Crimes

According to McCarthy, artificial intelligence (AI) is “the science and engineering of making intelligent machines, especially intelligent computer programs. It is related to the similar task of using computers to understand human intelligence, but AI does not have to confine itself to methods that are highly biologically observable”. AI may play an imminent role in committing criminal acts in the future and these acts are called AI- crimes (AIC). AIC has posed a threat towards cybersecurity, a few of AIC are, a) theft of sensitive data from government and private organizations, b) hacked and weaponized autonomous weapons, c) social engineering attacks, d) disruption in the political arena, such as “misinformation” campaigns, etc. There is no regulatory framework at the moment which targets AI-crimes, majority of the stakeholder’s policy recommendations are targeted towards fairness, accountability, and transparency in machine learning or they target the R&D of robots, none of the legal texts cover the much-debated topic of criminal liability of an AI as the law is still undecided. The traditional method of regulation of AI has been by, product licensing, research and development oversight and tort liability. However, the current AI principles are not enough to determine the threat posed by AI acting autonomously in the future. The paper is going to be divided into four parts, the first part of the paper will be an introduction to the concept of Artificial Intelligence Crime (AIC) and plausible AI malicious actions. The second part of the paper will delve into the existing cybercrime framework and try to fathom whether the frameworks incorporate AI crime. The third part of the paper will be a study of different international organizations that have made progress pertaining to AI crimes and their success in the applicability of legal principles to AI crimes. Further, the author will recommend legal principles specifically targeting AI crimes which should be made applicable in the cybercrime framework. The fourth part of the paper will deal with the challenges of determining the jurisdiction of AI crimes as AI crimes are committed on computer systems and networks across the globe. In the conclusive part, the author will analyze how should the law deal with the above challenges along with recommendations of how the legal framework should specifically deal with the above challenges.

Machine Learning and Transparency: A Scoping Exercise

As machine learning is increasingly embedded in every day, an active discussion around the need for transparency in these systems has emerged. However, there is little to no consensus within or between disciplines as to the merits, viability and forms of transparency we should be striving for. Here, I offer an analysis of existing scholarship and discussion around transparency in machine learning, with a view to identify emerging threads for consideration, and distill questions for the way forward. I will attempt to answer one overarching question: ‘Where do disagreements in current scholarship around transparency in machine learning lie?’ and in doing so will point to alternative configurations emerging in technical and policy discussions.

Beyond opening up the black box: Investigating the role of algorithmic systems in Wikipedian organizational culture

Scholars and practitioners across domains are increasingly concerned with algorithmic transparency and opacity, interrogating the values and assumptions embedded in automated, black-boxed systems, particularly in user-generated content platforms. I report from an ethnography of infrastructure in Wikipedia to discuss an often understudied aspect of this topic: the local, contextual, learned expertise involved in participating in a highly automated social-technical environment. Today, the organizational culture of Wikipedia is deeply intertwined with various data-driven algorithmic systems, which Wikipedians rely on to help manage and govern the "anyone can edit" encyclopedia at a massive scale. These bots, scripts, tools, plugins, and dashboards make Wikipedia more efficient for those who know how to work with them, but like all organizational culture, newcomers must learn them if they want to fully participate. I illustrate how cultural and organizational expertise is enacted around algorithmic agents by discussing two autoethnographic vignettes, which relate my personal experience as a veteran in Wikipedia. I present thick descriptions of how governance and gatekeeping practices are articulated through and in alignment with these automated infrastructures. Over the past 15 years, Wikipedian veterans and administrators have made specific decisions to support administrative and editorial workflows with automation in particular ways and not others. I use these cases of Wikipedia's bot-supported bureaucracy to discuss several issues in the fields of critical algorithms studies, critical data studies, and fairness, accountability, and transparency in machine learning -- most principally arguing that scholarship and practice must go beyond trying to "open up the black box" of such systems and also examine sociocultural processes like newcomer socialization.