Open Black Box Research Articles

Explainable AI-driven IoMT fusion: Unravelling techniques, opportunities, and challenges with Explainable AI in healthcare

Background and Objective:Artificial Intelligence (AI) has shown significant advancements across several industries, including healthcare, using better fusion methodologies, improved data accessibility, and heightened processing capabilities. Although deep learning algorithms excel in challenging scenarios, their widespread adoption presents a challenge due to their need for more transparency. The advent of Explainable AI (XAI) has addressed this limitation by offering AI-driven systems with attributes such as transparency, interpretability, and reliability. The need for transparency in critical sectors like healthcare has prompted an increased academic focus on investigating and understanding these frameworks. The present paper thoroughly examines the latest research and advancements in Explainable Artificial Intelligence (XAI), specifically focusing on its integration into the Internet of Medical Things (IoMT) in healthcare settings. The last section tackles outstanding issues concerning XAI and IoMT and outlines prospective directions for further research. Methods:A thorough investigation was carried out across many scholarly databases and online digital libraries, such as IEEE Xplore, ACM Digital Library, Wiley Interscience, Taylor and Francis, ScienceDirect, Scopus, Springer, Google Scholar, Citeseer Library, Semantic Scholar, and other relevant sources. An analysis was conducted on articles published from March 2004 to February 2024, specifically focusing on AI models that explain different healthcare issues. The search query included “Explainable AI” in conjunction with “Open Black Box, Healthcare, Transparent Models, Interpretable Deep Learning, Machine Learning, Medical Information System, Accountability, Smart Healthcare, and Responsible AI”, indicating the field’s dynamic nature. The authors also examined various techniques and datasets to elucidate healthcare difficulties, including incorporating the Internet of Medical Things (IoMT). Results:The evaluation included more than 105 published models that used clinical data input to diagnose diverse disorders, focusing on incorporating Internet of Medical Things (IoMT) components. In addition to illness diagnosis, these models also elucidate the decisions made. The models were classified according to the input data, used methodologies, and integration of the Internet of Medical Things (IoMT). The categorization included machine learning and deep learning methodologies, particularly on explainability models connected to the Internet of Medical Things (IoMT). These models were classified as model-agnostic, model-specific, global, local, ante-hoc, and post-hoc. Conclusion:This extensive study examines machine learning, and deep learning models developed based on clinical data and associated with the Internet of Medical Things (IoMT) for illness detection. These models improve efficiency and accuracy and provide essential assistance to medical personnel. Explaining each choice made by individuals contributes to the acceleration of sickness detection, resulting in decreased medical expenses for patients and healthcare systems. The integration of Explainable Artificial Intelligence (XAI) and the Internet of Medical Things (IoMT) highlights a fundamental change in healthcare towards more transparent and linked systems, leading to significant breakthroughs in medical decision-making.

Open your black box classifier.

A priority for machine learning in healthcare and other high stakes applications is to enable end-users to easily interpret individual predictions. This opinion piece outlines recent developments in interpretable classifiers and methods to open black box models.

Explainable artificial intelligence in information systems: A review of the status quo and future research directions

The quest to open black box artificial intelligence (AI) systems evolved into an emerging phenomenon of global interest for academia, business, and society and brought about the rise of the research field of explainable artificial intelligence (XAI). With its pluralistic view, information systems (IS) research is predestined to contribute to this emerging field; thus, it is not surprising that the number of publications on XAI has been rising significantly in IS research. This paper aims to provide a comprehensive overview of XAI research in IS in general and electronic markets in particular using a structured literature review. Based on a literature search resulting in 180 research papers, this work provides an overview of the most receptive outlets, the development of the academic discussion, and the most relevant underlying concepts and methodologies. Furthermore, eight research areas with varying maturity in electronic markets are carved out. Finally, directions for a research agenda of XAI in IS are presented.

How to Open a Black Box Classifier for Tabular Data

A lack of transparency in machine learning models can limit their application. We show that analysis of variance (ANOVA) methods extract interpretable predictive models from them. This is possible because ANOVA decompositions represent multivariate functions as sums of functions of fewer variables. Retaining the terms in the ANOVA summation involving functions of only one or two variables provides an efficient method to open black box classifiers. The proposed method builds generalised additive models (GAMs) by application of L1 regularised logistic regression to the component terms retained from the ANOVA decomposition of the logit function. The resulting GAMs are derived using two alternative measures, Dirac and Lebesgue. Both measures produce functions that are smooth and consistent. The term partial responses in structured models (PRiSM) describes the family of models that are derived from black box classifiers by application of ANOVA decompositions. We demonstrate their interpretability and performance for the multilayer perceptron, support vector machines and gradient-boosting machines applied to synthetic data and several real-world data sets, namely Pima Diabetes, German Credit Card, and Statlog Shuttle from the UCI repository. The GAMs are shown to be compliant with the basic principles of a formal framework for interpretability.

Robust extreme gradient boosting regression model for compressive strength prediction of blast furnace slag and fly ash concrete

In this study, a novel machine learning (ML) technique, eXtreme Gradient Boosting (XG Boost), was employed to train an extremely precise ML model. The developed XG Boost model was highly interpretable, filling the gap and opening black boxes in the literature. The study provides further a simple and free user interface to support the design of normal- and high-strength Blast Furnace Slag (BFS), and fly ash (FA) concrete. The compressive strength of 1030 concrete mixes containing cement (C), BFS, and FA were collected and analyzed. The baseline model tend to overfit, with R2 values of 0.996 and 0.919 for the training and testing datasets, respectively. The hyperparameters of the model have been optimized using vector multi-objective optimization to maximize the prediction capability of the model. The optimized XG Boost model exhibited a superior prediction performance with R2 of 0.992 and 0.949 for the training and testing datasets. Based on Gini indexes and SHAP values, C, FA, water, and aggregate were the most significant model parameters. According to this study, the best BFS, FA, sand, and superplasticizer contents for concrete strength optimization were 100–200, 100–200, 600–800, and 7–13 kg/m3, respectively. The SP has a negligible effect on concrete's compressive strength at low water contents (less than 180 kg/m3), but a stochastic effect at high contents. The various chemical properties of high-range water reducers may have resulted in the randomly generated response in the current study.

Black boxes and information pathways: An actor-network theory approach to breast cancer survivorship care

Many women diagnosed with breast cancer today can expect to live long after completing their treatment. This growing population of survivors encounters distinct post-treatment health and information needs. Existing survivorship care models take information as a given, black boxing it. I use Actor-Network Theory to examine how information actually works for women after they complete breast cancer treatment, and how it shapes their understanding of survivorship. I draw on in-depth interviews with breast cancer survivors (n = 82) and a wide range of providers (n = 84) in a medically underserved region of Southern California. Black boxes and information pathways convey experiential dimensions of cancer care; they are also metaphoric constructs. The black box metaphor refers to the cancer experience as a container; the pathways metaphor refers to a journey. Each of these metaphors expresses salient dimensions of the cancer experience and has implications for post-treatment survivorship. When healthcare information flows smoothly and invisibly, its pathways become black boxed. Black boxes can be helpful when they function effectively. But since black boxes conceal their inner workings, it is challenging to intervene when difficulties arise. I provide three examples of difficulties that complicate women's transition to post-treatment survivorship: (1) when survivors fail to recognize treatment-related late effects, (2) do not understand they have a terminal diagnosis, or (3) worry that their treatment accomplished nothing. Contextualized within survivorship scholarship, this study recommends opening black boxes to examine how information pathways could connect women differently to improve survivorship care.

Inconclusives, errors, and error rates in forensic firearms analysis:Three statistical perspectives

Error rates that have been published in recent open black box studies of forensic firearms examiner performance have been very low, typically below one percent. These low error rates have been challenged, however, as not properly taking into account one of the categories, “Inconclusive”, that examiners can reach in comparing a pair of bullets or cartridges. These challenges have themselves been challenged; how to consider the inconclusives and their effect on error rates is currently a matter of sharp debate.We review several viewpoints that have been put forth, and then examine the impact of inconclusives on error rates from three fresh statistical perspectives: (a) an ideal perspective using objective measurements combined with statistical algorithms, (b) basic sampling theory and practice, and (c) standards of experimental design in human studies. Our conclusions vary with the perspective:(a) inconclusives can be simple errors (or, on the other hand, simply correct or at least well justified);(b) inconclusives need not be counted as errors to bring into doubt assessments of error rates;(c) inconclusives are potential errors, more explicitly, inconclusives in studies are not necessarily the equivalent of inconclusives in casework and can mask potential errors in casework.From all these perspectives, it is impossible to simply read out trustworthy estimates of error rates from those studies which have been carried out to date. At most, one can put reasonable bounds on the potential error rates. These are much larger than the nominal rates reported in the studies.To get straightforward, sound estimates of error rates requires a challenging but critical improvement to the design of firearms studies. A proper study—one in which inconclusives are not potential errors, and which yields direct, sound estimates of error rates—will require new objective measures or blind proficiency testing embedded in ordinary casework.

Dodging Silver Bullets and Opening Black Boxes: Recommendations for Developing Athlete Monitoring Systems

Dodging Silver Bullets and Opening Black Boxes: Recommendations for Developing Athlete Monitoring Systems

A Survey of Methods for Explaining Black Box Models

In recent years, many accurate decision support systems have been constructed as black boxes, that is as systems that hide their internal logic to the user. This lack of explanation constitutes both a practical and an ethical issue. The literature reports many approaches aimed at overcoming this crucial weakness, sometimes at the cost of sacrificing accuracy for interpretability. The applications in which black box decision systems can be used are various, and each approach is typically developed to provide a solution for a specific problem and, as a consequence, it explicitly or implicitly delineates its own definition of interpretability and explanation. The aim of this article is to provide a classification of the main problems addressed in the literature with respect to the notion of explanation and the type of black box system. Given a problem definition, a black box type, and a desired explanation, this survey should help the researcher to find the proposals more useful for his own work. The proposed classification of approaches to open black box models should also be useful for putting the many research open questions in perspective.

War is Governance: Explaining the Logic of the Laws of War from a Principal-Agent Perspective

What is purpose of international law on armed conflict, and why would opponents bent on destroying each other’s capabilities commit to and obey rules designed to limit their choice of targets, weapons and tactics? Traditionally, answers to this question have been offered on one hand by moralists who regard law as being inspired by morality, and on other by realists who explain this branch of law on basis of reciprocity. Neither side's answers withstand close scrutiny. In this Article we develop an alternative explanation which is based on principal-agent model of domestic governance. We pry open black box of the state, and examine complex interaction between civilian and military apparatuses seething beneath veil of sovereignty. Our point of departure is that military conflicts raise significant intra-state conflicts of interest that result from delegation of authority to engage in combat: between civil society and elected officials, between elected officials and military commanders, and within military chain of command. We submit that most effective way to reduce domestic agency costs prevalent in war is by relying on external resources to monitor and discipline agents. Even though it may be costly, and reciprocity is not assured, principals who worry that agency slack may harm them or their nations' interests are likely to prefer that warfare be regulated by international norms. The Article expounds theory and uses it to explain evolution of law and its specific doctrines, and outlines normative implications of this new understanding of purpose of law. Ultimately, our analysis suggests that, as a practical matter, international law enhances ability of states to amass huge armies, because it lowers costs of controlling them. Therefore, although at times compliance with law may prove costly in short run, in long run it strengthens state against its enemies.

Using sensitivity analysis and visualization techniques to open black box data mining models

In this paper, we propose a new visualization approach based on a Sensitivity Analysis (SA) to extract human understandable knowledge from supervised learning black box data mining models, such as Neural Networks (NNs), Support Vector Machines (SVMs) and ensembles, including Random Forests (RFs). Five SA methods (three of which are purely new) and four measures of input importance (one novel) are presented. Also, the SA approach is adapted to handle discrete variables and to aggregate multiple sensitivity responses. Moreover, several visualizations for the SA results are introduced, such as input pair importance color matrix and variable effect characteristic surface. A wide range of experiments was performed in order to test the SA methods and measures by fitting four well-known models (NN, SVM, RF and decision trees) to synthetic datasets (five regression and five classification tasks). In addition, the visualization capabilities of the SA are demonstrated using four real-world datasets (e.g., bank direct marketing and white wine quality).

How Can I Help You? Call Centres, Classification Work and Coordination

As a comparatively novel but increasingly pervasive organizational arrangement, call centres have been a focus for much recent research. This paper identifies lessons for organizational and technological design through an examination of call centres and `classification work' --- explicating what Star [1992, Systems/Practice vol. 5, pp. 395---410] terms the `open black box'. Classification is a central means by which organizations standardize procedure, assess productivity, develop services and re-organize their business. Nevertheless, as Bowker and Star [1999, Sorting Things Out: Classification and Its Consequences. Cambridge MA: MIT Press] have pointed out, we know relatively little about the work that goes into making classification schema what they are. We will suggest that a focus on classification `work' in this context is a useful exemplar of the need for some kind of `meta-analysis' in ethnographic work also. If standardization is a major ambition for organizations under late capitalism, then comparison might be seen as a related but as-yet unrealized one for ethnographers. In this paper, we attempt an initial cut at a comparative approach, focusing on classification because it seemed to be the primary issue that emerged when we compared studies. Moreover, if technology is the principal means through which procedure and practice is implemented and if, as we believe, classifications are becoming ever more explicitly embedded within it (for instance with the development of so-called `semantic web' and associated approaches to ontology-based design), then there is clearly a case for identifying some themes which might underpin classification work in a given domain.

Opening the black boxes of global finance

ABSTRACT This paper advocates the application to global finance of one of the central heuristics of science studies: open the black box. Black boxes are devices, practices, or organizations that are opaque to outsiders, often because their contents are regarded as ‘technical’. The goal of opening black boxes is to discover how they are kept opaque; how they structure their ‘contexts’; and how those contexts are inscribed within them. Four types of black box in finance are discussed: option pricing theory; arbitrage; ‘ethnoaccountancy’; and regulation. The limitations of the opening of black boxes as an oppositional strategy are also discussed.

The open black box: the role of the end‐user in GIS integration

Formalist theories of knowledge that underpin GIS scholarship on integration neglect the importance and creativity of end‐users in knowledge construction. This has practical consequences for the success of large distributed databases that contribute to spatial‐data infrastructures. Spatial‐data infrastructures depend on participation at local levels, such as counties and watersheds, and they must be developed to support feedback from local users. Looking carefully at the work of scientists in a watershed in Puget Sound, Washington, USA during the salmon crisis reveals that the work of these end‐users articulates different worlds of knowledge. This view of the user is consonant with recent work in science and technology studies and research into computer‐supported cooperative work. GIS theory will be enhanced when it makes room for these users and supports their practical work.

Book Reviews : The cognitive neuropsychology of schizophrenia: opening black boxes and creating new ones

Book Reviews : The cognitive neuropsychology of schizophrenia: opening black boxes and creating new ones

An Anatomy of Engineering Knowledge

Nothing Unusual is Happening', in Roy Wallis (ed.), On the Margins of Science: The Social Construction of Rejected Knowledge, Sociological Review Monographs, No. 27 (1979), 237-69. 4. See Eugene Mallove's letter to Chemical and Engineering News (10 February 1992), 2. 5. M. Ashmore, 'The Theatre of the Blind', Social Studies of Science, Vol. 23, No. 1 (February 1993), 67-106, and T. Pinch and Ashmore, 'Defending the Daft and the Dangerous: Wilhelm Reich and the Strange Case of Orgone Energy', paper presented to 4S annual meeting, Gothenburg, Sweden, 13 August 1992. 6. T.J. Pinch, 'Opening Black Boxes: Science, Technology and Society', Social Studies of Science, Vol. 22 (1992), 487-510.

The trojan door: Organizations, work, and the ?open black box?

The paper summarizes recent work on organizations, artificial intelligence systems, human-computer interaction, etc., which emphasizes thesituated, distributed, andfluid nature of social systems. This contrasts with the traditional way of writing and thinking about social systems which sees them as disembodied, ideal, formal notions of thought. The implications of this new view of systems for social actors, information, knowledge, and technology are discussed. The literature reviewed offers a new way of talking about systems and their practices.

Opening Black Boxes: Science, Technology and Society

This lecture traces the theme of opening black boxes in the sociology of science. It is argued that now sociologists have got inside the black box of science, they need to make more impact upon the conduct of areas of science. The issue of what makes for `heroes' and `villains' within science is considered by tracing different styles of science. Two cases where scientific heroes became villains are examined: Andrew Crosse's claim to observe life emerging from a galvanic cell, and the claims by Pons and Fleischmann to observe cold fusion. The lecture concludes by considering possible characteristics which a hero within the field of STS would need to exhibit.