Machine Learning Experts Research Articles

Estimating Overhead Performance of Supervised Machine Learning Algorithms for Intrusion Detection

Estimating the energy and memory consumption of machine learning(ML) models for intrusion detection ensures efficient allocation of system resources. This study investigates the impact of supervised ML algorithms on the energy and memory consumption of intrusion detection systems. Experiments are conducted with seven ML algorithms and a proposed ensemble model, utilizing two intrusion detection datasets. Pearson correlation coefficient(PCC) and Spearman correlation coefficient are employed for the selection of optimum features. Regarding energy consumption, the findings reveal that the PCC with the UNSW-NB15 dataset uses the least amount of DRAM and CPU power. For ML methods, SVM utilizes the highest energy for both feature selection methods and datasets. Concerning memory consumption, the results show that decision tree uses the most current memory with PCC on the UNSW-NB15. The proposed ensemble model demonstrates the highest performance. These findings offer practical guidelines to ML experts when choosing the optimum model with the most efficient utilization of energy and memory.

ATICVis: A Visual Analytics System for Asymmetric Transformer Models Interpretation and Comparison

In recent years, natural language processing (NLP) technology has made great progress. Models based on transformers have performed well in various natural language processing problems. However, a natural language task can be carried out by multiple different models with slightly different architectures, such as different numbers of layers and attention heads. In addition to quantitative indicators such as the basis for selecting models, many users also consider the language understanding ability of the model and the computing resources it requires. However, comparing and deeply analyzing two transformer-based models with different numbers of layers and attention heads are not easy because it lacks the inherent one-to-one match between models, so comparing models with different architectures is a crucial and challenging task when users train, select, or improve models for their NLP tasks. In this paper, we develop a visual analysis system to help machine learning experts deeply interpret and compare the pros and cons of asymmetric transformer-based models when the models are applied to a user’s target NLP task. We propose metrics to evaluate the similarity between layers or attention heads to help users to identify valuable layers and attention head combinations to compare. Our visual tool provides an interactive overview-to-detail framework for users to explore when and why models behave differently. In the use cases, users use our visual tool to find out and explain why a large model does not significantly outperform a small model and understand the linguistic features captured by layers and attention heads. The use cases and user feedback show that our tool can help people gain insight and facilitate model comparison tasks.

Integrating Fairness in the Software Design Process: An Interview Study With HCI and ML Experts

The term Fairness is used in the field of Machine Learning to refer to a suite of evaluation techniques, measures, and process adjustments focused on the reduction of bias in statistical models. With many different potential definitions and methodologies for implementing fairness, some contradictory in goal or incompatible in approach, there are challenges in the selection of the most appropriate definition(s) to meet the ethical expectations society or end users may have for a model. This is further confounded by the potentially complex engineering challenges in applying the selected definition. The goal of designing and developing a fair system requires interdisciplinary collaboration between user experience researchers, focused on the socio-technical challenges of application usage and adoption, and data scientists and machine learning software developers, directly involved in the technical engineering and optimisation of such models. How these two groups interact and how their areas of specialisation can support and influence the other when developing fair AI is unknown. This leaves open questions about how these stakeholders view fairness, how it can be operationalised in their current work, and what are the obstacles/opportunities for improving the treatment of fairness in practice. In order to answer these questions, 18 subject matter experts from the fields of human-computer interaction (HCI) or Machine Learning (ML) were interviewed about their experience with fairness, their perception of current fairness adoption, and the challenges they would anticipate in a set of example scenarios. This work identifies some of the current challenges to developing fair artificial intelligence (AI). In dealing with these challenges, we propose approaches that include expanding the responsibilities of HCI and ML specialists, and identifying areas where joint responsibility is required.

Semantic graph based topic modelling framework for multilingual fake news detection

Fake news detection is one of the most alluring problems that has grabbed the interest of Machine Learning (ML) and Natural Language Processing (NLP) experts in recent years. The majority of existing studies on detecting fake news are written in English, restricting its application outside the English-speaking population. The lack of annotated corpora and technologies makes it difficult to identify false news in the scenario of low-resource languages, despite the growth in multilingual web content. Moreover, existing works cannot collect more semantic and contextual characteristics from documents in a particular multilingual text corpus. To bridge up these challenges and deal with the multilingual fake news detection challenge, we develop a new semantic graph attention-based representation learning framework to extract structural and semantic representations of texts. Our experiments on TALLIP fake news datasets showed that the classification performance had been significantly enhanced, ranging from 1% to 7% in terms of accuracy metric, and our proposed framework outperformed the state-of-the-art techniques for the multilingual fake news detection task.

Exploring Implementation Barriers of Machine Learning in Production Planning and Control

This paper studies the challenges encountered during the implementation of machine learning (ML) applications in production planning and control (PPC) functions of manufacturing companies. The application of ML in PPC promises numerous benefits, and several studies have revealed the positive impact of such an implementation on key performance indicators of production systems, such as lead time, due date reliability, and inventory levels. However, despite the theoretical potential and the increased research interest in the field over the last five years, practical applications remain rare and discourse on the existence of this gap between research and practice remains relatively sparse. In this paper, we identify general ML implementation barriers in the manufacturing domain based on a structured literature review and evaluate the relevance of these barriers within the PPC domain by interviewing ML and PPC experts. This research is beneficial for practitioners and researchers who aim to implement ML applications in PPC by enabling them to evaluate the possible barriers and challenges during implementation projects within their organizations.

Enhancing predictive accuracy of asset returns by experimenting with ML techniques

The unparalleled success of machine learning is indisputable. It has transformed the world with unimaginable solutions to insistent problems. The remarkable accuracy that machine learning manifests for making estimations is an object of fascination for plenty of researchers all over the world. The financial industry has also benefited from the growth of this electrifying field to predict asset returns, creditworthiness of a customer, and portfolio management, among others. In this research, we spotlight how this accuracy is contingent upon the analysis of various aspects of the data. We also experiment with simple techniques to make predictions and our findings suggest how these methods overshadow neural nets. The results indicate that the penalized linear models deliver the best performance. Random forest models had not been effective though. Machine learning models fitted with respect to median quantile loss were similarly observed to typically offer improvements across all machine learning models across all loss metrics. While little is known about the future of asset return that involves various risk and uncertainty, the recent enhancements in a machine learning field can contribute to deep domain training. Machine learning is increasingly gaining popularity nowadays in sectors including engineering, charity work, etc. Recently, even behavioral economics has started to leverage machine-learning expertise.

A narrative review on the use of camera traps and machine learning in wildlife research

Camera trapping has become an important tool in wildlife research in the past few decades. However, one of its main limiting factors is the processing of data, which is labour-intensive and time-consuming. Consequently, to aid this process, the use of machine learning has increased. A summary is provided on the use of both camera traps and machine learning and the main challenges that come with it by performing a general literature review. Remote cameras can be used in a variety of field applications, including investigating species distribution, disease transmission and vaccination, population estimation, nest predation, animal activity patterns, wildlife crossings, and diet analysis. Camera trapping has many benefits, including being less invasive, allowing for consistent monitoring and simultaneous observation (especially of secretive or aggressive animals even in dangerous or remote areas), providing photo/video evidence, reducing observer bias, and being cost effective. The main issues are that they are subject to their environment, dependent on human placements, can disrupt animal behaviour, need maintenance and repair, have limitations on photographic data, and are sensitive to theft and vandalism. When it comes to machine learning, the main aim is to identify species in camera (trap) images, although emerging technologies can provide individual recognition as well. The downsides in- clude the large amount of annotated data, computer power, and programming and machine learning expertise needed. Nonetheless, camera trapping and machine learning can greatly assist ecologists and conservationists in wildlife research, even more so as technology further develops.

HardVis: Visual Analytics to Handle Instance Hardness Using Undersampling and Oversampling Techniques

AbstractDespite the tremendous advances in machine learning (ML), training with imbalanced data still poses challenges in many real‐world applications. Among a series of diverse techniques to solve this problem, sampling algorithms are regarded as an efficient solution. However, the problem is more fundamental, with many works emphasizing the importance of instance hardness. This issue refers to the significance of managing unsafe or potentially noisy instances that are more likely to be misclassified and serve as the root cause of poor classification performance.This paper introduces HardVis, a visual analytics system designed to handle instance hardness mainly in imbalanced classification scenarios. Our proposed system assists users in visually comparing different distributions of data types, selecting types of instances based on local characteristics that will later be affected by the active sampling method, and validating which suggestions from undersampling or oversampling techniques are beneficial for the ML model. Additionally, rather than uniformly undersampling/oversampling a specific class, we allow users to find and sample easy and difficult to classify training instances from all classes. Users can explore subsets of data from different perspectives to decide all those parameters, while HardVis keeps track of their steps and evaluates the model's predictive performance in a test set separately. The end result is a well‐balanced data set that boosts the predictive power of the ML model. The efficacy and effectiveness of HardVis are demonstrated with a hypothetical usage scenario and a use case. Finally, we also look at how useful our system is based on feedback we received from ML experts.

Galaxy image classification using hierarchical data learning with weighted sampling and label smoothing

ABSTRACT With the development of a series of Galaxy sky surveys in recent years, the observations increased rapidly, which makes the research of machine learning methods for galaxy image recognition a hot topic. Available automatic galaxy image recognition researches are plagued by the large differences in similarity between categories, the imbalance of data between different classes, and the discrepancy between the discrete representation of Galaxy classes (DDRGC) and the essentially gradual changes from one morphological class to the adjacent class. These limitations have motivated several astronomers and machine learning experts to design projects with improved galaxy image recognition capabilities. Therefore, this paper proposes a novel learning method, ‘Hierarchical Imbalanced data learning with Weighted sampling and Label smoothing’ (HIWL). The HIWL consists of three key techniques respectively dealing with the above-mentioned three problems: (1) designed a hierarchical galaxy classification model based on an efficient backbone network; (2) utilized a weighted sampling scheme to deal with the imbalance problem; and (3) adopted a label smoothing technique to alleviate the DDRGC problem. We applied this method to galaxy photometric images from the Galaxy Zoo-The Galaxy Challenge, exploring the recognition of completely round smooth, in between smooth, cigar-shaped, edge-on, and spiral. The overall classification accuracy is 96.32 per cent, and some superiorities of the HIWL are shown based on recall, precision, and F1-Score in comparing with some related works. In addition, we also explored the visualization of the galaxy image features and model attention to understand the foundations of the proposed scheme.

Implementation of Machine Learning Pipelines for Clinical Practice: Development and Validation Study

Artificial intelligence (AI) technologies, such as machine learning and natural language processing, have the potential to provide new insights into complex health data. Although powerful, these algorithms rarely move from experimental studies to direct clinical care implementation. We aimed to describe the key components for successful development and integration of two AI technology-based research pipelines for clinical practice. We summarized the approach, results, and key learnings from the implementation of the following two systems implemented at a large, tertiary care children's hospital: (1) epilepsy surgical candidate identification (or epilepsy ID) in an ambulatory neurology clinic; and (2) an automated clinical trial eligibility screener (ACTES) for the real-time identification of patients for research studies in a pediatric emergency department. The epilepsy ID system performed as well as board-certified neurologists in identifying surgical candidates (with a sensitivity of 71% and positive predictive value of 77%). The ACTES system decreased coordinator screening time by 12.9%. The success of each project was largely dependent upon the collaboration between machine learning experts, research and operational information technology professionals, longitudinal support from clinical providers, and institutional leadership. These projects showcase novel interactions between machine learning recommendations and providers during clinical care. Our deployment provides seamless, real-time integration of AI technology to provide decision support and improve patient care.

An Educational Graphical User Interface to Construct Convolutional Neural Networks for Teaching Artificial Intelligence in Radiology.

Deep learning techniques using convolutional neural networks (CNNs) have been successfully developed for various medical image analysis tasks. However, the skills to understand and develop deep learning models are not usually taught during radiology training, which constitutes a barrier for radiologists looking to integrate machine learning (ML) into their research or clinical practice. In this work, we developed and evaluated an educational graphical user interface (GUI) to construct CNNs for teaching deep learning concepts to radiology trainees. The GUI was developed in Python using the PyQt and PyTorch frameworks. The functionality of the GUI was demonstrated through a binary classification task on a dataset of MR images of the brain. The usability of the GUI was assessed through 45-min user testing sessions with 5 neuroradiologists and neuroradiology fellows, assessing mean task completion times, the System Usability Scale (SUS), and a qualitative questionnaire as metrics. Task completion times were compared against a ML expert who performed the same tasks. After a 20-min introduction to CNNs and a walkthrough of the GUI, users were able to perform all assigned tasks successfully. There was no significant difference in task completion time compared to a ML expert. The educational GUI achieved a score of 82.5 on the SUS, suggesting that the system is highly usable. Users indicated that the GUI seems useful as an educational tool to teach ML topics to radiology trainees. An educational GUI allows interactive teaching in ML that can be incorporated into radiology training.

Outcome-Explorer: A Causality Guided Interactive Visual Interface for Interpretable Algorithmic Decision Making.

The widespread adoption of algorithmic decision-making systems has brought about the necessity to interpret the reasoning behind these decisions. The majority of these systems are complex black box models, and auxiliary models are often used to approximate and then explain their behavior. However, recent research suggests that such explanations are not overly accessible to lay users with no specific expertise in machine learning and this can lead to an incorrect interpretation of the underlying model. In this article, we show that a predictive and interactive model based on causality is inherently interpretable, does not require any auxiliary model, and allows both expert and non-expert users to understand the model comprehensively. To demonstrate our method we developed Outcome Explorer, a causality guided interactive interface, and evaluated it by conducting think-aloud sessions with three expert users and a user study with 18 non-expert users. All three expert users found our tool to be comprehensive in supporting their explanation needs while the non-expert users were able to understand the inner workings of a model easily.

Ten quick tips for sequence-based prediction of protein properties using machine learning.

The ubiquitous availability of genome sequencing data explains the popularity of machine learning-based methods for the prediction of protein properties from their amino acid sequences. Over the years, while revising our own work, reading submitted manuscripts as well as published papers, we have noticed several recurring issues, which make some reported findings hard to understand and replicate. We suspect this may be due to biologists being unfamiliar with machine learning methodology, or conversely, machine learning experts may miss some of the knowledge needed to correctly apply their methods to proteins. Here, we aim to bridge this gap for developers of such methods. The most striking issues are linked to a lack of clarity: how were annotations of interest obtained; which benchmark metrics were used; how are positives and negatives defined. Others relate to a lack of rigor: If you sneak in structural information, your method is not sequence-based; if you compare your own model to "state-of-the-art," take the best methods; if you want to conclude that some method is better than another, obtain a significance estimate to support this claim. These, and other issues, we will cover in detail. These points may have seemed obvious to the authors during writing; however, they are not always clear-cut to the readers. We also expect many of these tips to hold for other machine learning-based applications in biology. Therefore, many computational biologists who develop methods in this particular subject will benefit from a concise overview of what to avoid and what to do instead.

Anomaly detection in laser powder bed fusion using machine learning: A review

Metal Additive Manufacturing (MAM) applications are growing rapidly in high-tech industries such as biomedical and aerospace, and in many other industries including tooling, casting, automotive, oil and gas for production and prototyping. The onset of Laser Powder Bed Fusion (L-PBF) technology proved to be an efficient technique that can convert metal additive manufacturing into a reformed process if anomalies occurred during this process are eliminated. Industrial applications demand high accuracy and risk-free products whereas prototyping using MAM demand lower process and product development time. In order to address these challenges, Machine Learning (ML) experts and researchers are trying to adopt an efficient method for anomaly detection in L-PBF so that the MAM process can be optimized and desired final part properties can be achieved. This review provides an overview of L-PBF and outlines the ML methods used for anomaly detection in L-PBF. The paper also explains how ML methods are being used as a step forward toward enabling the real-time process control of MAM and the process can be optimized for higher accuracy, lower production time, and less material waste. Authors have a strong believe that ML techniques can reform MAM process, whereas research concerned to the anomaly detection using ML techniques is limited and needs attention.This review has been done with a hope that ML experts can easily find a direction and contribute in this field.

Gesture Spotter: A Rapid Prototyping Tool for Key Gesture Spotting in Virtual and Augmented Reality Applications.

In this paper we examine the task of key gesture spotting: accurate and timely online recognition of hand gestures. We specifically seek to address two key challenges faced by developers when integrating key gesture spotting functionality into their applications. These are: i) achieving high accuracy and zero or negative activation lag with single-time activation; and ii) avoiding the requirement for deep domain expertise in machine learning. We address the first challenge by proposing a key gesture spotting architecture consisting of a novel gesture classifier model and a novel single-time activation algorithm. This key gesture spotting architecture was evaluated on four separate hand skeleton gesture datasets, and achieved high recognition accuracy with early detection. We address the second challenge by encapsulating different data processing and augmentation strategies, as well as the proposed key gesture spotting architecture, into a graphical user interface and an application programming interface. Two user studies demonstrate that developers are able to efficiently construct custom recognizers using both the graphical user interface and the application programming interface.

From diagnosis to post-treatment: biopsychosocial predictors of sexual and mental health in prostate cancer survivors

The increasing incidence of prostate cancer, along with earlier detection and longer survival years has been raising clinician's questions on how to better treat this disease. Radical prostatectomy is one of the most common treatments, although it can have severe consequences, such as changes in sexual functioning and sexual satisfaction, thus affecting the quality of life and mental health. These difficulties can significantly decrease the adherence to treatment protocols and lead to drop-outs. Under the Horizon Europe funding programme for research and innovation, this multicentric project, combining the specialities of urology, psychology, and neuroscience, aims to investigate (1) the biopsychosocial predictors of mental and sexual health in prostate cancer and (2) the neural correlates of sexual functioning in prostate cancer, so as to develop a predictive model regarding post-treatment adjustment and sexual well-being. 300 Men with prostate cancer indicated to radical prostatectomy will be assessed before surgery and nine months afterwards. A subset of 250 individuals will participate in semi-structured interviews and complete self-report psychological measures so as to monitor sexual and mental health. The remainder will take part in a laboratory experiment, with neuroimaging techniques and other physiological measures, in order to identify the neural underpinnings of sexual response in relation to prostate cancer and radical prostatectomy. Results will contribute to the development of a mathematical algorithm, built by Machine Learning experts, that will help predict the biopsychosocial risk and protective factors for sexual health and mental well-being in men with prostate cancer. This study tackles an important health issue in an innovative way, by combining a longitudinal and a laboratorial/experimental design and integrating a wide range of procedures. Furthermore, the development of a prediction model that covers such broad sexual health determinants, is a promising tool that can help clinicians improve their intervention strategies. All the authors declate no known conflict of interest. This project is funded by Horizonte Europa (NORTE-01-0145-FEDER-000057), via the Support System for Scientific and Tecnological Research NORTE-45-2020-75, CCDRN.

From diagnosis to post-treatment: biopsychosocial predictors of sexual and mental health in prostate cancer survivors

<h3>Objectives</h3> The increasing incidence of prostate cancer, along with earlier detection and longer survival years has been raising clinician's questions on how to better treat this disease. Radical prostatectomy is one of the most common treatments, although it can have severe consequences, such as changes in sexual functioning and sexual satisfaction, thus affecting the quality of life and mental health. These difficulties can significantly decrease the adherence to treatment protocols and lead to drop-outs. Under the Horizon Europe funding programme for research and innovation, this multicentric project, combining the specialities of urology, psychology, and neuroscience, aims to investigate (1) the biopsychosocial predictors of mental and sexual health in prostate cancer and (2) the neural correlates of sexual functioning in prostate cancer, so as to develop a predictive model regarding post-treatment adjustment and sexual well-being. <h3>Methods</h3> 300 Men with prostate cancer indicated to radical prostatectomy will be assessed before surgery and nine months afterwards. A subset of 250 individuals will participate in semi-structured interviews and complete self-report psychological measures so as to monitor sexual and mental health. The remainder will take part in a laboratory experiment, with neuroimaging techniques and other physiological measures, in order to identify the neural underpinnings of sexual response in relation to prostate cancer and radical prostatectomy. <h3>Results</h3> Results will contribute to the development of a mathematical algorithm, built by Machine Learning experts, that will help predict the biopsychosocial risk and protective factors for sexual health and mental well-being in men with prostate cancer. <h3>Conclusions</h3> This study tackles an important health issue in an innovative way, by combining a longitudinal and a laboratorial/experimental design and integrating a wide range of procedures. Furthermore, the development of a prediction model that covers such broad sexual health determinants, is a promising tool that can help clinicians improve their intervention strategies. <h3>Conflicts of Interest</h3> All the authors declate no known conflict of interest. This project is funded by Horizonte Europa (NORTE-01-0145-FEDER-000057), via the Support System for Scientific and Tecnological Research NORTE-45-2020-75, CCDRN.

Model Protection: Real-Time Privacy-Preserving Inference Service for Model Privacy at the Edge

Major cloud service providers with well-equipped infrastructure, experienced machine learning (ML) expertise, and enriched training datasets are building ML-as-a-Service (MLaaS) systems, in which clients can query ML-based prediction services with their data. Instead of moving private data to the cloud, in this work, we design, implement, and evaluate a novel secure ML system to enable MLaaS on edge devices. To protect the proprietary ML models on edge devices from revealing to the clients while maintaining a real-time inference is challenging. Existing privacy-preserving ML techniques can hardly satisfy real-time requirements. In our solution, we employ a secure enclave (e.g., SGX) to offer security and provide better efficiency than cryptographic techniques. However, the enclave alone cannot achieve real-time capability due to its limited capacity. We observe that the ML model imposes a severe accuracy degradation when adding noise to a few model weights. Based on this, we design a suite of novel solutions to optimize the performance of secure enclave-based inference service at the edge by enclosing only <inline-formula><tex-math notation="LaTeX">$1\%$</tex-math></inline-formula> computation within secure enclaves. Our work can achieve up to a <inline-formula><tex-math notation="LaTeX">$7.8\times$</tex-math></inline-formula> increase in efficiency and a <inline-formula><tex-math notation="LaTeX">$27\times$</tex-math></inline-formula> reduction in memory usage compared to the state-of-the-art.

Artificial intelligence in healthcare: Setting new algo RHYTHM in medicine

Artificial intelligence (AI) is a rapidly expanding avenue in science and is often used to describe the use of computers, big data mining, modeling, and network that may simulate intelligent behavior and critical thinking similar to a human being but at an exceedingly fast pace. What lies at the heart of AI are intelligent algorithms that drive the whole process integratively, imparting the decisive ability to machines. The artificial intelligence market is expected to grow at a rate of 39.4% by 2022-2028. British mathematician Alan Turing’s 1950 paper “Computing Machinery and Intelligence " established AI's fundamental goal and vision. Fundamentally, AI is a branch of computer science that aims to answer Turing’s question of replicating human intelligence as machines. Since AI research aims to make smarter algorithms evolve and emulate human-like functioning, the degree to which an AI system can replicate human capabilities is used to determine the types of AI. The volume, magnitude, and complexity of data in healthcare mean that artificial intelligence (AI) will be increasingly applied and exploited, with several types of AI already being employed by researchers, clinicians, and data analysts. AI is a collective term, not just one technology, with an immediate relevance to healthcare. Few AI technologies of high importance to healthcare are: Machine learning, Neural Networks, and Deep Learning Natural Language Processing ( NLP) Physical Robots Rule-based expert systems The AI-associated healthcare market is growing at an exponential pace and is expected to reach USD 6.6 billion by 2021. Precision medicine initiatives can be divided into three types of clinical areas: complex algorithms, digital health applications, and “omics”-based tests. Large genomic datasets and informatics related to demographic data or diagnostic reports are exploited by machine learning algorithms for prediction and prognosis. This is coupled with patient healthcare data, such as treatment protocols, treatment response, response to therapy, and health monitoring, with data input from clinicians, researchers, and the public domain. High throughput big data from population-based studies are used with machine learning algorithms to establish correlations, build predictive models, and provide customized treatment protocols. In Omics based era, data from genomics, proteomics, and metabolomics are exceedingly used to address and identify health-related problems in large study cohorts . This will provide opportunities for translation and transition from bench to bedside through precision medicine.

Creative Wand: A System to Study Effects of Communications in Co-creative Settings

Recent neural generation systems have demonstrated the potential for procedurally generating game content, images, stories, and more. However, most neural generation algorithms are “uncontrolled” in the sense that the user has little say in creative decisions beyond the initial prompt specification. Co-creative, mixed-initiative systems require user-centric means of influencing the algorithm, especially when users are unlikely to have machine learning expertise. The key to co-creative systems is the ability to communicate ideas and intent from the user to the agent, as well as from the agent to the user. Key questions in co-creative AI include: How can users express their creative intentions? How can creative AI systems communicate their beliefs, explain their moves, or instruct users to act on their behalf? When should creative AI systems take initiative? The answer to such questions and more will enable us to develop better co-creative systems that make humans more capable of expressing their creative intents. We introduce CREATIVE-WAND, a customizable framework for investigating co-creative mixed-initiative generation. CREATIVE-WAND enables plug-and-play injection of generative models and human-agent communication channels into a chat-based interface. It provides a number of dimensions along which an AI generator and humans can communicate during the co-creative process. We illustrate the CREATIVE-WAND framework by using it to study one dimension of co-creative communication—global versus local creative intent specification by the user—in the context of storytelling.