Machine Learning Experts Research Articles

Machine Learning Applications in Surface Transportation Systems: A Literature Review

Surface transportation has evolved through technology advancements using parallel knowledge areas such as machine learning (ML). However, the transportation industry has not yet taken full advantage of ML. To evaluate this gap, we utilized a literature review approach to locate, categorize, and synthesize the principal concepts of research papers regarding surface transportation systems using ML algorithms, and we then decomposed them into their fundamental elements. We explored more than 100 articles, literature review papers, and books. The results show that 74% of the papers concentrate on forecasting, while multilayer perceptions, long short-term memory, random forest, supporting vector machine, XGBoost, and deep convolutional neural networks are the most preferred ML algorithms. However, sophisticated ML algorithms have been minimally used. The root-cause analysis revealed a lack of effective collaboration between the ML and transportation experts, resulting in the most accessible transportation applications being used as a case study to test or enhance a given ML algorithm and not necessarily to enhance a mobility or safety issue. Additionally, the transportation community does not define transportation issues clearly and does not provide publicly available transportation datasets. The transportation sector must offer an open-source platform to showcase the sector’s concerns and build spatiotemporal datasets for ML experts to accelerate technology advancements.

Developing Clinical Artificial Intelligence for Obstetric Ultrasound to Improve Access in Underserved Regions: Protocol for a Computer-Assisted Low-Cost Point-of-Care UltraSound (CALOPUS) Study.

BackgroundThe World Health Organization recommends a package of pregnancy care that includes obstetric ultrasound scans. There are significant barriers to universal access to antenatal ultrasound, particularly because of the cost and need for maintenance of ultrasound equipment and a lack of trained personnel. As low-cost, handheld ultrasound devices have become widely available, the current roadblock is the global shortage of health care providers trained in obstetric scanning.ObjectiveThe aim of this study is to improve pregnancy and risk assessment for women in underserved regions. Therefore, we are undertaking the Computer-Assisted Low-Cost Point-of-Care UltraSound (CALOPUS) project, bringing together experts in machine learning and clinical obstetric ultrasound.MethodsIn this prospective study conducted in two clinical centers (United Kingdom and India), participating pregnant women were scanned and full-length ultrasounds were performed. Each woman underwent 2 consecutive ultrasound scans. The first was a series of simple, standardized ultrasound sweeps (the CALOPUS protocol), immediately followed by a routine, full clinical ultrasound examination that served as the comparator. We describe the development of a simple-to-use clinical protocol designed for nonexpert users to assess fetal viability, detect the presence of multiple pregnancies, evaluate placental location, assess amniotic fluid volume, determine fetal presentation, and perform basic fetal biometry. The CALOPUS protocol was designed using the smallest number of steps to minimize redundant information, while maximizing diagnostic information. Here, we describe how ultrasound videos and annotations are captured for machine learning.ResultsOver 5571 scans have been acquired, from which 1,541,751 label annotations have been performed. An adapted protocol, including a low pelvic brim sweep and a well-filled maternal bladder, improved visualization of the cervix from 28% to 91% and classification of placental location from 82% to 94%. Excellent levels of intra- and interannotator agreement are achievable following training and standardization.ConclusionsThe CALOPUS study is a unique study that uses obstetric ultrasound videos and annotations from pregnancies dated from 11 weeks and followed up until birth using novel ultrasound and annotation protocols. The data from this study are being used to develop and test several different machine learning algorithms to address key clinical diagnostic questions pertaining to obstetric risk management. We also highlight some of the challenges and potential solutions to interdisciplinary multinational imaging collaboration.International Registered Report Identifier (IRRID)RR1-10.2196/37374

Hybrid Models That Combine Machine Learning and Simulations

Simulation experts are now well acquainted with machine learning (ML) techniques, using them to find patterns in data that can later be turned into rules of a simulation or enabling their simulated entities to adapt and learn. In the other direction, ML experts occasionally make use of simulated data to create controlled experiments in which learning algorithms can be evaluated. In this article, we go beyond these typical uses by focusing on current opportunities that have the potential to bring the two research communities together. These opportunities can be realized in areas where the potential of hybrid ML/simulation methods has not been fully attained yet. Such applications also motivate the development of innovative methods, for example, to combine the accuracy of ML with the interpretability of simulation models. Using select examples from our interdisciplinary team, this article reflects on opportunities in applications and techniques to promote productive conversations across research areas.

Griffin: Real-Time Network Intrusion Detection System via Ensemble of Autoencoder in SDN

Many efforts have been devoted to the development of efficient Network Intrusion Detection System (NIDS) using machine learning approaches in Software-defined Network (SDN). Unfortunately, existing solutions failed to detect real-time and zero-day attacks due to their limited throughput and prior knowledge-based detection. To this end, we propose Griffin, a NIDS that uses unsupervised machine learning expertise to detect both known and zero-day intrusion attacks in real-time with high accuracy. Specifically, Griffin uses an efficient feature extraction framework to capture the sequential features of the traffic packets. Then, it utilizes cluster analysis to reduce the feature scale to achieve low throughput. Moreover, an ensemble autoencoder is built automatically to further extract features with low complexity and high precision to train the model. We evaluate the accuracy, robustness, and complexity of the system using open datasets. The result shows that Griffin’s complexity is about 40% lower, and its accuracy is at most 19% higher than existing NIDS.Additionally, even in the situation with evasion, the Griffin has at most 9% decrease of AUC, which is a good performance compared with other solutions. Furthermore, this paper also utilizes the differential privacy framework during training autoencoders to protect datasets’ privacy which is inherent in machine learning approaches.

Beyond effective use: Integrating wise reasoning in machine learning development

The introduction of machine learning (ML), as the engine of many artificial intelligence (AI)-enabled systems in organizations, comes with the claim that ML models provide automated decisions or help domain experts improve their decision-making. Such a claim gives rise to the need to keep domain experts in the loop. Hence, data scientists, as those who develop ML models and infuse them with human intelligence during ML development, interact with various ML stakeholders and reflect their views within ML models. This interaction comes with (often conflicting) demands from various ML stakeholders and potential tensions. Building on the theories of effective use and wise reasoning, this mixed method study proposes a model to better understand how data scientists can use wisdom for managing these tensions when they develop ML models. In Study 1, through interviewing 41 analytics and ML experts, we investigate the dimensions of wise reasoning in the context of ML development. In Study 2, we test the overall model using a sample of 249 data scientists. Our results confirm that to develop effective ML models, data scientists need to not only use ML systems effectively, but also practice wise reasoning in their interactions with domain experts. We discuss the implications of these findings for research and practice.

Demonstration of collaborative and interactive workflow-based data analytics in texera

Collaborative data analytics is becoming increasingly important due to the higher complexity of data science, more diverse skills from different disciplines, more common asynchronous schedules of team members, and the global trend of working remotely. In this demo we will show how Texera supports this emerging computing paradigm to achieve high productivity among collaborators with various backgrounds. Based on our active joint projects on the system, we use a scenario of social media analysis to show how a data science task can be conducted on a user friendly yet powerful platform by a multi-disciplinary team including domain scientists with limited coding skills and experienced machine learning experts. We will present how to do collaborative editing of a workflow and collaborative execution of the workflow in Texera. We will focus on data-centric features such as synchronization of operator schemas among the users during the construction phase, and monitoring and controlling the shared runtime during the execution phase.

Toward Involving End-users in Interactive Human-in-the-loop AI Fairness

Ensuring fairness in artificial intelligence (AI) is important to counteract bias and discrimination in far-reaching applications. Recent work has started to investigate how humans judge fairness and how to support machine learning experts in making their AI models fairer. Drawing inspiration from an Explainable AI approach called explanatory debugging used in interactive machine learning, our work explores designing interpretable and interactive human-in-the-loop interfaces that allow ordinary end-users without any technical or domain background to identify potential fairness issues and possibly fix them in the context of loan decisions. Through workshops with end-users, we co-designed and implemented a prototype system that allowed end-users to see why predictions were made, and then to change weights on features to “debug” fairness issues. We evaluated the use of this prototype system through an online study. To investigate the implications of diverse human values about fairness around the globe, we also explored how cultural dimensions might play a role in using this prototype. Our results contribute to the design of interfaces to allow end-users to be involved in judging and addressing AI fairness through a human-in-the-loop approach.

The Core Human Microbiome: Does It Exist and How Can We Find It? A Critical Review of the Concept.

The core microbiome, which refers to a set of consistent microbial features across populations, is of major interest in microbiome research and has been addressed by numerous studies. Understanding the core microbiome can help identify elements that lead to dysbiosis, and lead to treatments for microbiome-related health states. However, defining the core microbiome is a complex task at several levels. In this review, we consider the current state of core human microbiome research. We consider the knowledge that has been gained, the factors limiting our ability to achieve a reliable description of the core human microbiome, and the fields most likely to improve that ability. DNA sequencing technologies and the methods for analyzing metagenomics and amplicon data will most likely facilitate higher accuracy and resolution in describing the microbiome. However, more effort should be invested in characterizing the microbiome’s interactions with its human host, including the immune system and nutrition. Other components of this holobiontic system should also be emphasized, such as fungi, protists, lower eukaryotes, viruses, and phages. Most importantly, a collaborative effort of experts in microbiology, nutrition, immunology, medicine, systems biology, bioinformatics, and machine learning is probably required to identify the traits of the core human microbiome.

Toward a sociology of machine learning explainability: Human–machine interaction in deep neural network-based automated trading

Machine learning systems are making considerable inroads in society owing to their ability to recognize and predict patterns. However, the decision-making logic of some widely used machine learning models, such as deep neural networks, is characterized by opacity, thereby rendering them exceedingly difficult for humans to understand and explain and, as a result, potentially risky to use. Considering the importance of addressing this opacity, this paper calls for research that studies empirically and theoretically how machine learning experts and users seek to attain machine learning explainability. Focusing on automated trading, we take steps in this direction by analyzing a trading firm’s quest for explaining its deep neural network system’s actionable predictions. We demonstrate that this explainability effort involves a particular form of human–machine interaction that contains both anthropomorphic and technomorphic elements. We discuss this attempt to attain machine learning explainability in light of reflections on cross-species companionship and consider it an example of human–machine companionship.

Challenges and Benchmark Datasets for Machine Learning in the Atmospheric Sciences: Definition, Status, and Outlook

Abstract Benchmark datasets and benchmark problems have been a key aspect for the success of modern machine learning applications in many scientific domains. Consequently, an active discussion about benchmarks for applications of machine learning has also started in the atmospheric sciences. Such benchmarks allow for the comparison of machine learning tools and approaches in a quantitative way and enable a separation of concerns for domain and machine learning scientists. However, a clear definition of benchmark datasets for weather and climate applications is missing with the result that many domain scientists are confused. In this paper, we equip the domain of atmospheric sciences with a recipe for how to build proper benchmark datasets, a (nonexclusive) list of domain-specific challenges for machine learning is presented, and it is elaborated where and what benchmark datasets will be needed to tackle these challenges. We hope that the creation of benchmark datasets will help the machine learning efforts in atmospheric sciences to be more coherent, and, at the same time, target the efforts of machine learning scientists and experts of high-performance computing to the most imminent challenges in atmospheric sciences. We focus on benchmarks for atmospheric sciences (weather, climate, and air-quality applications). However, many aspects of this paper will also hold for other aspects of the Earth system sciences or are at least transferable. Significance Statement Machine learning is the study of computer algorithms that learn automatically from data. Atmospheric sciences have started to explore sophisticated machine learning techniques and the community is making rapid progress on the uptake of new methods for a large number of application areas. This paper provides a clear definition of so-called benchmark datasets for weather and climate applications that help to share data and machine learning solutions between research groups to reduce time spent in data processing, to generate synergies between groups, and to make tool developments more targeted and comparable. Furthermore, a list of benchmark datasets that will be needed to tackle important challenges for the use of machine learning in atmospheric sciences is provided.

Major Challenges and Future Approaches in the Employment of Blockchain and Machine Learning Techniques in the Health and Medicine

According to the benefits in safeguarding and transferring medical information, illness assessment, evaluation of “Magnetic Resonance Mapping” images, and certain other disciplines, blockchain and machine learning (ML) technology has significantly piqued attention in the healthcare domains. Formerly, those chores have been performed out along with individuals; eventually, individuals acquired attraction because to its precision and efficiency. The proposed study will examine the activities and possible capabilities of learning algorithms and blockchain in the healthcare professions focusing on these fascinating facts. Primary and secondary data analysis has been executed, with primary analysis method consisting of a survey of 150 randomly picked medicine professionals with expertise in machine learning and blockchain. They gave their answers that were being subsequently transferred to figures and employed as response variable in SPSS examining. The length of time where learning and blockchain have been used in medicine is really the independent factor. To better understand the primary and small hurdles of integrating machine learning and blockchain, a correlation investigation was done. Thereafter, secondary methodology is employed to validate the primary study results.

ModelWise: Interactive Model Comparison for Model Diagnosis, Improvement and Selection

AbstractModel comparison is an important process to facilitate model diagnosis, improvement, and selection when multiple models are developed for a classification task. It involves careful comparison concerning model performance and interpretation. Current visual analytics solutions often ignore the feature selection process. They either do not support detailed analysis of multiple multi‐class classifiers or rely on feature analysis alone to interpret model results. Understanding how different models make classification decisions, especially classification disagreements of the same instances, requires a deeper model understanding. We present ModelWise, a visual analytics method to compare multiple multi‐class classifiers in terms of model performance, feature space, and model explanation. ModelWise adapts visualizations with rich interactions to support multiple workflows to achieve model diagnosis, improvement, and selection. It considers feature subspaces generated for use in different models and improves model understanding by model explanation. We demonstrate the usability of ModelWise with two case studies, one with a small exemplar dataset and another developed with a machine learning expert with real‐world perioperative data.

Artificial intelligence for right whale photo identification: from data science competition to worldwide collaboration

Photo identification is an important tool in the conservation management of endangered species, and recent developments in artificial intelligence are revolutionizing existing workflows to identify individual animals. In 2015, the National Oceanic and Atmospheric Administration hosted a Kaggle data science competition to automate the identification of endangered North Atlantic right whales (Eubalaena glacialis). The winning algorithms developed by Deepsense.ai were able to identify individuals with 87% accuracy using a series of convolutional neural networks to identify the region of interest, create standardized photographs of uniform size and orientation, and then identify the correct individual. Since that time, we have brought in many more collaborators as we moved from prototype to production. Leveraging the existing infrastructure by Wild Me, the developers of Flukebook, we have created a web-based platform that allows biologists with no machine learning expertise to utilize semi-automated photo identification of right whales. New models were generated on an updated dataset using the winning Deepsense.ai algorithms. Given the morphological similarity between the North Atlantic right whale and closely related southern right whale (Eubalaena australis), we expanded the system to incorporate the largest long-term photo identification catalogs around the world including the United States, Canada, Australia, South Africa, Argentina, Brazil, and New Zealand. The system is now fully operational with multi-feature matching for both North Atlantic right whales and southern right whales from aerial photos of their heads (Deepsense), lateral photos of their heads (Pose Invariant Embeddings), flukes (CurvRank v2), and peduncle scarring (HotSpotter). We hope to encourage researchers to embrace both broad data collaborations and artificial intelligence to increase our understanding of wild populations and aid conservation efforts.

Overcoming barriers in academic-industry partnerships to improve predictive modeling in immuno-oncology.

e13581 Background: In the past decade, immunotherapies have revolutionized oncology practice by prolonging patient survival in previously rapidly fatal cancers. However, severe immune toxicities present a challenge, affecting ̃20% and up to 50% of patients on immune monotherapies and combination immunotherapies, respectively. Oncologists must balance toxicity risk with potential efficacy, and pharmaceutical companies have a vested interest in selecting patients with the highest benefit–risk ratio during trial enrollment. Predictive toxicity–efficacy modeling has the potential to guide trial subject selection and clinical care, yet there remains a need for predictive models that can be practically implemented in these settings. Methods: A common academic–industry contract data–transfer framework—wherein academic medical institutions and industry counterparts act in isolation—creates barriers to development of high-quality algorithms with practical applications. In this framework, 1) academic medical institutions provide patient data as a “data dump;” these data are static and cannot be refined—reducing opportunities for quality control; 2) predictive model outcomes may include artifacts that are not identified; 3) manual curation of patient data is required to accurately replicate the model in real-world settings; and 4) lack of clinician participation reduces the potential clinical applications of models and reciprocal benefit to the academic institution. We outline a more contemporary, engaged approach to unite strengths of both partners to achieve a common goal. Results: In 2019, Vanderbilt University Medical Center and GE Healthcare partnered with the goal of enabling safer and more precise immunotherapy use. As part of this work, we formulated a recipe for academic–industry partnerships that offers unique advantages over a static contract framework. In our iterative, interactive approach, 1) clinical and curation experts meet with industry modelers to dynamically refine deidentified data sets by resolving discrepancies in data from different sources (e.g., manually curated vs. structured data); 2) clinical experts iteratively review outputs of predictive models to identify potential artifacts and refine final models; 3) expert curators iterate with in-house machine-learning experts to create algorithms to automate curation of natural language elements from the identified EHR data; and 4) clinical and industry stakeholders participate in regular meetings with modelers to ensure clinical and trial utility of the modeling approach. Conclusions: Compared to data transfer-only relationships, this partnership framework offers an opportunity to develop more informed, higher quality immunotherapy models with clinical and industry applications.

DeepOtolith v1.0: An Open-Source AI Platform for Automating Fish Age Reading from Otolith or Scale Images

Every year, marine scientists around the world read thousands of otolith or scale images to determine the age structure of commercial fish stocks. This knowledge is important for fisheries and conservation management. However, the age-reading procedure is time-consuming and costly to perform due to the specialized expertise and labor needed to identify annual growth zones in otoliths. Effective automated systems are needed to increase throughput and reduce cost. DeepOtolith is an open-source artificial intelligence (AI) platform that addresses this issue by providing a web system with a simple interface that automatically estimates fish age by combining otolith images with convolutional neural networks (CNNs), a class of deep neural networks that has been a dominant method in computer vision tasks. Users can upload otolith image data for selective fish species, and the platform returns age estimates. The estimates of multiple images can be exported to conduct conclusions or further age-related research. DeepOtolith currently contains classifiers/regressors for three fish species; however, more species will be included as related work on ageing will be tested and published soon. Herein, the architecture and functionality of the platform are presented. Current limitations and future directions are also discussed. Overall, DeepOtolith should be considered as the first step towards building a community of marine ecologists, machine learning experts, and stakeholders that will collaborate to support the conservation of fishery resources.

Towards sustainable North American wood product value chains, part 2: computer vision identification of ring-porous hardwoods

Wood identification is vitally important for ensuring the legality of North American hardwood value chains. Computer vision wood identification (CVWID) systems can identify wood without necessitating costly and time-consuming off-site visual inspections by highly trained wood anatomists. Previous work by Ravindran and colleagues presented macroscopic CVWID models for identification of North American diffuse porous hardwoods from 22 wood anatomically informed classes using the open-source XyloTron platform. This manuscript expands on that work by training and evaluating complementary 17-class XyloTron CVWID models for the identification of North American ring porous hardwoods — woods that display spatial heterogeneity in earlywood and latewood pore size and distribution and other radial growth-rate-related features. Deep-learning models trained using 4045 images from 452 ring-porous wood specimens from four xylaria demonstrated 98% five-fold cross-validation accuracy. A field model trained on all the training data and subsequently tested on 198 specimens drawn from two additional xylaria achieved top-1 and top-2 predictions of 91.4% and 100%, respectively, and images devoid of earlywood, latewood, or broad rays did not greatly reduce the prediction accuracy. This study advocates for continued cooperation between wood anatomy and machine-learning experts for implementing and evaluating field-operational CVWID systems.

Deep Learning for Vulnerability and Attack Detection on Web Applications: A Systematic Literature Review

Web applications are the best Internet-based solution to provide online web services, but they also bring serious security challenges. Thus, enhancing web applications security against hacking attempts is of paramount importance. Traditional Web Application Firewalls based on manual rules and traditional Machine Learning need a lot of domain expertise and human intervention and have limited detection results faced with the increasing number of unknown web attacks. To this end, more research work has recently been devoted to employing Deep Learning (DL) approaches for web attacks detection. We performed a Systematic Literature Review (SLR) and quality analysis of 63 Primary Studies (PS) on DL-based web applications security published between 2010 and September 2021. We investigated the PS from different perspectives and synthesized the results of the analyses. To the best of our knowledge, this study is the first of its kind on SLR in this field. The key findings of our study include the following. (i) It is fundamental to generate standard real-world web attacks datasets to encourage effective contribution in this field and to reduce the gap between research and industry. (ii) It is interesting to explore some advanced DL models, such as Generative Adversarial Networks and variants of Encoders–Decoders, in the context of web attacks detection as they have been successful in similar domains such as networks intrusion detection. (iii) It is fundamental to bridge expertise in web applications security and expertise in Machine Learning to build theoretical Machine Learning models tailored for web attacks detection. (iv) It is important to create a corpus for web attacks detection in order to take full advantage of text mining in DL-based web attacks detection models construction. (v) It is essential to define a common framework for developing and comparing DL-based web attacks detection models. This SLR is intended to improve research work in the domain of DL-based web attacks detection, as it covers a significant number of research papers and identifies the key points that need to be addressed in this research field. Such a contribution is helpful as it allows researchers to compare existing approaches and to exploit the proposed future work opportunities.

Leveraging deep contrastive learning for semantic interaction.

The semantic interaction process seeks to elicit a user’s mental model as they interact with and query visualizations during a sense-making activity. Semantic interaction enables the development of computational models that capture user intent and anticipate user actions. Deep learning is proving to be highly effective for learning complex functions and is, therefore, a compelling tool for encoding a user’s mental model. In this paper, we show that deep contrastive learning significantly enhances semantic interaction in visual analytics systems. Our approach does so by allowing users to explore alternative arrangements of their data while simultaneously training a parametric algorithm to learn their evolving mental model. As an example of the efficacy of our approach, we deployed our model in Z-Explorer, a visual analytics extension to the widely used Zotero document management system. The user study demonstrates that this flexible approach effectively captures users’ mental data models without explicit hyperparameter tuning or even requiring prior machine learning expertise.

Automated Machine Learning (AutoML): an overview of opportunities for application and research

ABSTRACT The wider adoption of AI and machine learning (ML) applications has been limited by the high costs of infrastructure and scarcity of ML experts and data scientists. To address some of these concerns, automated ML (AutoML) systems have been developed alongside cloud computing platforms to mitigate some of the constraints in the wider adoption of ML technologies, including by small and medium size organizations. In this paper, we introduce AutoML, identify some of the fundamental steps in model development, and currently available operationalizations of these systems, before concluding with an outline of potential research opportunities for IS researchers in the field.

Enabling low-cost automatic water leakage detection: a semi-supervised, autoML-based approach

ABSTRACT An important aspect of proper management of water resources is the reduction of losses in urban water distribution. Water loss is especially challenging in developing countries such as Brazil. The real-time monitoring of the distribution system followed by the application of outlier detection techniques on water flow data has been an effective strategy to reduce loss. However, these solutions require high investments in specialized personnel for building the models and data collection for machine learning. This work presents a semi-supervised application of outlier detection techniques and Automated Machine Learning (AutoML) resources on water flow data from District Metering Areas (DMAs). The system does not require experts for model configuration nor curated data for training. The system aims at reducing implementation and deployment costs related to (i) hiring machine learning experts for model configuration and (ii) curation of data for model training, enabling a low-investment deployment suitable for low-income regions.