Towards understanding the challenges faced by machine learning software developers and enabling automated solutions

Plants meet machines: Prospects in machine learning for plant biology

Jupyter notebooks have become central in data science, integrating code, text and output in a flexible environment. With the rise of machine learning (ML), notebooks are increasingly used for prototyping and data analysis. However, due to their dependence on complex ML libraries and the flexible notebook semantics that allow cells to be run in any order, notebooks are susceptible to software bugs that may lead to program crashes. This paper presents a comprehensive empirical study focusing on crashes in publicly available Python ML notebooks. We collect 64,031 notebooks containing 92,542 crashes from GitHub and Kaggle, and manually analyze a sample of 746 crashes across various aspects, including crash types and root causes. Our analysis identifies unique ML-specific crash types, such as tensor shape mismatches and dataset value errors that violate API constraints. Additionally, we highlight unique root causes tied to notebook semantics, including out-of-order execution and residual errors from previous cells, which have been largely overlooked in prior research. Furthermore, we identify the most error-prone ML libraries, and analyze crash distribution across ML pipeline stages. We find that over 40% of crashes stem from API misuse and notebook-specific issues. Crashes frequently occur when using ML libraries like TensorFlow/Keras and Torch. Additionally, over 70% of the crashes occur during data preparation, model training, and evaluation or prediction stages of the ML pipeline, while data visualization errors tend to be unique to ML notebooks.

Comprehensive Study for Breast Cancer Using Deep Learning and Traditional Machine Learning

Artificial intelligence in interdisciplinary life science and drug discovery research.

Enabled by a rich ecosystem of Machine Learning (ML) libraries, programming using learned models , i.e., Software-2.0 , has gained substantial adoption. However, we do not know what challenges developers encounter when they use ML libraries. With this knowledge gap, researchers miss opportunities to contribute to new research directions, tool builders do not invest resources where automation is most needed, library designers cannot make informed decisions when releasing ML library versions, and developers fail to use common practices when using ML libraries. We present the first large-scale quantitative and qualitative empirical study to shed light on how developers in Software-2.0 use ML libraries, and how this evolution affects their code. Particularly, using static analysis we perform a longitudinal study of 3,340 top-rated open-source projects with 46,110 contributors. To further understand the challenges of ML library evolution, we survey 109 developers who introduce and evolve ML libraries. Using this rich dataset we reveal several novel findings. Among others, we found an increasing trend of using ML libraries: The ratio of new Python projects that use ML libraries increased from 2% in 2013 to 50% in 2018. We identify several usage patterns including the following: (i) 36% of the projects use multiple ML libraries to implement various stages of the ML workflows, (ii) developers update ML libraries more often than the traditional libraries , (iii) strict upgrades are the most popular for ML libraries among other update kinds, (iv) ML library updates often result in cascading library updates, and (v) ML libraries are often downgraded (22.04% of cases). We also observed unique challenges when evolving and maintaining Software-2.0 such as (i) binary incompatibility of trained ML models and (ii) benchmarking ML models. Finally, we present actionable implications of our findings for researchers, tool builders, developers, educators, library vendors, and hardware vendors.

Developers rely on third-party library Application Programming Interfaces (APIs) when developing software. However, libraries typically come with assumptions and API usage constraints, whose violation results in API misuse. API misuses may result in crashes or incorrect behavior. Even though API misuse is a well-studied area, a recent study of API misuse of deep learning libraries showed that the nature of these misuses and their symptoms are different from misuses of traditional libraries, and as a result highlighted potential shortcomings of current misuse detection tools. We speculate that these observations may not be limited to deep learning API misuses but may stem from the data-centric nature of these APIs. Data-centric libraries often deal with diverse data structures, intricate processing workflows, and a multitude of parameters, which can make them inherently more challenging to use correctly. Therefore, understanding the potential misuses of these libraries is important to avoid unexpected application behavior. To this end, this paper contributes an empirical study of API misuses of five data-centric libraries that cover areas such as data processing, numerical computation, machine learning, and visualization. We identify misuses of these libraries by analyzing data from both Stack Overflow and GitHub. Our results show that many of the characteristics of API misuses observed for deep learning libraries extend to misuses of the data-centric library APIs we study. We also find that developers tend to misuse APIs from data-centric libraries, regardless of whether the API directive appears in the documentation. Overall, our work exposes the challenges of API misuse in data-centric libraries, rather than only focusing on deep learning libraries. Our collected misuses and their characterization lay groundwork for future research to help reduce misuses of these libraries.

Citation (2023), "Index", Sudhir, K. and Toubia, O. (Ed.) Artificial Intelligence in Marketing (Review of Marketing Research, Vol. 20), Emerald Publishing Limited, Bingley, pp. 309-318. https://doi.org/10.1108/S1548-643520230000020016 Publisher: Emerald Publishing Limited Copyright © 2023 K. Sudhir and Olivier Toubia. Published under exclusive licence by Emerald Publishing Limited INDEX Activation functions, 246 Advertising, 89–90 context of, 96 Agent-based simulation model, 183 Airbnb context, 117 Smart Pricing algorithm, 231 smart pricing tool, 107 Airlines, 105, 107 Alexa, 289 Algorithmic bias, 117 Algorithmic collusion, 33, 118–119 Algorithmic sellers, 109–110 Amazon Mechanical Turk, 163 Amazon’s current business model, 32–33 Anthropomorphism, 185, 274–275 in AI, 277–286 beneficial and harmful effects, 287–290 conceptual framework, 286–298 conditions, 290–293 cumulative distribution of articles, 275 future research directions, 298–302 individual characteristics of AI users, 293–296 insights emerging from literature, 284–286 journals included in literature search, 279 limitations, 302 literature review procedures, 278, 280, 283–284 related to context of employing AI anthropomorphism, 299–300 related to effects of AI anthropomorphism, 298–299 related to individual characteristics of AI users, 300–302 relationship perspective, 297–298 Apple (technology company), 13–14 Application programming interfaces (APIs), 159–160 Area Under the Curve (AUC), 87–88 Artificial intelligence (AI), 1–2, 13–14, 104–105, 125–126, 147–148, 170, 218, 274 advertising, persuasion, and communication, 153 agenda for future work, 34 AI-based algorithm, 29 AI-based innovation, 1–2 AI-based model selection tools, 28–29 AI-based queries, 154 AI-based solutions, 133 AI-supported content generation, 139–140 aiding marketing decisions, 4–6 algorithmic collusion, 118–119 anthropomorphism in, 277–286 applications of AI-powered VOC, 150–153 challenges in use of UGC, 149 consequences for pricing, 115–119 considerations for use of, 139–140 consumer reactions, 139 data available for AI and VOC, 154–156 decisions, 30–32 dynamic pricing, 115–116 economic framework of, 14–27 firms implementing AI for pricing, 104–115 identifying and organizing customer needs, 150 impact on consumers and society, 8–9 level of impact of, 27–28, 34 market research, 6 marketing purpose of, 4–6 opportunity identification for AI research, 10 personalized pricing, 117 potential abuse and need for regulation, 139–140 prediction, 28–30 price algorithms, 111 promise of, 149–150 promise of AI and Machine Learning, 149–150 promise of user-generated content, 149 reflection of branding through users, 151–153 research in marketing on, 16, 26, 40, 76 society, 33–34 strategy, 32–33 tools, 32, 136, 140 understanding and forecasting demand, 150–151 and VOC, 148–150 VOC practice before, 148–149 workforce implications, 140 Artificial Intelligence Assistants (AIAs), 289 Artificial neural networks (ANNs), 227, 240–241 Attenuation bias, 170, 175 Augmented reality (AR), 7, 228–229 Autocompletion for email and text messaging, 139 Autoencoders, 257 Autoencoding models, 203 Automated content generation, 129 Automation, 104–105 Autoregressive models, 203 Average treatment effect (ATE), 84 Azure’s Face API, 222–223 “Bag-of-words”–based methods, 180 Behavioral experiments, 229–230 Berry–Levinsohn–Pakes–type random coefficient choice model, 178 Bias mitigation, 230–231 Bibliometric network, 173 Bidirectional Encoder Representations from Transformers (BERT), 125–126, 157, 159, 180–181, 198–199, 202–203 Big data (see also Data), 104 VOC practice before, 148–149 Big GAN (BigGAN), 132 Binary Robust Independent Elementary Features, 221 Brand logos, 220 Brand perception, 162 Brand selfies concept, 153 Brand-related social tags, 30 Branding, 148 brand perception, 151–152 brand positioning, 152–153 reflection of branding through users, 151–153 user–brand interaction, 152–153 Brands marketing strategies, 154 Business leaders, 13–14 Business-to-consumer (B2C), 275 Canny edge detector, 221 Causal inference, 268 Causality, 194–195 Charge supracompetitive prices, 118 Classical ML models, 242 Click-through rate (CTR), 89–90 Clustering, 158 algorithms, 150 Co-citation analysis, 173 Collusive algorithm, 118–119 Color histogram descriptor, 221 Colors, 221 Common method bias, 170 Company-level topics, 223–224 Computer vision, 7–8 application domain, 223–224 data format, 219–221 future research, 228–231 in marketing research, 219–224 model structure, 221–223 techniques, 218 Conditional average treatment effect (CATE), 82 Conditional GAN (CGAN), 131–132 Conjoint analysis, 149 Construct validity, 164 Consumer reactions, 139 Consumer silence, 170–171 Consumer-centric perspective, 276 Consumer-level topics, 223 Content generation considerations for use of AI-supported content generation, 139–140 generating synthetic images, 131–133 generating textual content with language models, 129–131 potential for, 129–133 potential for AI throughout customer journey, 126–129 potential for content generation, 129–133 supporting customer equity management with content generation, 133–139 Content selection method, 130 Content-related marketing tasks, 140 Contextual bandit, 82–83 Contour, 221 Convolutional neural networks (CNNs), 28, 129, 157, 163, 202, 220–221, 246, 249, 254 Convolutional-LSTM, 157 Counterfactual explanations, 227–228 Counterfactual policy evaluation, 84–85 Counterfactual validity, 81–82 Criticisms, 227–228 Cross-entropy, 247 CTR prediction problems, 81 Customer acquisition, 134–136 Customer equity framework, 126–127 Customer equity management with content generation customer acquisition, 134–136 customer retention, 138–139 relationship development, 136–138 supporting, 133–139 Customer feedback, 170 future of customer feedback research, 183–185 online customer feedback, 175–183 publication count by journal, 172 publication count by year, 172 review methodology, 171–174 from user-generated content, 176 Customer relationship management (CRM), 8 Customer retention, 138–139 Customer reviews, 154 Customer satisfaction research, 174–175 DALL-E, 137 Data, 14, 154 available for AI AND VOC, 154–156 customer reviews, 154 data-generating process, 87–88 direct queries to customers, 154 images, 155 preprocessing, 156–157 social media, 154 sources, 154 text, 155 trading, 1 transformation, 160 types, 155–156 user engagement, 155–156 De-bias pricing algorithms, 117 Decision trees, 82 Decision types, 15–27 Decision-makers in marketing, 218 Deep learning (DL), 8, 82, 192, 230, 239–240 algorithms, 248, 250, 265 architectures for NLP, 200–202 causal inference, 268 combine unstructured data with structured data, 265–266 common testbeds, 268–269 customized algorithm development, 265–266 customized constraint, 266 deep learning–based language model, 174 future directions, 266–269 in marketing, 239–240, 243–244 model efficiency improvement, 267 models, 126, 220–221, 224 multimodal, five senses, and networks, 267 neural networks, 242–248 plug and play, 265 problems, 265–266 properties, 241–242 reinforcement learning, 267–268 theory-driven architecture design, 266 theory-driven initialization, 266 Deep neural networks (DNN), 81, 242 Deep Q-Network (DQN), 248–249 Deep reinforcement learning (DRL), 262–265 Deepfakes, 139–140 Demand, real-time swings in, 105–110 Diachronic word embeddings, 199–200 Dictionary and word frequency–based text mining, 179–180 Difference-in-difference estimation approach, 194–195 Digital cameras, 150 Digital exhaust of individual behavior, 1 Digital footprints, 147–148 Digital voice assistants, 139 Direct marketing context, 138 Direct method, 85 Direct queries to customers, 154 Discriminative deep learning models, 249–258 CNNs, 249–254 RNN, 254–255 transformers, 255–258 Discriminative models, 248–249 Discriminator network, 131 Disney, 88 Distributional hypothesis, 199 doc2vec, 200 Dominant color descriptor, 221 Double machine learning (DML), 178 Doubly Robust estimator (DR estimator), 86 Doubly robust method, 86 Dropout method, 248 Dynamic methods, 83–84 Dynamic models update customers, 138 Dynamic pricing, 105, 110, 115–116 E-commerce, 104 Emails, 132–133 Embedded Topic Model (ETM), 204 Embeddings, 157–158, 198–200, 204 Entropy, 87–88 Equilibrium analysis, strategic behavior and, 95–96 ERNIE 3.0, 203–204 European Union (EU), 33 Evaluation of AI methods, 161–162 Evaluative Lexicon 2.0, 197 Evidence lower bound (ELBO), 259 Example-based explanation techniques, 227–228 eXplainable Artificial Intelligence (XAI), 219 External validity, 164 Facebook (technology company), 13–14 engagement data, 156 news feed algorithm, 93 user-engagement data, 155–156 Fairness in marketing, 224 Fake reviews, 183 fastText, 198–199 Feature-level models, 221 Feedback data, 14 Field experimentation, 84 Field experiments, 229–230 Financial Times Top 50 journals (FT50 journals), 278–279, 284 Fine-tuning, 130 Firms, 3–4, 131 First-order methods, 247 Flexible supervised learning algorithms, 81 Frames, 219 Fuzzy SVM, 159 Gated recurrent unit (GRU), 255 GauGAN, 137 Gender differences, 294 General Data Protection Regulation (GDPR), 33 Generalizability, 81–82 Generative adversarial networks (GAN), 8, 131, 248–249, 260, 262 Generative deep learning models, 258–262 GAN, 260–262 VAE, 258–260 Generative models, 140, 248–249 Generative Pre-trained Transformer 3 (GPT-3), 125–126, 180–181, 198–199 Generative video models, 139 Generator network, 131 Global interpretability, 225 GLOVE, 180, 198–199 Google (technology company), 13–14 search engine algorithm, 134 search personalization, 93 Gradient-weighted class activation mapping (Grad-CAM), 161, 226–227 Heatmap method, 228 Hidden Markov Model (HMM), 165 Hulu, 88 Human–machine collaboration, 230 ImageNet, 268 Images, 155, 218 data, 220 image-based social media, 218 image/post clusters, 153 tagging, 157 Incentive-aware personalization, 96 Individual-level personalization, 77–78 InferNER approach, 195 Influence methods, 227 Input data, 14 Insight generation, 170–171 Instagram, 171 Instrumental variable approach (IV approach), 175 Interactive methods, 82–83 Internet, 151 Interpretability, 219, 228 issues, 224–228 Inverse Propensity Score estimator (IPS estimator), 85–86 ISI Web of Science, 171–172 Judgment, 14–15 Knowledge extraction, 227 LambdaMART ranking algorithm, 29 Language models, 131, 198, 204 generating textual content with, 129–131 marketing applications of, 204–205 Language structure and deep learning–based text mining, 180 Large language models, 140 Large-scale pretrained language models, 129–130 LDA, 195 Learning from audio visual data, 184 from interactive two-sided feedback, 185 Lexicons, 197–198 and word frequency–based methods, 180 Linguistic Inquiry and Word Count (LIWC), 197 Local interpretability, 226 Local Interpretable Model-Agnostic Explanations (LIME), 226 Long short-term memory (LSTM), 129, 157, 201–202, 255 Low response rates, 170 Lyft, 107 Machine learning (ML) (see also Deep learning (DL)), 14–15, 29–30, 32, 147–148, 170, 219, 240 algorithms, 150 methods, 82 promise of AI and, 149–150 Manual encoding, 219–220 Manual inspection, 160 Mapping methods to research questions, 162–165 posteriori–identified phenomena and constructs, 162–163 priori–defined constructs, 163–164 validation, 164–165 Market fairness, 224 research, 6 Marketers, 1–2 Marketing, 274 AI’s impact on consumers and society and vice versa, 8–9 algorithms and methods, 7–8 applications of language models, 204–206 communications, 132–133 data, 6–7 marketing-AI ecosystem, 2–4 modelers, 242–246 novel approaches for established tasks, 204 novel approaches for novel tasks, 204–205 opportunity identification for AI research, 10 purpose of AI, 4–6 research in marketing on Artificial Intelligence, 40–76 scholars, 1–2, 14, 28–30, 34, 193 Markov decision process (MDP), 263 Matrix factorization approaches, 81 Maximum likelihood estimation (MLE), 247 Mean Average Error (MAE), 87–88 Measurement error, 180 Megatron-Turing NLG, 125–126 Menu costs, 103 Metaphor elicitation technique, 151–152 Methodological approaches to personalization, 79–84 dynamic methods, 83–84 generalizability and counterfactual validity, 81–82 online and interactive methods, 82–83 scalability, 80–81 Metric-based evaluation, 87–88 Mind perception, 299 mini-Xception, 222–223 Model interpretability, 219 Model interpretation, 160–161 with manual inspection and data transformation, 160 post hoc model explanation, 161 Model-agnostic interpretability, 227 Model-agnostic techniques, 226 Model-specific interpretability, 226–227 Multi-armed bandit (MAB), 113 Multiarmed bandit models (MAB models), 5 Multihead attention, 202 Naive Bayes classifier, 178 Named entity extraction (NER), 195 Natural language generation models (NLG models), 125–126, 129–130 Natural language inference task (NLI task), 205 Natural language processing (NLP), 3, 7, 150, 172–173, 192 applications, 192 challenges, biases, and potential harms, 208–209 concept and topic extraction, 195–197 current state of NLP in marketing, 195–198 embeddings, language models, transfer learning, 198–204 established and novel tools for diverse text-based marketing applications, 196 marketing applications of language models, 204–205 relationship extraction, 197 roadmap and future trends, 206–207 sentiment and writing style extraction, 197–198 text in marketing, 193–195 Netflix, 88 Network embeddings, 163 Neural networks (NN), 242, 248 activation functions, 246 architecture, 246 objective function, 247 optimizer, 247–248 regularization, 248 News personalization, 83 Nonconvergence, 262 Nonparametric approach, 115 Nontech firms, 267 Nontextual data, 194 Objective function, 247 Offline beacons, 1 Online customer feedback, 175–183 AI and machine learning in analyzing unstructured review data, 178–181 challenges in learning from, 181–183 economic impact of online reviews, 177–178 Online forum discussions, 30 Online methods, 82–83 Online platforms, 147–148 Online reviews, 30 OpenCV, 222–223 Optimal algorithm, 115 Optimizer, 247–248 Overlap assumption, 82 Peer influence, 182–183 Personality, 295 Personalization algorithms, 82–83, 91 alternative approaches, 87–88 direct method, 85 doubly robust method, 86 evaluation, 84–88 extensions to special settings, 86–87 IPS estimator, 85–86 methodological approaches to personalization, 79–84 models, 94–95 multiple objectives and long-term outcomes, 94–95 problem definition, 78–79 returns to personalization, 88–90 signal-to-noise ratio, 94 strategic behavior and equilibrium analysis, 95–96 time drifts, 95 and welfare, 90–93 Personalized policy design, 78–79 Personalized pricing, 89–90, 110, 113, 117 Personification, 278 Photorealistic images, 131 Pix2pix approach, 132 Pixel-level models, 222 Plug and play language models (PPLM), 130 Poisson factorization, 197 Polarization, 93 Position encoding, 255, 257 Post hoc model explanation, 161 Posteriori–identified phenomena and constructs, 162–163 Prediction Machines , 14, 32–33 Predictions, 4, 28, 30 prediction-based algorithms, 158–159 process, 14 Predictive ML algorithms, 163 Preprocessing images, 157 Price discrimination, 110–113 Price experimentation, 113–115 Pricing automation, 105–107 consequences of AI for pricing, 115–119 dynamic pricing, 105–110 firms implementing AI for pricing, 104–115 personalized pricing, 110–113 price experimentation, 113–115 Primary data, 149 Prime Video, 88 Principal component analysis (PCA), 258 Priori–defined constructs, 163–164 Privacy, personalization and welfare, 91–92 Probabilistic content generation process, 130 Product development, 150 Propensity-based approaches, 87 Prospective customers, 126 Prototypes, 227–228 Q-learning algorithm, 115 models, 118 Q-value function approximator, 264–265 Quantitative marketers, 1 Racist language, 130 Random Forests, 81–82 Recency Frequency Monetary value (RFM value), 115 Rectified linear units (ReLu), 246 Recurrent neural networks (RNNs), 129, 159, 200–201, 248–249, 254–255 Recursive neural networks, 129 Regression models, 225 Regularization, 248 Regulators, 4 Reinforcement learning (RL), 248–249, 267–268 Relationship development, 136–138 Relationship extraction, 197 Relative Information Gain (RIG), 87–88 Representation learning, 240–241 Reputation platforms, 171 systems, 175 ResNet-50, 222 Restricted Boltzmann machine (RBM), 242 Ride-hailing platforms, 107 RoBERTa, 157, 198–199 Robots, 289–290 Rule-based learners, 226–227 Scalability, methodological approaches to personalization, 80–81 Scale-Invariant Feature Transform (SIFT), 221 SCImago Journal & Country Rank, 278–279 SE-ResNet-50, 222 Search engine optimization (SEO), 30, 133, 205 Second-order methods, 247 Seeded LDA, 195–196 Selection bias, 183 Self-attention, 255 Self-selection, 182 Self-supervised representation learning, 200 Semantic network analysis, 179–180 Sentence-based LDA, 195–196 SentenceBERT, 200 Sentiment analysis, 151, 198 Sentiment and writing style extraction, 197–198 Sequence-to-sequence models, 203 SHapley Additive exPlanations (SHAP), 161, 226 algorithm, 226 values, 163 “Shipping then shopping” strategy, 32–33 “Shop, then ship” model, 4 Short-term rental market, 107 Signal-to-noise ratio, 94 Small-and medium-sized enterprises (SMEs), 267 “Smart pricing” tool, 117 Social media, 147–148, 154, 218 messages, 135–136 messaging, 135 posts, 132–133 Social Sciences Citation Index (SSCI), 171–172 Speeded-Up Robust Features, 221 Standard reinforcement learning algorithm, 118 Stanford Named Entity recognizer, 195 “Stick-and carrot” strategies, 118 Stochastic gradient descent (SGD), 247–248 Stochastic parrots, 140 Stroop test performance, 293 Structural models, 225 Style-based GAN (StyleGAN), 131–132 Subnetworks, 131 Subscription-based “shipping-then-shopping” business model, 32–33 Supervised learning algorithms, 81 Supervised ML models, 151 Supply, real-time swings in, 105–110 Support vector machines (SVM), 29, 159, 178 Surge pricing algorithms, 107 Survey-based perceptual maps, 152 Synthetic images, generating, 131–133 Technology, 274 companies, 13–14 Text data, 7, 155 Text in marketing, 193–195 causality, 194–195 dependent variable, 194 dual role of language, 193 independent variables, 194 Text mining, 192 algorithms, 240 Textual analysis in marketing, 192–193 Textual consumer feedback, 179–181 Textual content with language models, generating, 129–131 Textures, 221 3D convolutional neural network, 220–221 TikTok, 218 Time drifts, 95 Topic modeling, 158, 192 Traditional LDA approach, 195–196 Training data, 14 Training process, 131 Transaction data, 149 Transfer learning, 198, 202, 204, 222 Transform data, 157 Transformer-based models, 157, 202, 204 Transformers, 202, 255, 258 Twitter, 171 Uber, 107 Unconditional counterfactual explanations, 227–228 Unconfoundedness assumption, 82 Underspecification, 209 Uniform policy, 79 Unstructured data, 170, 192, 218 Unsupervised learning, 157–158 clustering, 158 embeddings, 157–158 topic modeling, 158 Upper confidence bound algorithm (UCB algorithm), 115 US Congress, 116 User clusters, 153 User engagement, 155–156 User-generated content (UGC), 30, 147–149, 170–171 challenges in use of, 149 customer feedback from, 176 data preprocessing, 156–157 evaluation, 161–162 hybrid of unsupervised and supervised learning, 159–160 model interpretation, 160–161 prediction-based algorithms, 158–159 promise of, 149 tools and methods to understand, 156–162 unsupervised learning, 157–158 User-generated text, 156 User–brand interaction, 152–153 VADER, 197 Validation, 164–165 Value functions, 263 Variational autoencoders (VAE), 8, 160, 248–249, 258, 260 Vector semantics, 199–200 VGG-16 algorithm, 159 Video analytics, 7 Video content, 137 Video data, 220 Video platforms, 218 Virtual reality (VR), 7, 228–229 Visual consumer feedback, 181 Visual content, 137 Visual data, 7 Visualization techniques, 227 Voice of the Customer (VOC), 6, 147–148, 150 data available for AI AND, 154–156 importance of, 148 practice before artificial intelligence and big data, 148–149 Volume, velocity, variety (3Vs), 3 VOSviewer software, 173 Welfare fairness, 92–93 personalization and, 90–93 polarization, 93 privacy, 91–92 search cost, 91 White House’s Council of Economic Advisors (White House’s CEA), 117 Word embeddings, 198–199 Word-of-mouth (WOM), 172–173 Word2Vec (language embedding algorithm), 157, 180, 198–200 XAI methods, 224–228 model specificity, 226–228 model transparency, 224–225 scope of explanation, 225–226 XGBoost, 81, 159 Yelp, 171 YouTube, 88, 93, 218 ZIP codes, 111 Book Chapters Prelims The State of AI Research in Marketing: Active, Fertile, and Ready for Explosive Growth The Economics of Artificial Intelligence: A Marketing Perspective AI and Personalization Artificial Intelligence and Pricing Leveraging AI for Content Generation: A Customer Equity Perspective Artificial Intelligence and User-Generated Data Are Transforming How Firms Come to Understand Customer Needs Artificial Intelligence Applications to Customer Feedback Research: A Review Natural Language Processing in Marketing Marketing Through the Machine's Eyes: Image Analytics and Interpretability Deep Learning in Marketing: A Review and Research Agenda Anthropomorphism in Artificial Intelligence: A Review of Empirical Work Across Domains and Insights for Future Research Index

Web Services (WSs) are gaining worldwide popularity due to reliable and fast intercommunication services for the development of web and mobile applications. WSs are provided to client application developers through web Application Programming Interfaces (APIs), such as YouTube API, Twitter API, Facebook API, etc. Due to the popularity of WSs, the developers frequently discuss various WSs-based application’ issues on online forums, such as Stack Overflow (SO). This study aims to highlight the problems faced by client developers in the development process of WSs-based applications using the dataset of SO. The comprehension of developers’ conversations on SO can give insight into the frequency, difficulty, and popularity of different WSs-related problems of developers. We downloaded 12,746 posts from SO relevant to WSs-related issues for this article. We used the topic modeling technique (LDA) to extract various topics from the SO dataset. The topics are labeled and organized into categories and sub-categories according to relationships among them. The difficulty and popularity of each topic have been analyzed. Our investigation yield several findings. First, developers focus on six topics related to WSs on SO: Client APIs development, Data Processing, Web services Authorization, Framework Support, Web APIs, and Mobile Applications. Secondly, the advantages and disadvantages of web applications topic (Fused_Popularity=0.39), from the Clients APIs development category have the highest prevalence, followed by Database (DB) and Data Processing in Applications topic (Fused_Popularity=0.38) from the Data Processing category. Third, most WSs-related topics in all categories are evolving promptly on SO, i.e., new questions are added daily about WSs development, deployment, and authorization. Fourth, the questions of type “how” are primarily asked in Framework support, Client APIs development, and Web APIs categories. Although, many questions in other categories are of the kind “What”. It is also observed that WSs developers not only used SO to ask How and What types of questions but they also used SO to ask information-seeking questions (i.e., in Data processing and Client APIs development categories). Fifth, the topics relevant to Web APIs (Fused_Popularity=10.8) and Client API Development ((Fused_Popularity=9.35) categories of WSs are very popular on SO. Sixth, the questions relevant to the Web APIs (Fused_Difficulty =3) & Client APIs development (Fused_Difficulty=2.25) categories are more difficult than the other four categories. The results of our research may be helpful for the following WSs stakeholders: WSs Client application developers, WSs Educators, and WSs researchers. The WSs Educators and investigators can get more knowledge of new methods and discover novel techniques to make challenging WSs topics easy to understand. WSs framework developers can utilize our extracted WSs topics and categories to know the preferences of WSs developers that may support them in upgrading existing frameworks or developing new ones.

Machine Learning is a sub-category of Artificial Intelligence enabling computers with the ability of pattern recognition, or to continuously learn from, making predictions based on data, and carry out decisions without being specifically programmed for doing so. In this context, Machine Learning is a broader category of algorithms being able to use datasets to identify patterns, discover insights, and enhance understanding and make decisions or predictions. Compared with Machine Learning, Deep Learning is a particular branch of Machine Learning that makes use of Machine Learning functionality, and moves beyond its capabilities. Deep Learning Algorithm is interpreted as a layered structure that tries to replicate the structure of the human brain. These capabilities enable Machine Learning and Deep Learning Algorithms usage in applications to identify and respond to cybercriminals manifold cyberattacks. This is achieved by analyzing Big Datasets of cybersecurity incidents to identify patterns of malicious activities. For this purpose, Machine Learning and Deep Learning compare known threat event attacks with detected threat event attacks to identify similarities they automatically dealt with trained Machine Learning or Deep Learning model for response. Against this background, this chapter seeks to offer a clear explanation of the classification of Machine Learning and Deep Learning and comparing them with regard to effectivity and efficiency in their specific application domains. This requires (i) discussing the methodological background of Machine Learning and Deep Learning; (ii) introducing relevant application areas of Machine Learning and Deep Learning like Intrusion Detection Systems; and (iii) use cases showing how to combat against threat event attacks based cybersecurity risks. In this context, this chapter provides, in Sect. 8.1, a brief introduction in classical Machine Learning, which consists of Supervised, Unsupervised, and Reinforcement Machine Learning. In this regard, Sect. 8.1.1.1 introduces Supervised Machine Learning, while Sect. 8.1.1.2 refers to Unsupervised Machine Learning, and Sect. 8.1.1.3 focuses on Reinforcement Machine Learning. Sect. 8.1.1.4 finally compares the different Machine Learning methods with regard to advantages and disadvantages. Based on this methodological introduction of classical Machine Learning, Sect. 8.2.1 introduces in Machine Learning and cybersecurity issues. Machine Learning-based intrusion detection in industrial application is therefore the topic of Sect. 8.2.1.1. Section 8.2.1.2 introduces Machine Learning-based intrusion detection based on feature learning, and Machine Learning-based intrusion detection of unknown cyberattacks is the topic of Sect. 8.2.1.3. In Section 8.3, the classification of Deep Learning methods is given which contains in Sect. 8.3.1 the topics Feedforward Deep Neural Networks, Convolutional Feedforward Deep Neural Networks, Recurrent Deep Neural Networks, Deep Beliefs Networks, and the Deep Bayesian Neural Network. Based on this methodological background of Deep Learning methods, Sect. 8.3.2 introduces Deep Bayesian Neural Networks, while Sect. 8.3.3 refers to Deep Learning-based intrusion detection. Finally, Sect. 8.4 refers to Deep Learning methods in cybersecurity applications. Section 8.5 contains comprehensive questions from the topics Machine Learning and Deep Learning, followed by “References” with references for further reading.

GenSyth: a new way to understand deep learning

Artificial Intelligence and the Common Sense of Animals.

Recent breakthroughs in Machine Learning (ML) applications, and especially in Deep Learning (DL), have made DL models a key component in almost every modern computing system. The increased popularity of DL applications deployed on a wide-spectrum of platforms (from mobile devices to datacenters) have resulted in a plethora of design challenges related to the constraints introduced by the hardware itself. "What is the latency or energy cost for an inference made by a Deep Neural Network (DNN)?" "Is it possible to predict this latency or energy consumption before a model is even trained?" "If yes, how can machine learners take advantage of these models to design the hardware-optimal DNN for deployment?" From lengthening battery life of mobile devices to reducing the runtime requirements of DL models executing in the cloud, the answers to these questions have drawn significant attention. One cannot optimize what isn't properly modeled. Therefore, it is important to understand the hardware efficiency of DL models during serving for making an inference, before even training the model. This key observation has motivated the use of predictive models to capture the hardware performance or energy efficiency of ML applications. Furthermore, ML practitioners are currently challenged with the task of designing the DNN model, i.e., of tuning the hyper-parameters of the DNN architecture, while optimizing for both accuracy of the DL model and its hardware efficiency. Therefore, state-of-the-art methodologies have proposed hardware-aware hyper-parameter optimization techniques. In this paper, we provide a comprehensive assessment of state-of-the-art work and selected results on the hardware-aware modeling and optimization for ML applications. We also highlight several open questions that are poised to give rise to novel hardware-aware designs in the next few years, as DL applications continue to significantly impact associated hardware systems and platforms.

Using Deep Learning to Predict Fracture Patterns in Crystalline Solids

Recent performance breakthroughs in artificial intelligence (AI) and machine learning (ML), especially advances in deep learning, the availability of powerful and easy-to-use ML libraries (e.g., scikit-learn, TensorFlow, PyTorch), and increasing computational power, have led to unprecedented interest in AI/ML among nuclear engineers. For physics-based computational models, verification, validation, and uncertainty quantification (VVUQ) processes have been very widely investigated, and many methodologies have been developed. However, VVUQ of ML models has been relatively less studied, especially in nuclear engineering. This work focuses on uncertainty quantification (UQ) of ML models as a preliminary step of ML VVUQ, more specifically Deep Neural Networks (DNNs) because they are the most widely used supervised ML algorithm for both regression and classification tasks. This work aims at quantifying the prediction or approximation uncertainties of DNNs when they are used as surrogate models for expensive physical models. Three techniques for UQ of DNNs are compared, namely, Monte Carlo Dropout (MCD), Deep Ensembles (DE), and Bayesian Neural Networks (BNNs). Two nuclear engineering examples are used to benchmark these methods: (1) time-dependent fission gas release data using the Bison code and (2) void fraction simulation based on the Boiling Water Reactor Full-size Fine-Mesh Bundle Tests (BFBT) benchmark using the TRACE code. It is found that the three methods typically require different DNN architectures and hyperparameters to optimize their performance. The UQ results also depend on the amount of training data available and the nature of the data. Overall, all three methods can provide reasonable estimations of the approximation uncertainties. The uncertainties are generally smaller when the mean predictions are close to the test data while the BNN methods usually produce larger uncertainties than MCD and DE.

Optimisation and interpretation of machine and deep learning models for improved water quality management in Lake Loktak