A State-of-the-Art Survey on Deep Learning Theory and Architectures

In the recent era Deep learning is showing exemplary results in many application areas. Researches from different disciplines have incorporated deep learning in to their research to solve different interdisciplinary problems. Deep learning applications areas include Speech recognition, Natural Language Processing, Computer vision, Networking, Healthcare, IoT, Robotics, Agriculture, Remote sensing and many other areas. Thus, this paper contributes a review on various deep learning approaches which include Convolution Neural Network (CNN), Deep Neural Network (DNN), Auto-Encoder (AE), Recurrent Neural Network (RNN) enclosed with Gated Recurrent Units (GRU), Long Short-Term Memory (LSTM) and ConvLSTM, Deep Reinforcement Learning (DRL), Generative based Adversarial Network (GAN) and Deep Belief Network (DBN). For extraction of the most important features of an image various data extraction procedures have designed. CNN's tremendous learning capacity is based on the utilisation of several feature extraction stages that continuously learn from data. Some of the popular CNN architectures name as LeNet, AlexNet, ZFNet / Clarifai, VGGNET, GoogLeNet, ResNet, DenseNet, FractalNet and CapsuleNet. Recent works extended to a combination of two CNN models like Inception, ResNetV2 to attained different existing approaches consequences in deep learning. Identification of the significant objects, their parameters and relationships of an object images are required for image captioning. Syntactically and semantically correct sentences need to be generated. The intricacies and problems of picture captioning may be handled using deep learning approaches. A comparison between these models was also presented. We also discussed about different standard datasets which are utilized for executing and estimating the deep learning approaches.

In recent years, machine learning and Deep Learning have increased and gathered epic success in traditional application domains and new areas of Artificial Intelligence. The performance using Deep Learning has dominated experimental results compared to conventional machine learning algorithms. This paper presents an overview of the progress that has occurred in Deep Learning (DL) concerning some application domains like Autonomous Driving, Healthcare, Voice Recognition, Image Recognition, Advertising, Predicting Natural Calamities, National Stock Exchange and many more. Additionally, deeper insights into several Deep Learning techniques, their working principles, and experimental results are scrutinized. The survey covers Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), including Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU), Auto-Encoder (AE), Deep Belief Network (DBN), Generative Adversarial Network (GAN), and Deep Reinforcement Learning (DRL).

Citation (2023), "Index", Sudhir, K. and Toubia, O. (Ed.) Artificial Intelligence in Marketing (Review of Marketing Research, Vol. 20), Emerald Publishing Limited, Bingley, pp. 309-318. https://doi.org/10.1108/S1548-643520230000020016 Publisher: Emerald Publishing Limited Copyright © 2023 K. Sudhir and Olivier Toubia. Published under exclusive licence by Emerald Publishing Limited INDEX Activation functions, 246 Advertising, 89–90 context of, 96 Agent-based simulation model, 183 Airbnb context, 117 Smart Pricing algorithm, 231 smart pricing tool, 107 Airlines, 105, 107 Alexa, 289 Algorithmic bias, 117 Algorithmic collusion, 33, 118–119 Algorithmic sellers, 109–110 Amazon Mechanical Turk, 163 Amazon’s current business model, 32–33 Anthropomorphism, 185, 274–275 in AI, 277–286 beneficial and harmful effects, 287–290 conceptual framework, 286–298 conditions, 290–293 cumulative distribution of articles, 275 future research directions, 298–302 individual characteristics of AI users, 293–296 insights emerging from literature, 284–286 journals included in literature search, 279 limitations, 302 literature review procedures, 278, 280, 283–284 related to context of employing AI anthropomorphism, 299–300 related to effects of AI anthropomorphism, 298–299 related to individual characteristics of AI users, 300–302 relationship perspective, 297–298 Apple (technology company), 13–14 Application programming interfaces (APIs), 159–160 Area Under the Curve (AUC), 87–88 Artificial intelligence (AI), 1–2, 13–14, 104–105, 125–126, 147–148, 170, 218, 274 advertising, persuasion, and communication, 153 agenda for future work, 34 AI-based algorithm, 29 AI-based innovation, 1–2 AI-based model selection tools, 28–29 AI-based queries, 154 AI-based solutions, 133 AI-supported content generation, 139–140 aiding marketing decisions, 4–6 algorithmic collusion, 118–119 anthropomorphism in, 277–286 applications of AI-powered VOC, 150–153 challenges in use of UGC, 149 consequences for pricing, 115–119 considerations for use of, 139–140 consumer reactions, 139 data available for AI and VOC, 154–156 decisions, 30–32 dynamic pricing, 115–116 economic framework of, 14–27 firms implementing AI for pricing, 104–115 identifying and organizing customer needs, 150 impact on consumers and society, 8–9 level of impact of, 27–28, 34 market research, 6 marketing purpose of, 4–6 opportunity identification for AI research, 10 personalized pricing, 117 potential abuse and need for regulation, 139–140 prediction, 28–30 price algorithms, 111 promise of, 149–150 promise of AI and Machine Learning, 149–150 promise of user-generated content, 149 reflection of branding through users, 151–153 research in marketing on, 16, 26, 40, 76 society, 33–34 strategy, 32–33 tools, 32, 136, 140 understanding and forecasting demand, 150–151 and VOC, 148–150 VOC practice before, 148–149 workforce implications, 140 Artificial Intelligence Assistants (AIAs), 289 Artificial neural networks (ANNs), 227, 240–241 Attenuation bias, 170, 175 Augmented reality (AR), 7, 228–229 Autocompletion for email and text messaging, 139 Autoencoders, 257 Autoencoding models, 203 Automated content generation, 129 Automation, 104–105 Autoregressive models, 203 Average treatment effect (ATE), 84 Azure’s Face API, 222–223 “Bag-of-words”–based methods, 180 Behavioral experiments, 229–230 Berry–Levinsohn–Pakes–type random coefficient choice model, 178 Bias mitigation, 230–231 Bibliometric network, 173 Bidirectional Encoder Representations from Transformers (BERT), 125–126, 157, 159, 180–181, 198–199, 202–203 Big data (see also Data), 104 VOC practice before, 148–149 Big GAN (BigGAN), 132 Binary Robust Independent Elementary Features, 221 Brand logos, 220 Brand perception, 162 Brand selfies concept, 153 Brand-related social tags, 30 Branding, 148 brand perception, 151–152 brand positioning, 152–153 reflection of branding through users, 151–153 user–brand interaction, 152–153 Brands marketing strategies, 154 Business leaders, 13–14 Business-to-consumer (B2C), 275 Canny edge detector, 221 Causal inference, 268 Causality, 194–195 Charge supracompetitive prices, 118 Classical ML models, 242 Click-through rate (CTR), 89–90 Clustering, 158 algorithms, 150 Co-citation analysis, 173 Collusive algorithm, 118–119 Color histogram descriptor, 221 Colors, 221 Common method bias, 170 Company-level topics, 223–224 Computer vision, 7–8 application domain, 223–224 data format, 219–221 future research, 228–231 in marketing research, 219–224 model structure, 221–223 techniques, 218 Conditional average treatment effect (CATE), 82 Conditional GAN (CGAN), 131–132 Conjoint analysis, 149 Construct validity, 164 Consumer reactions, 139 Consumer silence, 170–171 Consumer-centric perspective, 276 Consumer-level topics, 223 Content generation considerations for use of AI-supported content generation, 139–140 generating synthetic images, 131–133 generating textual content with language models, 129–131 potential for, 129–133 potential for AI throughout customer journey, 126–129 potential for content generation, 129–133 supporting customer equity management with content generation, 133–139 Content selection method, 130 Content-related marketing tasks, 140 Contextual bandit, 82–83 Contour, 221 Convolutional neural networks (CNNs), 28, 129, 157, 163, 202, 220–221, 246, 249, 254 Convolutional-LSTM, 157 Counterfactual explanations, 227–228 Counterfactual policy evaluation, 84–85 Counterfactual validity, 81–82 Criticisms, 227–228 Cross-entropy, 247 CTR prediction problems, 81 Customer acquisition, 134–136 Customer equity framework, 126–127 Customer equity management with content generation customer acquisition, 134–136 customer retention, 138–139 relationship development, 136–138 supporting, 133–139 Customer feedback, 170 future of customer feedback research, 183–185 online customer feedback, 175–183 publication count by journal, 172 publication count by year, 172 review methodology, 171–174 from user-generated content, 176 Customer relationship management (CRM), 8 Customer retention, 138–139 Customer reviews, 154 Customer satisfaction research, 174–175 DALL-E, 137 Data, 14, 154 available for AI AND VOC, 154–156 customer reviews, 154 data-generating process, 87–88 direct queries to customers, 154 images, 155 preprocessing, 156–157 social media, 154 sources, 154 text, 155 trading, 1 transformation, 160 types, 155–156 user engagement, 155–156 De-bias pricing algorithms, 117 Decision trees, 82 Decision types, 15–27 Decision-makers in marketing, 218 Deep learning (DL), 8, 82, 192, 230, 239–240 algorithms, 248, 250, 265 architectures for NLP, 200–202 causal inference, 268 combine unstructured data with structured data, 265–266 common testbeds, 268–269 customized algorithm development, 265–266 customized constraint, 266 deep learning–based language model, 174 future directions, 266–269 in marketing, 239–240, 243–244 model efficiency improvement, 267 models, 126, 220–221, 224 multimodal, five senses, and networks, 267 neural networks, 242–248 plug and play, 265 problems, 265–266 properties, 241–242 reinforcement learning, 267–268 theory-driven architecture design, 266 theory-driven initialization, 266 Deep neural networks (DNN), 81, 242 Deep Q-Network (DQN), 248–249 Deep reinforcement learning (DRL), 262–265 Deepfakes, 139–140 Demand, real-time swings in, 105–110 Diachronic word embeddings, 199–200 Dictionary and word frequency–based text mining, 179–180 Difference-in-difference estimation approach, 194–195 Digital cameras, 150 Digital exhaust of individual behavior, 1 Digital footprints, 147–148 Digital voice assistants, 139 Direct marketing context, 138 Direct method, 85 Direct queries to customers, 154 Discriminative deep learning models, 249–258 CNNs, 249–254 RNN, 254–255 transformers, 255–258 Discriminative models, 248–249 Discriminator network, 131 Disney, 88 Distributional hypothesis, 199 doc2vec, 200 Dominant color descriptor, 221 Double machine learning (DML), 178 Doubly Robust estimator (DR estimator), 86 Doubly robust method, 86 Dropout method, 248 Dynamic methods, 83–84 Dynamic models update customers, 138 Dynamic pricing, 105, 110, 115–116 E-commerce, 104 Emails, 132–133 Embedded Topic Model (ETM), 204 Embeddings, 157–158, 198–200, 204 Entropy, 87–88 Equilibrium analysis, strategic behavior and, 95–96 ERNIE 3.0, 203–204 European Union (EU), 33 Evaluation of AI methods, 161–162 Evaluative Lexicon 2.0, 197 Evidence lower bound (ELBO), 259 Example-based explanation techniques, 227–228 eXplainable Artificial Intelligence (XAI), 219 External validity, 164 Facebook (technology company), 13–14 engagement data, 156 news feed algorithm, 93 user-engagement data, 155–156 Fairness in marketing, 224 Fake reviews, 183 fastText, 198–199 Feature-level models, 221 Feedback data, 14 Field experimentation, 84 Field experiments, 229–230 Financial Times Top 50 journals (FT50 journals), 278–279, 284 Fine-tuning, 130 Firms, 3–4, 131 First-order methods, 247 Flexible supervised learning algorithms, 81 Frames, 219 Fuzzy SVM, 159 Gated recurrent unit (GRU), 255 GauGAN, 137 Gender differences, 294 General Data Protection Regulation (GDPR), 33 Generalizability, 81–82 Generative adversarial networks (GAN), 8, 131, 248–249, 260, 262 Generative deep learning models, 258–262 GAN, 260–262 VAE, 258–260 Generative models, 140, 248–249 Generative Pre-trained Transformer 3 (GPT-3), 125–126, 180–181, 198–199 Generative video models, 139 Generator network, 131 Global interpretability, 225 GLOVE, 180, 198–199 Google (technology company), 13–14 search engine algorithm, 134 search personalization, 93 Gradient-weighted class activation mapping (Grad-CAM), 161, 226–227 Heatmap method, 228 Hidden Markov Model (HMM), 165 Hulu, 88 Human–machine collaboration, 230 ImageNet, 268 Images, 155, 218 data, 220 image-based social media, 218 image/post clusters, 153 tagging, 157 Incentive-aware personalization, 96 Individual-level personalization, 77–78 InferNER approach, 195 Influence methods, 227 Input data, 14 Insight generation, 170–171 Instagram, 171 Instrumental variable approach (IV approach), 175 Interactive methods, 82–83 Internet, 151 Interpretability, 219, 228 issues, 224–228 Inverse Propensity Score estimator (IPS estimator), 85–86 ISI Web of Science, 171–172 Judgment, 14–15 Knowledge extraction, 227 LambdaMART ranking algorithm, 29 Language models, 131, 198, 204 generating textual content with, 129–131 marketing applications of, 204–205 Language structure and deep learning–based text mining, 180 Large language models, 140 Large-scale pretrained language models, 129–130 LDA, 195 Learning from audio visual data, 184 from interactive two-sided feedback, 185 Lexicons, 197–198 and word frequency–based methods, 180 Linguistic Inquiry and Word Count (LIWC), 197 Local interpretability, 226 Local Interpretable Model-Agnostic Explanations (LIME), 226 Long short-term memory (LSTM), 129, 157, 201–202, 255 Low response rates, 170 Lyft, 107 Machine learning (ML) (see also Deep learning (DL)), 14–15, 29–30, 32, 147–148, 170, 219, 240 algorithms, 150 methods, 82 promise of AI and, 149–150 Manual encoding, 219–220 Manual inspection, 160 Mapping methods to research questions, 162–165 posteriori–identified phenomena and constructs, 162–163 priori–defined constructs, 163–164 validation, 164–165 Market fairness, 224 research, 6 Marketers, 1–2 Marketing, 274 AI’s impact on consumers and society and vice versa, 8–9 algorithms and methods, 7–8 applications of language models, 204–206 communications, 132–133 data, 6–7 marketing-AI ecosystem, 2–4 modelers, 242–246 novel approaches for established tasks, 204 novel approaches for novel tasks, 204–205 opportunity identification for AI research, 10 purpose of AI, 4–6 research in marketing on Artificial Intelligence, 40–76 scholars, 1–2, 14, 28–30, 34, 193 Markov decision process (MDP), 263 Matrix factorization approaches, 81 Maximum likelihood estimation (MLE), 247 Mean Average Error (MAE), 87–88 Measurement error, 180 Megatron-Turing NLG, 125–126 Menu costs, 103 Metaphor elicitation technique, 151–152 Methodological approaches to personalization, 79–84 dynamic methods, 83–84 generalizability and counterfactual validity, 81–82 online and interactive methods, 82–83 scalability, 80–81 Metric-based evaluation, 87–88 Mind perception, 299 mini-Xception, 222–223 Model interpretability, 219 Model interpretation, 160–161 with manual inspection and data transformation, 160 post hoc model explanation, 161 Model-agnostic interpretability, 227 Model-agnostic techniques, 226 Model-specific interpretability, 226–227 Multi-armed bandit (MAB), 113 Multiarmed bandit models (MAB models), 5 Multihead attention, 202 Naive Bayes classifier, 178 Named entity extraction (NER), 195 Natural language generation models (NLG models), 125–126, 129–130 Natural language inference task (NLI task), 205 Natural language processing (NLP), 3, 7, 150, 172–173, 192 applications, 192 challenges, biases, and potential harms, 208–209 concept and topic extraction, 195–197 current state of NLP in marketing, 195–198 embeddings, language models, transfer learning, 198–204 established and novel tools for diverse text-based marketing applications, 196 marketing applications of language models, 204–205 relationship extraction, 197 roadmap and future trends, 206–207 sentiment and writing style extraction, 197–198 text in marketing, 193–195 Netflix, 88 Network embeddings, 163 Neural networks (NN), 242, 248 activation functions, 246 architecture, 246 objective function, 247 optimizer, 247–248 regularization, 248 News personalization, 83 Nonconvergence, 262 Nonparametric approach, 115 Nontech firms, 267 Nontextual data, 194 Objective function, 247 Offline beacons, 1 Online customer feedback, 175–183 AI and machine learning in analyzing unstructured review data, 178–181 challenges in learning from, 181–183 economic impact of online reviews, 177–178 Online forum discussions, 30 Online methods, 82–83 Online platforms, 147–148 Online reviews, 30 OpenCV, 222–223 Optimal algorithm, 115 Optimizer, 247–248 Overlap assumption, 82 Peer influence, 182–183 Personality, 295 Personalization algorithms, 82–83, 91 alternative approaches, 87–88 direct method, 85 doubly robust method, 86 evaluation, 84–88 extensions to special settings, 86–87 IPS estimator, 85–86 methodological approaches to personalization, 79–84 models, 94–95 multiple objectives and long-term outcomes, 94–95 problem definition, 78–79 returns to personalization, 88–90 signal-to-noise ratio, 94 strategic behavior and equilibrium analysis, 95–96 time drifts, 95 and welfare, 90–93 Personalized policy design, 78–79 Personalized pricing, 89–90, 110, 113, 117 Personification, 278 Photorealistic images, 131 Pix2pix approach, 132 Pixel-level models, 222 Plug and play language models (PPLM), 130 Poisson factorization, 197 Polarization, 93 Position encoding, 255, 257 Post hoc model explanation, 161 Posteriori–identified phenomena and constructs, 162–163 Prediction Machines , 14, 32–33 Predictions, 4, 28, 30 prediction-based algorithms, 158–159 process, 14 Predictive ML algorithms, 163 Preprocessing images, 157 Price discrimination, 110–113 Price experimentation, 113–115 Pricing automation, 105–107 consequences of AI for pricing, 115–119 dynamic pricing, 105–110 firms implementing AI for pricing, 104–115 personalized pricing, 110–113 price experimentation, 113–115 Primary data, 149 Prime Video, 88 Principal component analysis (PCA), 258 Priori–defined constructs, 163–164 Privacy, personalization and welfare, 91–92 Probabilistic content generation process, 130 Product development, 150 Propensity-based approaches, 87 Prospective customers, 126 Prototypes, 227–228 Q-learning algorithm, 115 models, 118 Q-value function approximator, 264–265 Quantitative marketers, 1 Racist language, 130 Random Forests, 81–82 Recency Frequency Monetary value (RFM value), 115 Rectified linear units (ReLu), 246 Recurrent neural networks (RNNs), 129, 159, 200–201, 248–249, 254–255 Recursive neural networks, 129 Regression models, 225 Regularization, 248 Regulators, 4 Reinforcement learning (RL), 248–249, 267–268 Relationship development, 136–138 Relationship extraction, 197 Relative Information Gain (RIG), 87–88 Representation learning, 240–241 Reputation platforms, 171 systems, 175 ResNet-50, 222 Restricted Boltzmann machine (RBM), 242 Ride-hailing platforms, 107 RoBERTa, 157, 198–199 Robots, 289–290 Rule-based learners, 226–227 Scalability, methodological approaches to personalization, 80–81 Scale-Invariant Feature Transform (SIFT), 221 SCImago Journal & Country Rank, 278–279 SE-ResNet-50, 222 Search engine optimization (SEO), 30, 133, 205 Second-order methods, 247 Seeded LDA, 195–196 Selection bias, 183 Self-attention, 255 Self-selection, 182 Self-supervised representation learning, 200 Semantic network analysis, 179–180 Sentence-based LDA, 195–196 SentenceBERT, 200 Sentiment analysis, 151, 198 Sentiment and writing style extraction, 197–198 Sequence-to-sequence models, 203 SHapley Additive exPlanations (SHAP), 161, 226 algorithm, 226 values, 163 “Shipping then shopping” strategy, 32–33 “Shop, then ship” model, 4 Short-term rental market, 107 Signal-to-noise ratio, 94 Small-and medium-sized enterprises (SMEs), 267 “Smart pricing” tool, 117 Social media, 147–148, 154, 218 messages, 135–136 messaging, 135 posts, 132–133 Social Sciences Citation Index (SSCI), 171–172 Speeded-Up Robust Features, 221 Standard reinforcement learning algorithm, 118 Stanford Named Entity recognizer, 195 “Stick-and carrot” strategies, 118 Stochastic gradient descent (SGD), 247–248 Stochastic parrots, 140 Stroop test performance, 293 Structural models, 225 Style-based GAN (StyleGAN), 131–132 Subnetworks, 131 Subscription-based “shipping-then-shopping” business model, 32–33 Supervised learning algorithms, 81 Supervised ML models, 151 Supply, real-time swings in, 105–110 Support vector machines (SVM), 29, 159, 178 Surge pricing algorithms, 107 Survey-based perceptual maps, 152 Synthetic images, generating, 131–133 Technology, 274 companies, 13–14 Text data, 7, 155 Text in marketing, 193–195 causality, 194–195 dependent variable, 194 dual role of language, 193 independent variables, 194 Text mining, 192 algorithms, 240 Textual analysis in marketing, 192–193 Textual consumer feedback, 179–181 Textual content with language models, generating, 129–131 Textures, 221 3D convolutional neural network, 220–221 TikTok, 218 Time drifts, 95 Topic modeling, 158, 192 Traditional LDA approach, 195–196 Training data, 14 Training process, 131 Transaction data, 149 Transfer learning, 198, 202, 204, 222 Transform data, 157 Transformer-based models, 157, 202, 204 Transformers, 202, 255, 258 Twitter, 171 Uber, 107 Unconditional counterfactual explanations, 227–228 Unconfoundedness assumption, 82 Underspecification, 209 Uniform policy, 79 Unstructured data, 170, 192, 218 Unsupervised learning, 157–158 clustering, 158 embeddings, 157–158 topic modeling, 158 Upper confidence bound algorithm (UCB algorithm), 115 US Congress, 116 User clusters, 153 User engagement, 155–156 User-generated content (UGC), 30, 147–149, 170–171 challenges in use of, 149 customer feedback from, 176 data preprocessing, 156–157 evaluation, 161–162 hybrid of unsupervised and supervised learning, 159–160 model interpretation, 160–161 prediction-based algorithms, 158–159 promise of, 149 tools and methods to understand, 156–162 unsupervised learning, 157–158 User-generated text, 156 User–brand interaction, 152–153 VADER, 197 Validation, 164–165 Value functions, 263 Variational autoencoders (VAE), 8, 160, 248–249, 258, 260 Vector semantics, 199–200 VGG-16 algorithm, 159 Video analytics, 7 Video content, 137 Video data, 220 Video platforms, 218 Virtual reality (VR), 7, 228–229 Visual consumer feedback, 181 Visual content, 137 Visual data, 7 Visualization techniques, 227 Voice of the Customer (VOC), 6, 147–148, 150 data available for AI AND, 154–156 importance of, 148 practice before artificial intelligence and big data, 148–149 Volume, velocity, variety (3Vs), 3 VOSviewer software, 173 Welfare fairness, 92–93 personalization and, 90–93 polarization, 93 privacy, 91–92 search cost, 91 White House’s Council of Economic Advisors (White House’s CEA), 117 Word embeddings, 198–199 Word-of-mouth (WOM), 172–173 Word2Vec (language embedding algorithm), 157, 180, 198–200 XAI methods, 224–228 model specificity, 226–228 model transparency, 224–225 scope of explanation, 225–226 XGBoost, 81, 159 Yelp, 171 YouTube, 88, 93, 218 ZIP codes, 111 Book Chapters Prelims The State of AI Research in Marketing: Active, Fertile, and Ready for Explosive Growth The Economics of Artificial Intelligence: A Marketing Perspective AI and Personalization Artificial Intelligence and Pricing Leveraging AI for Content Generation: A Customer Equity Perspective Artificial Intelligence and User-Generated Data Are Transforming How Firms Come to Understand Customer Needs Artificial Intelligence Applications to Customer Feedback Research: A Review Natural Language Processing in Marketing Marketing Through the Machine's Eyes: Image Analytics and Interpretability Deep Learning in Marketing: A Review and Research Agenda Anthropomorphism in Artificial Intelligence: A Review of Empirical Work Across Domains and Insights for Future Research Index

It was only up till recent times that computer science and its were sufficient for the application in basic principles. With the in the field of artificial intelligence, the subset Deep learning is towards substantial research and advances, creating diverse We cannot consider deep learning to be an individual approach; it is a collective term which comprises fields from contrasting to be associated with the common spine—Deep learning. Basis for strong approach in deep learning lies in cognizance of the of deep learning. The implementations can be performed vastly in fields through implication of not just one but numerous algorithms achieving our goal. The architecture of deep learning has enhanced in previous years exponentially, and as per demand, the refinement of learning implying that the architecture is dynamic. A few of the most improvised architectures are mentioned below: 30Recurrent neural networks (RNNs) Long short-term memory (LSTM)/gated recurrent unit (GRU) Convolutional neural networks (CNNs) Deep belief networks (DBN) and deep stacking networks (DSNs) Open source software options for deep learning. The area of implementation for deep learning in problem solving is vast. Feed forward networks are very effective as well as recurrent networks can be a good source for the solution of the deep learning problems. The Framework for deep learning can be implemented in software packages for the useful creation of neural network. The framework needs an implementation on a standardized scale and hence needs industrial experts for the framework to be implemented. The entire framework is in simple terms based on the Diagnosis of the problem and further, evaluating the problem. It is evident that the architecture and framework of deep learning is vast and expanding its horizons to every field possible for implementation. Therefore deep learning architecture and framework would be vitalized, with step by step conception. The architecture would be simplified as well as illustrated. All the aforesaid architecture like Recurrent neural network, Long short term memory/gated recurrent unit, convolutional, Deep belief—deep stack as well as open source would be simplified as well as illustrated.

Efficiently forecasting the demands within a hospital’s Emergency Department (ED) is critical for optimal resource allocation and patient care management. This study focuses on leveraging deep learning techniques to predict various types of ED patient flows, facilitating informed decision-making by ED managers. The rising success of deep learning networks in modeling timeseries data makes them a compelling choice for patient flow forecasting. In this context, we investigate and compare seven deep learning models-Deep Belief Network (DBN), Restricted Boltzmann Machines (RBM), Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), combined GRU and Convolutional Neural Networks (CNN-GRU), LSTM-CNN, and Generative Adversarial Network based on Recurrent Neural Networks (GAN-RNN)—to accurately forecast patient flow within a hospital’s emergency department. To enable traffic flow forecasting, a forecaster layer is introduced for each model. Real-world patient flow data spanning different ED services (biology, radiology, scanner, and echography) at Lille regional hospital in France serve as a case study to evaluate these models. Four effectiveness metrics are employed to assess and compare the forecasting methods. The outcomes demonstrate the superior performance of deep learning models in predicting ED patient flows compared to conventional shallow approaches like ridge regression and support vector regression. Significantly, the Deep Belief Network (DBN) stands out, achieving an averaged mean absolute percentage error of approximately 4.097.

Advancements in Deep Learning Algorithms is a comprehensive exploration of the cutting-edge developments in deep learning, a subset of artificial intelligence that has revolutionized the way machines learn from data. This book starts with the basics, introducing the reader to the fundamental concepts and terminologies of deep learning, before delving into the core algorithms that form the backbone of this field, including neural networks, convolutional neural networks (CNNs), recurrent neural networks (RNNs), and generative adversarial networks (GANs). It further explores advanced architectures and techniques such as attention mechanisms, deep reinforcement learning, federated learning, and autoencoders, providing a deep dive into the mechanisms that enable machines to mimic human-like learning processes. The book also addresses critical aspects of data handling and preprocessing, optimization and regularization techniques, and the practical applications of deep learning in various industries, highlighting real-world case studies. Additionally, it discusses the challenges, ethical considerations, and future implications of deploying deep learning technologies. With an eye towards recent trends and the future directions of deep learning, this book aims to equip researchers, practitioners, and enthusiasts with the knowledge to understand and leverage the potential of deep learning in solving complex problems. Keywords: Deep Learning, Neural Networks, Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Generative Adversarial Networks (GANs), Attention Mechanisms, Deep Reinforcement Learning, Federated Learning, Autoencoders, Data Preprocessing, Optimization Techniques, Artificial Intelligence, Industry Applications, Ethical Considerations, Future Directions.

Deep learning, which is originated from an artificial neural network (ANN), is one of the major technologies of today’s smart cybersecurity systems or policies to function in an intelligent manner. Popular deep learning techniques, such as multi-layer perceptron, convolutional neural network, recurrent neural network or long short-term memory, self-organizing map, auto-encoder, restricted Boltzmann machine, deep belief networks, generative adversarial network, deep transfer learning, as well as deep reinforcement learning, or their ensembles and hybrid approaches can be used to intelligently tackle the diverse cybersecurity issues. In this paper, we aim to present a comprehensive overview from the perspective of these neural networks and deep learning techniques according to today’s diverse needs. We also discuss the applicability of these techniques in various cybersecurity tasks such as intrusion detection, identification of malware or botnets, phishing, predicting cyberattacks, e.g. denial of service, fraud detection or cyberanomalies, etc. Finally, we highlight several research issues and future directions within the scope of our study in the field. Overall, the ultimate goal of this paper is to serve as a reference point and guidelines for the academia and professionals in the cyber industries, especially from the deep learning point of view.

Deep learning uses artificial neural networks to recognize patterns and learn from them to make decisions. Deep learning is a type of machine learning that uses artificial neural networks to mimic the human brain. It uses machine learning methods such as supervised, semi-supervised, or unsupervised learning strategies to learn automatically in deep architectures and has gained much popularity due to its superior ability to learn from huge amounts of data. It was found that deep learning approaches can be used for big data analysis successfully. Applications include virtual assistants such as Alexa and Siri, facial recognition, personalization, natural language processing, autonomous cars, automatic handwriting generation, news aggregation, the colorization of black and white images, the addition of sound to silent films, pixel restoration, and deep dreaming. As a review, this paper aims to categorically cover several widely used deep learning algorithms along with their architectures and their practical applications: backpropagation, autoencoders, variational autoencoders, restricted Boltzmann machines, deep belief networks, convolutional neural networks, recurrent neural networks, generative adversarial networks, capsnets, transformer, embeddings from language models, bidirectional encoder representations from transformers, and attention in natural language processing. In addition, challenges of deep learning are also presented in this paper, such as AutoML-Zero, neural architecture search, evolutionary deep learning, and others. The pros and cons of these algorithms and their applications in healthcare are explored, alongside the future direction of this domain. This paper presents a review and a checkpoint to systemize the popular algorithms and to encourage further innovation regarding their applications. For new researchers in the field of deep learning, this review can help them to obtain many details about the advantages, disadvantages, applications, and working mechanisms of a number of deep learning algorithms. In addition, we introduce detailed information on how to apply several deep learning algorithms in healthcare, such as in relation to the COVID-19 pandemic. By presenting many challenges of deep learning in one section, we hope to increase awareness of these challenges, and how they can be dealt with. This could also motivate researchers to find solutions for these challenges.

Nowadays, the most revolutionary area in computer science is deep learning algorithms and models. This paper discusses deep learning and various supervised, unsupervised, and reinforcement learning models. An overview of Artificial neural network(ANN), Convolutional neural network(CNN), Recurrent neural network (RNN), Long short-term memory(LSTM), Self-organizing maps(SOM), Restricted Boltzmann machine(RBM), Deep Belief Network (DBN), Generative adversarial network(GAN), autoencoders, long short-term memory(LSTM), Gated Recurrent Unit(GRU) and Bidirectional-LSTM is provided. Various deep-learning application areas are also discussed. The most trending Chat GPT, which can understand natural language and respond to needs in various ways, uses supervised and reinforcement learning techniques. Additionally, the limitations of deep learning are discussed. This paper provides a snapshot of deep learning.

Deep Learning Algorithms for Cybersecurity Applications: A Technological and Status Review

Purpose In the previous study of financial distress prediction, deep learning techniques performed better than traditional techniques over time-series data. This study investigates the performance of deep learning models: recurrent neural network, long short-term memory and gated recurrent unit for the financial distress prediction among the Malaysian public listed corporation over the time-series data. This study also compares the performance of logistic regression, support vector machine, neural network, decision tree and the deep learning models on single-year data. Design/methodology/approach The data used are the financial data of public listed companies that been classified as PN17 status (distress) and non-PN17 (not distress) in Malaysia. This study was conducted using machine learning library of Python programming language. Findings The findings indicate that all deep learning models used for this study achieved 90% accuracy and above with long short-term memory (LSTM) and gated recurrent unit (GRU) getting 93% accuracy. In addition, deep learning models consistently have good performance compared to the other models over single-year data. The results show LSTM and GRU getting 90% and recurrent neural network (RNN) 88% accuracy. The results also show that LSTM and GRU get better precision and recall compared to RNN. The findings of this study show that the deep learning approach will lead to better performance in financial distress prediction studies. To be added, time-series data should be highlighted in any financial distress prediction studies since it has a big impact on credit risk assessment. Research limitations/implications The first limitation of this study is the hyperparameter tuning only applied for deep learning models. Secondly, the time-series data are only used for deep learning models since the other models optimally fit on single-year data. Practical implications This study proposes recommendations that deep learning is a new approach that will lead to better performance in financial distress prediction studies. Besides that, time-series data should be highlighted in any financial distress prediction studies since the data have a big impact on the assessment of credit risk. Originality/value To the best of authors' knowledge, this article is the first study that uses the gated recurrent unit in financial distress prediction studies based on time-series data for Malaysian public listed companies. The findings of this study can help financial institutions/investors to find a better and accurate approach for credit risk assessment.

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

Industrial processes contribute significantly to environmental degradation through emissions, waste, and resource depletion. The need for real-time monitoring and mitigation strategies has led to the adoption of deep learning (DL) models for predictive analytics and automated decision-making. This study explores the application of deep learning techniques in predicting and mitigating the environmental impact of industrial activities. We review state-of-the-art deep learning architectures, including convolutional neural networks (CNNs), recurrent neural networks (RNNs), long short-term memory (LSTM) networks, and transformers, in processing large-scale environmental data. These models analyze real-time sensor data, satellite imagery, and industrial parameters to forecast pollution levels, detect anomalies, and optimize industrial operations for sustainability. Key advancements in deep learning, such as hybrid architectures integrating deep reinforcement learning (DRL) and generative adversarial networks (GANs), enhance predictive accuracy and robustness in environmental monitoring systems. Transfer learning and federated learning approaches facilitate scalable and adaptive solutions across diverse industrial sectors. The study highlights the role of DL in early detection of air and water pollution, energy consumption optimization, and emission control through predictive maintenance and process adjustments. Moreover, integrating explainable artificial intelligence (XAI) ensures model interpretability, fostering trust among policymakers and industry stakeholders. Challenges in deploying deep learning models include data heterogeneity, computational complexity, and model interpretability. To address these issues, we discuss techniques such as data augmentation, adversarial training, and edge AI implementation for real-time processing. Ethical and regulatory considerations surrounding AI-driven environmental monitoring are also examined to ensure compliance with sustainability standards. This research underscores the transformative potential of deep learning in industrial sustainability, emphasizing its role in real-time decision support systems. Future directions involve integrating quantum computing and neuromorphic computing for enhanced model efficiency and expanding interdisciplinary collaborations for AI-driven environmental governance. By leveraging deep learning for predictive environmental impact assessment, industries can transition toward greener and more efficient operational frameworks.

Deep learning has gained increasing attention in automatic speech recognition, computer vision, natural language processing, drug discovery toxicology, audio recognition, bioinformatics, and automatic driving of vehicles due to its potential benefits such as feature extraction and data classification problems. It is an evolving research domain in diverse applications which increases the overall potential cost benefits for maintenance and refurbishes activities. Deep learning is an ubiquitous technology, and the machine learning algorithms assist in modelling high-level abstract view of data by means of processing layers which encompasses complex structures. The software tools in this area provide finer representations from massive volume of unlabeled data. The software in deep learning identifies patterns in the form of digital representation such as images, data, sound, etc. According to Gartner’s hype cycle, deep learning is on “permanent peak” since 2015, and HFS research survey states that 86% of respondents believe that the technology makes a huge business impact in the industry sector. It is a rapid growing domain with a set of powerful techniques and huge amount of computational power, where machine identifies objects and translates the recognized speech in real time. The main key aspects of deep learning are (i) models comprising several stages or layers of nonlinear information processing and (ii) methodologies for supervised or unsupervised learning for feature extraction at an abstraction level. The significant reasons for its popularity are the increased size of training data set, chip processing capabilities, and the recent advancement in signal processing and machine learning research. The deep learning techniques are effectually exploiting intricate nonlinear functions to acquire hierarchical and distributed feature representations in the perspective of utilizing both labeled and unlabeled data. The different deep learning methods and architectures such as convolutional deep neural network (CDNN), deep neural network (DNN), recurrent neural network (RNN), deep belief network (DBN), artificial neural network (ANN), and long short-term memory (LSTM) are discussed.

This paper presents a detailed review of existing and emerging deep learning algorithms for time series forecasting in geotechnics and geoscience applications. Deep learning has shown promising results in addressing complex prediction problems involving large datasets and multiple interacting variables without requiring extensive feature extraction. This study provides an in-depth description of prominent deep learning methods, including recurrent neural networks (RNNs), convolutional neural networks (CNNs), generative adversarial network, deep belief network, reinforcement learning, attention and transformer algorithms as well as hybrid networks using a combination of these architectures. In addition, this paper summarizes the applications of these models in various fields, including mining and tunnelling, railway and road construction, seismology, slope stability, earth retaining and stabilizing structures, remote sensing, as well as scour and erosion. This review reveals that RNN-based models, particularly Long Short-Term Memory networks, are the most commonly used models for time series forecasting. The advantages of deep learning models over traditional machine learning, including their superior ability to handle complex patterns and process large-scale data more effectively, are discussed. Furthermore, in time series forecasting within the fields of geotechnics and geosciences, studies frequently reveal that deep learning methods tend to surpass traditional machine learning techniques in effectiveness.