Stochastic Gradient Descent Optimization Research Articles

An ensemble deep learning model for automatic classification of cotton leaves diseases

<p>Cotton plant (Gossypium herbaceum), is one of the significant fiber crop grown worldwide. However, the crop is quite prone to leaves diseases, for which deep learning (DL) techniques can be utilized for early disease prediction and prevent stakeholders from losing the harvest. The objective of this paper is to develop a novel ensemble based deep convolutional neural network (DCNN) model developed on two base pretrained models named: VGG16 and InceptionV3 for early detection of cotton leaves diseases. The proposed ensemble model trained on cotton leaves dataset reports higher training and testing prediction accuracies as compared to the base pretrained models. Given that, deep learning architectures have hyper-parameters, this paper presents exhaustive experimental evaluations on ensemble model to tune hyper-parameters named learning rate, optimizer and no of epochs. The suggested hyper-parameter settings can be directly utilized while employing the ensemble model for cotton plant leaves disease detection and prediction. With suggested hyper-parameters settings of learning rate 0.0001, 20 epochs and stochastic gradient descent (SGD) optimizer, ensemble model reported training and testing accuracies of 98% and 95% respectively, which was higher than the training and testing accuracies of VGG16 and InceptionV3 pretrained DCNN models.</p>

Telugu letters dataset and parallel deep convolutional neural network with a SGD optimizer model for TCR

<p>Because of the rapid growth in technology breakthroughs, including multimedia and cell phones, Telugu character recognition (TCR) has recently become a popular study area. It is still necessary to construct automated and intelligent online TCR models, even if many studies have focused on offline TCR models. The Telugu character dataset construction and validation using an Inception and ResNet-based model are presented. The collection of 645 letters in the dataset includes 18 Achus, 38 Hallus, 35 Othulu, 34×16 Guninthamulu, and 10 Ankelu. The proposed technique aims to efficiently recognize and identify distinctive Telugu characters online. This model's main pre-processing steps to achieve its goals include normalization, smoothing, and interpolation. Improved recognition performance can be attained by using stochastic gradient descent (SGD) to optimize the model's hyperparameters.</p>

Performance analysis of data resampling on class imbalance and classification techniques on multi-omics data for cancer classification.

Cancer, in any of its forms, remains a significant public health concern worldwide. Advances in early detection and treatment could lead to a decline in the overall death rate from cancer in recent decades. Therefore, tumor prediction and classification play an important role in fighting cancer. This study built computational models for a joint analysis of RNA seq, copy number variation (CNV), and DNA methylation to classify normal and tumor samples across liver cancer, breast cancer, and colon adenocarcinoma from The Cancer Genome Atlas (TCGA) dataset. Total of 18 machine learning methods were evaluated based on the AUC, precision, recall, and F-measure. Besides, five techniques were compared to ameliorate problems of class imbalance in the cancer datasets. Synthetic Minority Oversampling Technique (SMOTE) demonstrated the best performance. The results indicate that the model applying Stochastic Gradient Descent (SGD) for learning binary class SVM with hinge loss has the highest classification results on liver cancer and breast cancer datasets, with accuracy over 99% and AUC greater than or equal to 0.999. For colon adenocarcinoma dataset, both SGD and Sequential Minimal Optimization (SMO) that implements John Platt's sequential minimal optimization algorithm for training a support vector machine shows an outstanding classification performance with accuracy of 100%, AUC, precision, recall, and F-measure all at 1.000.

Artificial intelligence-based model for predicting pulmonary arterial hypertension on chest x-ray images.

Pulmonary arterial hypertension is a serious medical condition. However, the condition is often misdiagnosed or a rather long delay occurs from symptom onset to diagnosis, associated with decreased 5-year survival. In this study, we developed and tested a deep-learning algorithm to detect pulmonary arterial hypertension using chest X-ray (CXR) images. From the image archive of Chiba University Hospital, 259 CXR images from 145 patients with pulmonary arterial hypertension and 260 CXR images from 260 control patients were identified; of which 418 were used for training and 101 were used for testing. Using the testing dataset for each image, the algorithm outputted a numerical value from 0 to 1 (the probability of the pulmonary arterial hypertension score). The training process employed a binary cross-entropy loss function with stochastic gradient descent optimization (learning rate parameter, α = 0.01). In addition, using the same testing dataset, the algorithm's ability to identify pulmonary arterial hypertension was compared with that of experienced doctors. The area under the curve (AUC) of the receiver operating characteristic curve for the detection ability of the algorithm was 0.988. Using an AUC threshold of 0.69, the sensitivity and specificity of the algorithm were 0.933 and 0.982, respectively. The AUC of the algorithm's detection ability was superior to that of the doctors. The CXR image-derived deep-learning algorithm had superior pulmonary arterial hypertension detection capability compared with that of experienced doctors.

LB4TL: A Smooth Semantics for Temporal Logic to Train Neural Feedback Controllers

This paper presents a framework for training neural network (NN)-based feedback controllers for autonomous agents with deterministic nonlinear dynamics to satisfy task objectives and safety constraints expressed in discrete-time Signal Temporal Logic (DT-STL). Control synthesis that uses the robustness semantics of DT-STL poses challenges due to its non-convexity, non-differentiability, and recursive definition, in particular when it is used to train NN-based controllers. We introduce a smooth neuro-symbolic computation graph to encode DT-STL robustness to represent a smooth approximation of the robustness, enabling the use of powerful stochastic gradient descent and backpropagation-based optimization for training. Our approximation guarantees that it lower bounds the robustness value of a given DT-STL formula, and shows orders of magnitude improvement over existing smooth approximations when applied to control synthesis. We demonstrate our approach on planning to satisfy complex spatio-temporal and sequential tasks, and show scalability with formula complexity.

Convolution Neural Networks of Dynamically Sized Filters with Modified Stochastic Gradient Descent Optimizer for Sound Classification

Convolution Neural Networks of Dynamically Sized Filters with Modified Stochastic Gradient Descent Optimizer for Sound Classification

Automated Detection of Dental Caries from Oral Images using Deep Convolutional Neural Networks

The urgent demand for accurate and efficient diagnostic methods to combat oral diseases, particularly dental caries, has led to the exploration of advanced techniques. Dental caries, caused by bacterial activities that weaken tooth enamel, can result in severe cavities and infections if not promptly treated. Despite existing imaging techniques, consistent and early diagnoses remain challenging. Traditional approaches, such as visual and tactile examinations, are prone to variations in expertise, necessitating more objective diagnostic tools. This study leverages deep learning to propose an explainable methodology for automated dental caries detection in images. Utilizing pre-trained convolutional neural networks (CNNs) including VGG-16, VGG-19, DenseNet-121, and Inception V3, we investigate different models and preprocessing techniques, such as histogram equalization and Sobel edge detection, to enhance the detection process. Our comprehensive experiments on a dataset of 884 oral images demonstrate the efficacy of the proposed approach in achieving accurate caries detection. Notably, the VGG-16 model achieves the best accuracy of 98.3% using the stochastic gradient descent (SGD) optimizer with Nesterov’s momentum. This research contributes to the field by introducing an interpretable deep learning-based solution for automated dental caries detection, enhancing diagnostic accuracy, and offering potential insights for dental health assessment.

Artificial intelligence applied to analyzes during the pandemic: COVID-19 beds occupancy in the state of Rio Grande do Norte, Brazil.

The COVID-19 pandemic is already considered one of the biggest global health crises. In Rio Grande do Norte, a Brazilian state, the RegulaRN platform was the health information system used to regulate beds for patients with COVID-19. This article explored machine learning and deep learning techniques with RegulaRN data in order to identify the best models and parameters to predict the outcome of a hospitalized patient. A total of 25,366 bed regulations for COVID-19 patients were analyzed. The data analyzed comes from the RegulaRN Platform database from April 2020 to August 2022. From these data, the nine most pertinent characteristics were selected from the twenty available, and blank or inconclusive data were excluded. This was followed by the following steps: data pre-processing, database balancing, training, and test. The results showed better performance in terms of accuracy (84.01%), precision (79.57%), and F1-score (81.00%) for the Multilayer Perceptron model with Stochastic Gradient Descent optimizer. The best results for recall (84.67%), specificity (84.67%), and ROC-AUC (91.6%) were achieved by Root Mean Squared Propagation. This study compared different computational methods of machine and deep learning whose objective was to classify bed regulation data for patients with COVID-19 from the RegulaRN Platform. The results have made it possible to identify the best model to help health professionals during the process of regulating beds for patients with COVID-19. The scientific findings of this article demonstrate that the computational methods used applied through a digital health solution, can assist in the decision-making of medical regulators and government institutions in situations of public health crisis.

AdaInject: Injection-Based Adaptive Gradient Descent Optimizers for Convolutional Neural Networks

The convolutional neural networks (CNNs) are generally trained using stochastic gradient descent (SGD) based optimization techniques. The existing SGD optimizers generally suffer with the overshooting of the minimum and oscillation near minimum. In this paper, we propose a new approach, hereafter referred as AdaInject, for the gradient descent optimizers by injecting the second order moment into the first order moment. Specifically, the short-term change in parameter is used as a weight to inject the second order moment in the update rule. The AdaInject optimizer controls the parameter update, avoids the overshooting of the minimum and reduces the oscillation near minimum. The proposed approach is generic in nature and can be integrated with any existing SGD optimizer. The effectiveness of the AdaInject optimizer is explained intuitively as well as through some toy examples. We also show the convergence property of the proposed injection based optimizer. Further, we depict the efficacy of the AdaInject approach through extensive experiments in conjunction with the state-of-the-art optimizers, namely AdamInject, diffGradInject, RadamInject, and AdaBeliefInject on four benchmark datasets. Different CNN models are used in the experiments. A highest improvement in the top-1 classification error rate of 16.54% is observed using diffGradInject optimizer with ResNeXt29 model over the CIFAR10 dataset. Overall, we observe very promising performance improvement of existing optimizers with the proposed AdaInject approach.

A data-driven game theoretic multi-objective hybrid algorithm for the Dial-A-Ride Problem with multiple time windows

The Dial-A-Ride Problem (DARP) designs pick-up and delivery routes for a set of customers. It arises in door-to-door transport services tailored to elderly and impaired people. It minimizes operational costs while accommodating as many drivers’ and customers’ constraints as possible; e.g., constraints on transit time and time windows on pick up and drop off.This paper develops a two-layer and a three-layer artificial neural network (ANN) to predict DARP parameters. The ANN is trained with real data provided by a transit agency from the Canadian city/region of Vancouver. Experimental results show that a three-layer ANN with rectified linear activation functions in conjunction with a stochastic gradient descent optimizer provides the most accurate output forecasts.Utilizing the predictions as parameter estimates for a deterministic mean DARP with multiple time windows (DARPMTW), this paper models the problem as a Company-Drivers-Customers’ theoretic game that minimizes the total route duration and maximizes the minimal satisfaction of the drivers and the customers. It shows that non-dominated Pareto feasible solutions satisfy the Nash equilibrium conditions of the game-theoretic model. It then creates a multi-objective hybrid adaptive large neighborhood search (DD-HALNS) to estimate the Pareto front. The multi-objective DD-HALNS algorithm confines its search to feasible solutions that satisfy the Nash equilibrium conditions and employs local search operators based on their success rates. The operators are controlled by a learning mechanism that utilizes previous successful moves, cost savings, and utility maximization. Four hybridization operators are applied, namely Simulated Annealing, Tabu Lists, Genetic Crossover, and Restarts. The results of experiments conducted on real-world DARPMTW instances demonstrate the ability of the multi-objective DD-HALNS to enhance the best-known routing solutions, as well as its implementability.

Internet of things (IoT) based heart disease classification framework using deep learning techniques

The usage of cloud-based grid computing services and Internet of Things (IoT) devices in medical diagnoses is increasing enormously. The cloud service provider’s data centers store vast amounts of data without processing it. This big data need some intelligent technique to analyze and classify heart disease from the considerable volume of data; it is a challenging task. Many deep learning techniques are introduced earlier for heart disease diagnosis in the literature study. Still, all other classification techniques failed to achieve the minimum loss in heart disease classification with the highest accuracy and faster performance. This research introduces a new classification approach to overcome these issues: elephant herding optimizer turned restricted Boltzmann machine EHO-RBM network. The optimizer is used in this network to optimize the number of neuron utilization during the learning process by updating the network weight without compromising the loss. The previous research proves that the optimizer is performed well in reaching global minima efficiently. Therefore, the new classifier incorporates the optimizers instead of the classical stochastic gradient descent optimizer to improve the network performance by minimizing the global minima faster with less loss in predicting heart disease. The simulation result of the new heart disease classification framework shows that the elephant herding optimizer-trained classification model has reduced the loss rate and maximized the accuracy rate up to 0.0027 then the comparison method. As a result, the new classifier has obtained a maximum accuracy of up to 99.96% .

Prediction of leather footwear export using learning algorithms based on ANN model

Leather footwear export plays a crucial role in the Indian economy as India is the second largest footwear producer in the world. As a commodity, it is unavoidable to emphasize its export performance by forecasting. This paper aims to bring out an Artificial Neural Network based model to predict India’s leather footwear export. Towards forecasting Leather footwear export, the dataset comprising five commodities covered under leather footwear has been taken from 1996 to 97 to 2021–22. The authors have proposed India’s Leather Footwear Export - Artificial Neural Network (ILFE-ANN) model with SGD optimizer and activation functions such as Sigmoid / Logistic and Rectified linear unit (ReLU). The authors have kept null values as it is in the data than replacing them with imputation methods such as mean and median while modelling. Outliers are replaced with the mean value of the remaining data before modelling. Moreover, different learning algorithms such as Adaptive Moment (Adam), RMS Propagation (RMSProp), Stochastic Gradient Descent (SGD) and SGD with Momentum (SGDM) have been compared to choose an optimal one before being implemented in the ILFE-ANN. The validation of the ILFE-ANN model has been implemented for the prediction of livestock population and compared with the Regression model. The variation percentage confirms that the proposed ILFE-ANN model performs significantly with 0.51% RMSProp, 1.68 % SGDM and 2.54 % SGD. Further, the minimum value of performance metrics MAE, MAPE and RMSE obtained are 0.4, 0.5 and 0.5 respectively for the prediction of sheep population for the year 2017. It shows that SGD performs better with the least error rate of 8% MAPE for the export of leather Commodity 64035113. Hence, the study confirms that the ANN model with SGD optimizer performs better for the prediction of India’s leather footwear trade data.

Gradient bald vulture optimization enabled multi-objective Unet++ with DCNN for prostate cancer segmentation and detection

Prostate cancer (PCa) represents the general type of cancer and is considered the third leading reason of death worldwide. As a combined part of computer-aided detection (CAD) applications, magnetic resonance imaging (MRI) is extensively studied for the precise detection of PCa. However, various issues rely on MRI, which includes the complexity of interpretation and increased time. Thus, deep learning-based tumor detection and segmentation methods attempt to be imperative techniques for radiologists to execute their tasks more precisely. The objective is to present Gradient Bald vulture optimization (GBVO)-based Deep Convolution Neural Networks (DCNN) with U-Net++ for segmenting and detecting prostate cancer. Initially, image pre-processing is done using Non-Local Means (NLM) filter. After pre-processing, image segmentation is carried out using the proposed optimized multi-objective Unet++, where the objective function in Unet++ is modified using pixel-wise cross entropy and the Jaccard coefficient. In addition, the training of Unet++ is done using the newly designed Gradient Bald Eagle Optimization (GBEO), which is a combination of Stochastic Gradient Descent (SGD) and Bald Eagle optimization (BEO). Finally, cancer detection is done using DCNN. DCNN is trained using GBVO, which is the integration of the African Vultures Optimization Algorithm (AVOA) and GBEO. The proposed method outperformed state-of-the-art techniques. The developed method achieved the highest accuracy of 0.916, a false negative ratio (FNR) of 0.104, a false positive ratio (FPR) of 0.100, and a negative predictive value (NPV) of 0.903.

Semiparametric regression for spatial data via deep learning

In this work, we propose a deep learning-based method to perform semiparametric regression analysis for spatially dependent data. To be specific, we use a sparsely connected deep neural network with rectified linear unit (ReLU) activation function to estimate the unknown regression function that describes the relationship between response and covariates in the presence of spatial dependence. Under some mild conditions, the estimator is proven to be consistent, and the rate of convergence is determined by three factors: (1) the architecture of neural network class, (2) the smoothness and (intrinsic) dimension of true mean function, and (3) the magnitude of spatial dependence. Our method can handle well large data set owing to the stochastic gradient descent optimization algorithm. Simulation studies on synthetic data are conducted to assess the finite sample performance, the results of which indicate that the proposed method is capable of picking up the intricate relationship between response and covariates. Finally, a real data analysis is provided to demonstrate the validity and effectiveness of the proposed method.

Deep Learning Based Quality Prediction of Retinal Fundus Images

Abstract The accuracy of diagnosing and monitoring eye diseases using fundus imaging is strongly dependent on the quality of the images. Poor image quality can result in delays or inaccuracies in diagnosis, thus risking patient health. Image quality assessment (IQA) is crucial for regulating retinal imaging quality and assisting in successful diagnoses. In this paper, a learning-based model for evaluating the quality of fundus images is proposed. The EyeQ dataset is selected due to its ease of availability and large collection of images. The images in the dataset are classified into three labels based on their perceptual quality. Label 0 denotes good, 1 denotes fair and 2 denotes poor quality. The ResNet50 deep learning model is fine-tuned with different hyperparameters to develop two models, which are labelled as network-1 and network-2. Network-1 uses the Stochastic Gradient Descent (SGD) optimizer and network-2 utilizes the Adam optimizer. Both networks utilise the Categorical Cross Entropy (CCE) loss function, are trained for 40 epochs and has a learning rate of 1e-3. The performance of the trained models is compared using the accuracy validation curve and the loss validation curve. Network-1 demonstrates an accuracy value of 85.8% with a loss of 0.37 and network-2 achieves an accuracy of 86.2% with a loss of 0.34. The results indicate that network-2 slightly outperforms the performance of the other in terms of accuracy, and should be preferred. Moreover, it is evident that the ResNet50 model is highly adept in appropriate feature extraction and quality evaluation of retinal fundus imaging.

Parameter Efficient Neural Networks With Singular Value Decomposed Kernels.

Traditionally, neural networks are viewed from the perspective of connected neuron layers represented as matrix multiplications. We propose to compose these weight matrices from a set of orthogonal basis matrices by approaching them as elements of the real matrices vector space under addition and multiplication. Making use of the Kronecker product for vectors, this composition is unified with the singular value decomposition (SVD) of the weight matrix. The orthogonal components of this SVD are trained with a descent curve on the Stiefel manifold using the Cayley transform. Next, update equations for the singular values and initialization routines are derived. Finally, acceleration for stochastic gradient descent optimization using this formulation is discussed. Our proposed method allows more parameter-efficient representations of weight matrices in neural networks. These decomposed weight matrices achieve maximal performance in both standard and more complicated neural architectures. Furthermore, the more parameter-efficient decomposed layers are shown to be less dependent on optimization and better conditioned. As a tradeoff, training time is increased up to a factor of 2. These observations are consequently attributed to the properties of the method and choice of optimization over the manifold of orthogonal matrices.

A Modified Stochastic Gradient Descent Optimization Algorithm With Random Learning Rate for Machine Learning and Deep Learning

A Modified Stochastic Gradient Descent Optimization Algorithm With Random Learning Rate for Machine Learning and Deep Learning

Convolutional Neural Networks (CNNs) for Pneumonia Classification on Pediatric Chest Radiographs.

Pneumonia is an infectious disease that is especially harmful to those with weak immune systems, such as children under the age of 5. While radiologists' diagnosis of pediatric pneumonia on chest radiographs (CXRs) is often accurate, subtle findings can be missed due to the subjective nature of the diagnosis process. Artificial intelligence (AI) techniques, such as convolutional neural networks (CNNs), can help make the process more objective and precise. However, off-the-shelf CNNs may perform poorly if they are not tuned to their appropriate hyperparameters. Our study aimed to identify the CNNs and their hyperparameter combinations (dropout, batch size, and optimizer) that optimize model performance. Sixty models based on five CNNs (VGG 16, VGG 19, DenseNet 121, DenseNet 169, and InceptionResNet V2) and 12 hyperparameter combinations were tested. Adam, Root Mean Squared Propagation (RmsProp), and Mini-Batch Stochastic Gradient Descent (SGD) optimizers were used. Two batch sizes, 32 and 64, were utilized. A dropout rate of either 0.5 or 0.7 was used in all dropout layers. We used a deidentified CXR dataset of 4200 pneumonia (Figure 1a) and 1600 normal images (Figure 1b).Seventy percent of the CXRs in the dataset were used for training the model, 20% were used for validating the model, and 10% were used for testing the model. All CNNs were trained first on the ImageNet dataset. They were then trained, with frozen weights, on the CXR-containing dataset. Results:Among the 60 models, VGG-19 (dropout of 0.5, batch size of 32, and Adam optimizer) was the most accurate. This model achieved an accuracy of 87.9%. A dropout of 0.5 consistently gave higher accuracy, area under the receiver operating characteristics curve (AUROC), and area under the precision-recall curve (AUPRC) compared to a dropout of 0.7. The CNNs InceptionResNet V2, DenseNet 169, VGG 16, and VGG 19 significantly outperformed the DenseNet121 CNN in accuracy and AUROC. The Adam and RmsProp optimizer had improved AUROC and AUPRC compared to the SGD optimizer. The batch size had no statistically significant effect on model performance. We recommend using low dropout rates (0.5) and RmsProp or Adam optimizer for pneumonia-detecting CNNs. Additionally, we discourage using the DenseNet121 CNN when other CNNs are available. Finally, the batch size may be set to any value, dependent on computational resources.

Learning the random variables in Monte Carlo simulations with stochastic gradient descent: Machine learning for parametric PDEs and financial derivative pricing

AbstractIn financial engineering, prices of financial products are computed approximately many times each trading day with (slightly) different parameters in each calculation. In many financial models such prices can be approximated by means of Monte Carlo (MC) simulations. To obtain a good approximation the MC sample size usually needs to be considerably large resulting in a long computing time to obtain a single approximation. A natural deep learning approach to reduce the computation time when new prices need to be calculated as quickly as possible would be to train an artificial neural network (ANN) to learn the function which maps parameters of the model and of the financial product to the price of the financial product. However, empirically it turns out that this approach leads to approximations with unacceptably high errors, in particular when the error is measured in the ‐norm, and it seems that ANNs are not capable to closely approximate prices of financial products in dependence on the model and product parameters in real life applications. This is not entirely surprising given the high‐dimensional nature of the problem and the fact that it has recently been proved for a large class of algorithms, including the deep learning approach outlined above, that such methods are in general not capable to overcome the curse of dimensionality for such approximation problems in the ‐norm. In this article we introduce a new numerical approximation strategy for parametric approximation problems including the parametric financial pricing problems described above and we illustrate by means of several numerical experiments that the introduced approximation strategy achieves a very high accuracy for a variety of high‐dimensional parametric approximation problems, even in the ‐norm. A central aspect of the approximation strategy proposed in this article is to combine MC algorithms with machine learning techniques to, roughly speaking, learn the random variables (LRV) in MC simulations. In other words, we employ stochastic gradient descent (SGD) optimization methods not to train parameters of standard ANNs but instead to learn random variables appearing in MC approximations. In that sense, the proposed LRV strategy has strong links to Quasi‐Monte Carlo (QMC) methods as well as to the field of algorithm learning. Our numerical simulations strongly indicate that the LRV strategy might indeed be capable to overcome the curse of dimensionality in the ‐norm in several cases where the standard deep learning approach has been proven not to be able to do so. This is not a contradiction to the established lower bounds mentioned above because this new LRV strategy is outside of the class of algorithms for which lower bounds have been established in the scientific literature. The proposed LRV strategy is of general nature and not only restricted to the parametric financial pricing problems described above, but applicable to a large class of approximation problems. In this article we numerically test the LRV strategy in the case of the pricing of European call options in the Black‐Scholes model with one underlying asset, in the case of the pricing of European worst‐of basket put options in the Black‐Scholes model with three underlying assets, in the case of the pricing of European average put options in the Black‐Scholes model with three underlying assets and knock‐in barriers, as well as in the case of stochastic Lorentz equations. For these examples the LRV strategy produces highly convincing numerical results when compared with standard MC simulations, QMC simulations using Sobol sequences, SGD‐trained shallow ANNs, and SGD‐trained deep ANNs.

Bi‐directional LSTM based channel estimation in 5G massive MIMO OFDM systems over TDL‐C model with Rayleigh fading distribution

SummaryIn this work, a deep learning (DL)‐based massive multiple‐input multiple‐output (mMIMO) orthogonal frequency division multiplexing (OFDM) system is investigated over the tapped delay line type C (TDL‐C) model with a Rayleigh fading distribution at frequencies ranging from 0.5 to 100 GHz. The proposed bi‐directional long short‐term memory (Bi‐LSTM) channel state information (CSI) estimator uses online learning during training and offline learning during the practical implementation phase. The design of the estimator takes into account situations in which prior knowledge of channel statistics is limited and targets excellent performance, even with limited pilot symbols (PS). Three separate loss functions (mean square logarithmic error [MSLE], Huber, and Kullback–Leibler Distance [KLD]) are assessed in three classification layers. The symbol error rate (SER) and outage probability performance of the proposed estimator are evaluated using a number of optimization techniques, such as stochastic gradient descent (SGD), momentum, and the adaptive gradient (AdaGrad) algorithm. The Bi‐LSTM‐based CSI estimator is trained considering a specific number of PS. It can be readily seen that by incorporating a cyclic prefix (CP), the system becomes more resilient to channel impairments, resulting in a lower SER. Simulations show that the SGD optimization approach and Huber loss function‐trained Bi‐LSTM‐based CSI estimator have the lowest SER and very high estimation accuracy. By using deep neural networks (DNNs), the Bi‐LSTM method for CSI estimation achieves a superior channel capacity (in bps/Hz) at 10 dB than long short‐term memory (LSTM) and other conventional CSI estimators, such as minimum mean square error (MMSE) and least squares (LS). The simulation results validate the analytical results in the study.