Low Learning Rate Research Articles

Sigmoidal learning rate optimizer for deep neural network training using a two-phase adaptation approach

The optimization of machine learning models is an open research question since an optimal procedure to change the learning rate throughout training is still unknown. Manually defining a learning rate schedule involves troublesome, time-consuming try-and-error procedures to determine hyperparameters, such as learning rate decay epochs and learning rate decay rates, in order to navigate the intricate landscape of loss functions. Although adaptive learning rate optimizers automatize this process, recent studies suggest they may produce overfitting and reduce performance compared to fine-tuned learning rate schedules. Considering that the new machine learning loss function approaches present landscapes with much more saddle points than local minima, we proposed the Training Aware Sigmoidal Optimizer (TASO). This automated two-phase learning rate adaptation mechanism significantly reduces the need for manual hyperparameter tuning. The first phase uses a high learning rate to quickly traverse the numerous saddle points in the error surface, while the second phase uses a low learning rate to gradually approach the center of the local minimum previously found. We compared the proposed approach with commonly used adaptive learning rate schedules such as Adam, RMSProp, and Adagrad. The validation experiments were performed in image and text datasets and showed that TASO outperformed all competing methods in optimal (i.e., performing hyperparameter validation) and suboptimal (i.e., using default hyperparameters) scenarios. In our benchmark tests, TASO demonstrated promising performance, achieving an 8.32% increase in accuracy and a significant 46.62% decrease in training loss on average across various datasets and models. This remarkable performance positions TASO ahead of well-established adaptive optimizers, suggesting higher effectiveness and consistency in performance.

Progressing from first-of-a-kind to Nth-of-a-kind: Applying learning rates to carbon capture deployment in Sweden

The deployment of CO2 capture technologies presents opportunities to store fossil fuel emissions from industries and power generation (CCS) and to enable carbon utilization (CCU). However, the costs for early CCS projects are high, and this is a challenge in terms of their economic viability, requiring a strong climate policy with high carbon prices for implementation. This work details a techno-economic assessment of the cost of carbon capture based on a hybrid method and individual project approach, using first-of-a-kind contingency factors and learning rates to study the evolution of carbon capture costs as installed capacity increases over time. The work is based on a case study of 147 Swedish industrial and combined heat and power plants (total of 176 stacks). The results are presented as marginal abatement cost curves, with consideration of early mover CCS projects and learning rates. Deployment scenarios are also presented that take into account an expected increase in the CO2 price. The findings indicate that when accounting for first-of-a-kind contingencies (100 % and 200 % increases in Nth-of-a-kind costs), 90 and 17 projects, respectively, of the total 176 emission sources studied have specific CO2 costs of <300 €/t. However, high learning rates (12 %) can reduce the capture costs from first-of-a-kind to Nth-of-a-kind levels within some 30 project installations (100 % contingency). With lower learning rates (3 %), the first-of-a-kind costs are reduced by 10 %–20 %. With the expected increase in CO2 prices, a peak in carbon capture deployment is observed around Year 2035, at a carbon price of 200 €/t.

Signatures of Bayesian inference emerge from energy-efficient synapses.

Biological synaptic transmission is unreliable, and this unreliability likely degrades neural circuit performance. While there are biophysical mechanisms that can increase reliability, for instance by increasing vesicle release probability, these mechanisms cost energy. We examined four such mechanisms along with the associated scaling of the energetic costs. We then embedded these energetic costs for reliability in artificial neural networks (ANNs) with trainable stochastic synapses, and trained these networks on standard image classification tasks. The resulting networks revealed a tradeoff between circuit performance and the energetic cost of synaptic reliability. Additionally, the optimised networks exhibited two testable predictions consistent with pre-existing experimental data. Specifically, synapses with lower variability tended to have (1) higher input firing rates and (2) lower learning rates. Surprisingly, these predictions also arise when synapse statistics are inferred through Bayesian inference. Indeed, we were able to find a formal, theoretical link between the performance-reliability cost tradeoff and Bayesian inference. This connection suggests two incompatible possibilities: evolution may have chanced upon a scheme for implementing Bayesian inference by optimising energy efficiency, or alternatively, energy-efficient synapses may display signatures of Bayesian inference without actually using Bayes to reason about uncertainty.

Signatures of Bayesian inference emerge from energy-efficient synapses

Biological synaptic transmission is unreliable, and this unreliability likely degrades neural circuit performance. While there are biophysical mechanisms that can increase reliability, for instance by increasing vesicle release probability, these mechanisms cost energy. We examined four such mechanisms along with the associated scaling of the energetic costs. We then embedded these energetic costs for reliability in artificial neural networks (ANNs) with trainable stochastic synapses, and trained these networks on standard image classification tasks. The resulting networks revealed a tradeoff between circuit performance and the energetic cost of synaptic reliability. Additionally, the optimised networks exhibited two testable predictions consistent with pre-existing experimental data. Specifically, synapses with lower variability tended to have (1) higher input firing rates and (2) lower learning rates. Surprisingly, these predictions also arise when synapse statistics are inferred through Bayesian inference. Indeed, we were able to find a formal, theoretical link between the performance-reliability cost tradeoff and Bayesian inference. This connection suggests two incompatible possibilities: evolution may have chanced upon a scheme for implementing Bayesian inference by optimising energy efficiency, or alternatively, energy-efficient synapses may display signatures of Bayesian inference without actually using Bayes to reason about uncertainty.

EFL Learners' Belief in English Only Instruction: A promising or ineffective approach at the tertiary level in Iraq

Learning English has increased its popularity exponentially thanks to global needs in numerous fields. Accordingly, various means of instruction have been adopted to master English. English-only instruction, the alternative to code-switching, is one of them that urges educators and learners to communicate through English. Although many studies have examined the effectiveness of English-only instruction, a gap has been observed in the literature regarding students’ opinions at a tertiary level in the Iraq context. To this aim, 50 students who studied at TISHK International University’s Language Preparatory School in the 2022–2023 Academic Year were chosen via snowball sampling. Participants evaluated English-only instruction in terms of the overall program, instructors, and administration. The primary instrument used to collect data was a survey. Collected data, analyzed by SPSS 26 and NVivo, respectively, unearthed that English-only instruction received favorable feedback in terms of holistic English mastery, creativity, motivation, and time management. However, some communication barriers, thereby having low learning rates were cited by a few students as weaknesses of chosen instruction. Likewise, the instructors’ efforts to teach English through Web 2.0 tools, practice exams, real-life examples, and principles of student centered approach were appreciated without leaving any room for hesitation. Similarly, the administrators’ initiatives to guide, form, monitor, examine and take appropriate action to promote the quality of education with an English-only policy received positive feedback. The findings of this study may have certain implications for stakeholders who are interested in the implementation of English-only instruction at their institutions on a global scale.

Comparative Analysis of the Performance of Artificial Neural Networks in Assessing the Technical Condition of Steel Ropes

Introduction. Currently, artificial neural networks (ANN) are successfully used for technical diagnostics of steel ropes. Expensive software products with an adapted neural network implementation environment, such as STATISTICA, Amygdala, MatLab Simulink, are often used for this purpose. The most affordable way to build and train an ANN, from a financial point of view, is to write your own program code using interactive libraries such as TensorFlow, PyTorch, Scikit-learn. However, such libraries are not fully adapted for building an ANN, and to use them you need to have basic programming skills. As a result, the quality of an ANN depends not only on its architecture, training data, and composition, but also on the environment in which it is built. The aim of the work was to compare the quality of the ANN, built and trained by various methods according to the criterion of test network performance, confidence levels for assessing the technical condition of the rope, as well as the complexity and speed of training. For this purpose, new software has been developed to solve the problem of assessing the technical condition of a steel rope using a combination of various rejection indicators. Materials and Methods. The basis for an ANN training was a statistical database of typical damages of steel ropes and, an expert assessment of the technical condition of steel ropes. The software was written in the Python programming language. Various methods of programming a neural network were presented: an ANN built on the basis of the STATISTICA software package and an ANN built using the interactive Scikit-learn library. Ten test samples were prepared to verify the operation of the ANN. The ANN quality was assessed based on the test network performance and confidence probabilities (activation levels of the “winning” neuron) of determining the technical condition of the rope. Results. The construction of the ANN using the interactive library Scikit-learn showed a relatively high complexity of construction and a relatively low learning rate of the ANN. Test performance of the network, with a test sample size of ten, turned out to be the same for both built ANNs. At the same time, there was a difference in the indicator of the average confidence level for determining the technical condition of a steel rope between the results of the ANN built on the basis of the STATISTICA software package and the ANN built using the Scikit-learn interactive library. Discussion and Conclusion. The results showed that the ANN built using the STATISTICA software package with the same architecture and network learning parameters had more optimal software algorithms according to the criteria of confidence probability and network learning speed in comparison with the ANN built using the free Skicit-learn library. However, the indicator of the ANN test performance turned out to be the same for both ANNs. This result justified the use of TensorFlow, PyTorch, and Skicit-learn libraries by the world's leading research and commercial centers in the field of artificial intelligence. The obtained scientific result allows us to numerically evaluate and compare the quality of an ANN having the same architecture and learning parameters, but built using different methods. This will be useful for future scientific research in the field and for selecting the optimal environment for constructing ANNs in industrial applications.

Effects of transcranial alternating current stimulation to the supplementary motor area on motor learning.

Transcranial alternating current stimulation (tACS) is a noninvasive method for brain stimulation that artificially modulates oscillatory brain activity in the cortical region directly beneath the electrodes by applying a weak alternating current. Beta (β) oscillatory activity in the supplementary motor area (SMA) is involved in motor planning and maintenance, whereas gamma (γ) oscillatory activity is involved in the updating of motor plans. However, the effect of applying tACS to the SMA on motor learning has not yet been investigated. This study assessed the effects of applying tACS to the SMA on motor learning. Forty-two right-handed healthy adults (age 20.6 ± 0.5 years, 24 men and 18 women) were included. Motor learning was assessed using a visuomotor tracking task with pinch tension of the right thumb and right forefinger. Each trial lasted 60 s, and the error rates were measured. Conductive rubber electrodes were attached to the SMA and the left shoulder for tACS. Stimulation was applied at an intensity of 1.0 mA and frequencies of 70 and 20 Hz in the γ-tACS and β-tACS treatment groups, respectively. The sham group was only administered a fade-in/out. The visuomotor tracking task was performed for 10 trials before tACS and 10 trials after tACS. Two trials were conducted on the following day to determine motor skill retention. The average deviation measured during 60 s was considered the error value. Pre-stimulation learning rate was calculated as the change in error rate. Post-stimulation learning rate and retention rate were calculated as the change in error rate after stimulation and on the day after stimulation, respectively. In both the stimulation groups, differences in pre-stimulation learning, post-stimulation learning, and retention rates were not significant. However, in the γ-tACS group, baseline performance and pre-stimulation learning rate were positively correlated with post-stimulation learning rate. Therefore, applying γ-tACS to the SMA can increase post-stimulation learning rate in participants exhibiting low baseline performance and high pre-stimulation learning rate. Our findings suggest that motor learning can be effectively enhanced by applying γ-tACS to the SMA based on an individual's motor and learning abilities.

Different learning aberrations relate to delusion-like beliefs with different contents.

The prediction error account of delusions has had success. However, its explanation of delusions with different contents has been lacking. Persecutory delusions and paranoia are the common unfounded beliefs that others have harmful intentions towards us. Other delusions include believing that one's thoughts or actions are under external control or that events in the world have specific personal meaning. We compare learning in two different cognitive tasks, probabilistic reversal learning and Kamin blocking, that have relationships to paranoid and non-paranoid delusion-like beliefs, respectively. We find that clinical high-risk status alone does not result in different behavioural results in the probabilistic reversal learning task but that an individual's level of paranoia is associated with excessive switching behaviour. During the Kamin blocking task, paranoid individuals learned inappropriately about the blocked cue. However, they also had decreased learning about the control cue, suggesting more general learning impairments. Non-paranoid delusion-like belief conviction (but not paranoia) was associated with aberrant learning about the blocked cue but intact learning about the control cue, suggesting specific impairments in learning related to cue combination. We fit task-specific computational models separately to behavioural data to explore how latent parameters vary within individuals between tasks and how they can explain symptom-specific effects. We find that paranoia is associated with low learning rates in the probabilistic reversal learning task and the blocking task. Non-paranoid delusion-like belief conviction is instead related to parameters controlling the degree and direction of similarity between cue updating during simultaneous cue presentation. These results suggest that paranoia and other delusion-like beliefs involve dissociable deficits in learning and belief updating, which, given the transdiagnostic status of paranoia, might have differential utility in predicting psychosis.

A Comparative Study of Transformer-based Models for Text Summarization of News Articles

Transformer-based models such as GPT, T5, BART, and PEGASUS have made substantial progress in text summarization, a sub-domain of natural language processing that entails extracting important information from lengthy texts. The main objective of this research was to conduct a comparative analysis of these four transformer-based models based on their performance in text summarization of news articles. In achieving this objective, the transformer models pre-trained on extensive datasets were fine-tuned on the CNN/DailyMail dataset using a low learning rate to preserve the learned representations. The T5 transformer records the highest scores of 35.12, 22.75, 32.82, and 28.59 in ROUGE-1, ROUGE-2, ROUGE-L, and ROUGE-Lsum respectively, surpassing GPT, BART, and PEGASUS across all ROUGE metrics. The findings deduced from this study establish the proficiency of encoder-decoder models such as T5 in summary generation. Furthermore, the findings also demonstrated that the fine-tuning process's effectiveness in pre-trained models is improved when the pre-training objective closely aligns with the downstream task.

Learning and metacognition under volatility in GD: Lower learning rates and distorted coupling between action and confidence.

Decisions and learning processes are under metacognitive control, where confidence in one's actions guides future behaviour. Indeed, studies have shown that being more confident results in less action updating and learning, and vice versa. This coupling between action and confidence can be disrupted, as has been found in individuals with high compulsivity symptoms. Patients with Gambling Disorder (GD) have been shown to exhibit both higher confidence and deficits in learning. In this study, we tested the hypotheses that patients with GD display increased confidence, reduced action updating and lower learning rates. Additionally, we investigated whether the action-confidence coupling was distorted in patients with GD. To address this, 27 patients with GD and 30control participants performed a predictive inference task designed to assess action and confidence dynamics during learning under volatility. Action-updating, confidence and their coupling were assessed and computational modeling estimated parameters for learning rates, error sensitivity, and sensitivity to environmental changes. Contrary to our expectations, results revealed no significant group differences in action updating or confidence levels. Nevertheless, GD patients exhibited a weakened coupling between confidence and action, as well as lower learning rates. This suggests that patients with GD may underutilize confidence when steering future behavioral choices. Ultimately, these findings point to a disruption of metacognitive control in GD, without a general overconfidence bias in neutral, non-incentivized volatile learning contexts.

Faster Stochastic Variance Reduction Methods for Compositional MiniMax Optimization

This paper delves into the realm of stochastic optimization for compositional minimax optimization—a pivotal challenge across various machine learning domains, including deep AUC and reinforcement learning policy evaluation. Despite its significance, the problem of compositional minimax optimization is still under-explored. Adding to the complexity, current methods of compositional minimax optimization are plagued by sub-optimal complexities or heavy reliance on sizable batch sizes. To respond to these constraints, this paper introduces a novel method, called Nested STOchastic Recursive Momentum (NSTORM), which can achieve the optimal sample complexity and obtain the nearly accuracy solution, matching the existing minimax methods. We also demonstrate that NSTORM can achieve the same sample complexity under the Polyak-Lojasiewicz (PL)-condition—an insightful extension of its capabilities. Yet, NSTORM encounters an issue with its requirement for low learning rates, potentially constraining its real-world applicability in machine learning. To overcome this hurdle, we present ADAptive NSTORM (ADA-NSTORM) with adaptive learning rates. We demonstrate that ADA-NSTORM can achieve the same sample complexity but the experimental results show its more effectiveness. All the proposed complexities indicate that our proposed methods can match lower bounds to existing minimax optimizations, without requiring a large batch size in each iteration. Extensive experiments support the efficiency of our proposed methods.

Identifying slow learners in an e-learning environment using k-means clustering approach

Currently, the majority of e-learning lessons created and disseminated advocate a “one-size-fits-all” teaching philosophy. The e-learning environment, however, includes slow learners in a noticeable way, just like in traditional classroom settings. Learning analytics of educational data from a learning management system (LMS) have been considered by the researchers as a potential means of identifying slow e-learners and supporting, contesting, and altering present educational practices in e-learning. We used the students’ rates of learning and grade points along with the total learning time, which is calculated from the time series log data, to cluster the learners. The rate at which a student learns determines whether he or she is a slow learner, an average learner, or a gifted learner. For classifying learners, we followed a step-by-step procedure that included instructional design to create a dataset, learning analytics of the dataset, and a machine learning strategy to cluster e-learners. The system has been adequately integrated with the methods for measuring student learning. A strategy based on the revised Bloom’s Taxonomy is offered for the assessment of learners. The K-Means clustering approach is used to group learners who have similar performance without collecting a learner’s previous academic records or demographic information. In the experimental evaluation, 7.7% of e-learners are grouped as slow learners, while advanced learners make up 61.3 percent of the student body and average learners make up 31 percent. According to the study, there is a correlation between learning rate and academic success, with fast learners having a lower learning rate.

A Modified Gated Recurrent Unit Approach for Epileptic Electroencephalography Classification

Epilepsy is one of the most severe non-communicable brain disorders associated with sudden attacks. Electroencephalography (EEG), a non-invasive technique, records brain activities, and these recordings are routinely used for the clinical evaluation of epilepsy. EEG signal analysis for seizure identification relies on expert manual examination, which is labour-intensive, time-consuming, and prone to human error. To overcome these limitations, researchers have proposed machine learning and deep learning approaches. Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) have shown significant results in automating seizure prediction, but due to complex gated mechanisms and the storage of excessive redundant information, these approaches face slow convergence and a low learning rate. The proposed modified GRU approach includes an improved update gate unit that adjusts the update gate based on the output of the reset gate. By decreasing the amount of superfluous data in the reset gate, convergence is speeded, which improves both learning efficiency and the accuracy of epilepsy seizure prediction. The performance of the proposed approach is verified on a publicly available epileptic EEG dataset collected from the University of California, Irvine machine learning repository (UCI) in terms of performance metrics such as accuracy, precision, recall, and F1 score when it comes to diagnosing epileptic seizures. The proposed modified GRU has obtained 98.84% accuracy, 96.9% precision, 97.1 recall, and 97% F1 score. The performance results are significant because they could enhance the diagnosis and treatment of neurological disorders, leading to better patient outcomes.

A Lightweight Diabetic Retinopathy Detection Model Using a Deep-Learning Technique.

Diabetic retinopathy (DR) is a severe complication of diabetes. It affects a large portion of the population of the Kingdom of Saudi Arabia. Existing systems assist clinicians in treating DR patients. However, these systems entail significantly high computational costs. In addition, dataset imbalances may lead existing DR detection systems to produce false positive outcomes. Therefore, the author intended to develop a lightweight deep-learning (DL)-based DR-severity grading system that could be used with limited computational resources. The proposed model followed an image pre-processing approach to overcome the noise and artifacts found in fundus images. A feature extraction process using the You Only Look Once (Yolo) V7 technique was suggested. It was used to provide feature sets. The author employed a tailored quantum marine predator algorithm (QMPA) for selecting appropriate features. A hyperparameter-optimized MobileNet V3 model was utilized for predicting severity levels using images. The author generalized the proposed model using the APTOS and EyePacs datasets. The APTOS dataset contained 5590 fundus images, whereas the EyePacs dataset included 35,100 images. The outcome of the comparative analysis revealed that the proposed model achieved an accuracy of 98.0 and 98.4 and an F1 Score of 93.7 and 93.1 in the APTOS and EyePacs datasets, respectively. In terms of computational complexity, the proposed DR model required fewer parameters, fewer floating-point operations (FLOPs), a lower learning rate, and less training time to learn the key patterns of the fundus images. The lightweight nature of the proposed model can allow healthcare centers to serve patients in remote locations. The proposed model can be implemented as a mobile application to support clinicians in treating DR patients. In the future, the author will focus on improving the proposed model's efficiency to detect DR from low-quality fundus images.

How averaging individual curves transforms their shape: Mathematical analyses with application to learning and forgetting curves

This paper demonstrates how averaging over individual learning and forgetting curves gives rise to transformed averaged curves. In an earlier paper (Murre and Chessa, 2011), we already showed that averaging over exponential functions tends to give a power function. The present paper expands on the analyses with exponential functions. Also, it is shown that averaging over power functions tends to give a log power function. Moreover, a general proof is given how averaging over logarithmic functions retains that shape in a specific manner. The analyses assume that the learning rate has a specific statistical distribution, such as a beta, gamma, uniform, or half-normal distribution. Shifting these distributions to the right, so that there are no low learning rates (censoring), is analyzed as well and some general results are given. Finally, geometric averaging is analyzed, and its limits are discussed in remedying averaging artefacts.

Classification of Mild Cognitive Impairment with Deep Transfer Learning Approach using CWT based Scalogram Images

Mild Cognitive Impairment (MCI) is a condition that can occur as a person gets older, and faces problems like recognition, memory, and language skills. Early detection of MCI is crucial, as it can progress to more severe conditions like Alzheimer's disease. This study proposes a method to use Scalogram images, obtained by applying Continuous Wavelet Transform (CWT) to EEG signals and pre-trained models like ResNet50, VGG16, InceptionV3, Inception_ResNetV2 through transfer learning to classify MCI and Healthy Control (HC). Fine-tuning of the models is also used to improve the results, and various performance metrics are employed for classification. The study concludes that Inception_ResNetV2 transfer learning yielded good results, while ResNet50 and InceptionV3 transfer learning with fine-tuning resulted in higher accuracy using a low learning rate.

Postnatal Phencyclidine-Induced Deficits in Decision Making Are Ameliorated by Optogenetic Inhibition of Ventromedial Orbitofrontal Cortical Glutamate Neurons

BackgroundThe orbitofrontal cortex (OFC) is essential for decision making, and functional disruptions within the OFC are evident in schizophrenia. Postnatal phencyclidine (PCP) administration in rats is a neurodevelopmental manipulation that induces schizophrenia-relevant cognitive impairments. We aimed to determine whether manipulating OFC glutamate cell activity could ameliorate postnatal PCP-induced deficits in decision making. MethodsMale and female Wistar rats (n = 110) were administered saline or PCP on postnatal days 7, 9, and 11. In adulthood, we expressed YFP (yellow fluorescent protein) (control), ChR2 (channelrhodopsin-2) (activation), or eNpHR 3.0 (enhanced halorhodopsin) (inhibition) in glutamate neurons within the ventromedial OFC (vmOFC). Rats were tested on the probabilistic reversal learning task once daily for 20 days while we manipulated the activity of vmOFC glutamate cells. Behavioral performance was analyzed using a Q-learning computational model of reinforcement learning. ResultsCompared with saline-treated rats expressing YFP, PCP-treated rats expressing YFP completed fewer reversals, made fewer win-stay responses, and had lower learning rates. We induced similar performance impairments in saline-treated rats by activating vmOFC glutamate cells (ChR2). Strikingly, PCP-induced performance deficits were ameliorated when the activity of vmOFC glutamate cells was inhibited (halorhodopsin). ConclusionsPostnatal PCP-induced deficits in decision making are associated with hyperactivity of vmOFC glutamate cells. Thus, normalizing vmOFC activity may represent a potential therapeutic target for decision-making deficits in patients with schizophrenia.

Prolonged development of forced-choice recognition when targets are paired with non-corresponding lures

Previous research suggests that mnemonic discrimination (i.e., the ability to discriminate between previously encountered and novel stimuli even when they are highly similar) improves substantially during childhood. To further understand the development of mnemonic discrimination during childhood, the current study had 4-year-old children, 6-year-old children, and young adults complete the forced-choice Mnemonic Similarity Task (MST). The forced-choice MST offers a significant advantage in the context of developmental research because it is not sensitive to age-related differences in response criteria and includes three test formats that are theorized to be supported by different cognitive processes. A target (i.e., a previously encountered item) is paired with either a novel item (A–X), a corresponding lure (A–A′; i.e., an item mnemonically similar to the target), or a non-corresponding lure (A–B′; i.e., an item mnemonically similar to a different previously encoded item). We observed that 4-year-olds performed more poorly than 6-year-olds on the A–X and A–A′ test formats, whereas both 4- and 6-year-olds performed more poorly than young adults on the A–B′ test format. The MINERVA 2.2 computational model effectively accounted for these age-related differences. The model suggested that 4-year-olds have a lower learning rate (i.e., probability of encoding stimulus features) than 6-year-olds and young adults and that both 4- and 6-year-olds have greater encoding variability than young adults. These findings provide new insight into possible mechanisms underlying memory development during childhood and serve as the basis for multiple avenues of future research.

Research on the Deep Deterministic Policy Algorithm Based on the First-Order Inverted Pendulum

With the mature development of artificial intelligence technology, the application of intelligent control algorithms in control systems has become a trend to meet the high-performance requirements of modern society. This paper proposes a deep deterministic policy gradient (DDPG) controller design method based on deep reinforcement learning to improve system control performance. Firstly, the optimal control policy of the DDPG algorithm is derived from the Markov decision process and the Actor–Critic algorithm. Secondly, in order to avoid local optima in traditional control systems, the capacity and the settlement method of the DDPG experience pool are adjusted to absorb positive experience to accelerate convergence and to complete efficient training. In response, and to solve the overestimation of the Q value in DDPG, the overall structure of the Critic network is changed to shorten the convergence period of DDPG at low learning rates. Finally, a first-order inverted pendulum control system was constructed in a simulation environment to verify the control effectiveness of PID, DDPG, and improved DDPG. The simulation results reveal that the improved DDPG controller has a faster response to disturbances, smaller displacement, and angular displacement of the first-order inverted pendulum. The simulation further proves that the improved DDPG algorithm has better stability and convergence and stronger anti-interference ability and stability recovery. This control method provides a certain reference for the application of reinforcement learning in traditional control systems.

DEEP LEARNING FOR ORAL EPITHELIAL DYSPLASIA GRADING

Objective The present study proposes a Deep Learning model based on the binary system for grading oral epithelial dysplasia (OED) at the whole slide imaging level to eliminate inter-pathologist variability. Study Design A dataset of 99 whole slide images from three institutions were manually annotated, segmented, and fragmented into smaller patches of 299 × 299 pixels. A total of 40,893 images were sampled into 80%:10%:10% for training, validation, and independent test sets. An adaptation of ResNet50 with 2 hidden layers and 512 neurons in the fully connected layer (FC) was implemented with a low learning rate of 0.00001 for 200 epochs. Results The proposed ResNet50 reached 85.30% accuracy during training and 85.11% for validation, showing the potential of learning. The independent test showed an overall accuracy of 60%, with 61% sensitivity, 59% specificity, and 0.64AUROC, showing a lack of generalization ability for the present classification problem. Conclusion The proposed DL-based model presented a capacity for learning with the potential of achieving high accuracy but with relatively low generalization. Further work will encompass a 2-class problem (premalignant and malignant) to reinforce class separation and investigate the stability of accuracy and generalization of alternative DL models. The present study proposes a Deep Learning model based on the binary system for grading oral epithelial dysplasia (OED) at the whole slide imaging level to eliminate inter-pathologist variability. A dataset of 99 whole slide images from three institutions were manually annotated, segmented, and fragmented into smaller patches of 299 × 299 pixels. A total of 40,893 images were sampled into 80%:10%:10% for training, validation, and independent test sets. An adaptation of ResNet50 with 2 hidden layers and 512 neurons in the fully connected layer (FC) was implemented with a low learning rate of 0.00001 for 200 epochs. The proposed ResNet50 reached 85.30% accuracy during training and 85.11% for validation, showing the potential of learning. The independent test showed an overall accuracy of 60%, with 61% sensitivity, 59% specificity, and 0.64AUROC, showing a lack of generalization ability for the present classification problem. The proposed DL-based model presented a capacity for learning with the potential of achieving high accuracy but with relatively low generalization. Further work will encompass a 2-class problem (premalignant and malignant) to reinforce class separation and investigate the stability of accuracy and generalization of alternative DL models.