Continuous Space Research Articles

A Wide-Range TCSC Based ADN in Mountainous Areas Considering Hydropower-Photovoltaic-ESS Complementarity.

Due to the radial network structures, small cross-sectional lines, and light loads characteristic of existing AC distribution networks in mountainous areas, the development of active distribution networks (ADNs) in these regions has revealed significant issues with integrating distributed generation (DGs) and consuming renewable energy. Focusing on this issue, this paper proposes a wide-range thyristor-controlled series compensation (TCSC)-based ADN and presents a deep reinforcement learning (DRL)-based optimal operation strategy. This strategy takes into account the complementarity of hydropower, photovoltaic (PV) systems, and energy storage systems (ESSs) to enhance the capacity for consuming renewable energy. In the proposed ADN, a wide-range TCSC connects the sub-networks where PV and hydropower systems are located, with ESSs configured for each renewable energy generation. The designed wide-range TCSC allows for power reversal and improves power delivery efficiency, providing conditions for the optimization operation. The optimal operation issue is formulated as a Markov decision process (MDP) with continuous action space and solved using the twin delayed deep deterministic policy gradient (TD3) algorithm. The optimal objective is to maximize the consumption of renewable energy sources (RESs) and minimize line losses by coordinating the charging/discharging of ESSs with the operation mode of the TCSC. The simulation results demonstrate the effectiveness of the proposed method.

Active Inference and Reinforcement Learning: A Unified Inference on Continuous State and Action Spaces Under Partial Observability.

Reinforcement learning (RL) has garnered significant attention for developing decision-making agents that aim to maximize rewards, specified by an external supervisor, within fully observable environments. However, many real-world problems involve partial or noisy observations, where agents cannot access complete and accurate information about the environment. These problems are commonly formulated as partially observable Markov decision processes (POMDPs). Previous studies have tackled RL in POMDPs by either incorporating the memory of past actions and observations or by inferring the true state of the environment from observed data. Nevertheless, aggregating observations and actions over time becomes impractical in problems with large decision-making time horizons and high-dimensional spaces. Furthermore, inference-based RL approaches often require many environmental samples to perform well, as they focus solely on reward maximization and neglect uncertainty in the inferred state. Active inference (AIF) is a framework naturally formulated in POMDPs and directs agents to select actions by minimizing a function called expected free energy (EFE). This supplies reward-maximizing (or exploitative) behavior, as in RL, with information-seeking (or exploratory) behavior. Despite this exploratory behavior of AIF, its use is limited to problems with small time horizons and discrete spaces due to the computational challenges associated with EFE. In this article, we propose a unified principle that establishes a theoretical connection between AIF and RL, enabling seamless integration of these two approaches and overcoming their limitations in continuous space POMDP settings. We substantiate our findings with rigorous theoretical analysis, providing novel perspectives for using AIF in designing and implementing artificial agents. Experimental results demonstrate the superior learning capabilities of our method compared to other alternative RL approaches in solving partially observable tasks with continuous spaces. Notably, our approach harnesses information-seeking exploration, enabling it to effectively solve reward-free problems and rendering explicit task reward design by an external supervisor optional.

Deep generative models of protein structure uncover distant relationships across a continuous fold space

Our views of fold space implicitly rest upon many assumptions that impact how we analyze, interpret and understand protein structure, function and evolution. For instance, is there an optimal granularity in viewing protein structural similarities (e.g., architecture, topology or some other level)? Similarly, the discrete/continuous dichotomy of fold space is central, but remains unresolved. Discrete views of fold space bin similar folds into distinct, non-overlapping groups; unfortunately, such binning can miss remote relationships. While hierarchical systems like CATH are indispensable resources, less heuristic and more conceptually flexible approaches could enable more nuanced explorations of fold space. Building upon an Urfold model of protein structure, here we present a deep generative modeling framework, termed DeepUrfold, for analyzing protein relationships at scale. DeepUrfold’s learned embeddings occupy high-dimensional latent spaces that can be distilled for a given protein in terms of an amalgamated representation uniting sequence, structure and biophysical properties. This approach is structure-guided, versus being purely structure-based, and DeepUrfold learns representations that, in a sense, define superfamilies. Deploying DeepUrfold with CATH reveals evolutionarily-remote relationships that evade existing methodologies, and suggests a mostly-continuous view of fold space—a view that extends beyond simple geometric similarity, towards the realm of integrated sequence ↔ structure ↔ function properties.

Enhancing bird habitat networks in metropolitan areas: Resilience assessment and improvement strategies – A case study from Shanghai

Constructing bird habitat networks (BHNs) is crucial for maintaining the health and service equilibrium of urban ecosystems, especially in large metropolitan areas where the pressure of urbanization is intense. However, most existing BHNs fail to account for the dynamic changes and unique requirements of local species, leading to homogenized construction outcomes and ecological corridor objectives. This study employs a comprehensive approach to identify bird habitat patches using multiple high-quality sources, then utilize circuit theory and complex network theory to construct and assess the resilience of BHN. Our key findings showed: (1)93 bird habitat sources were identified, predominantly situated in the continuous green spaces of southern and southeastern Shanghai, whereas habitat sources in the city center and other densely built-up areas are more dispersed, highlighting them as prime targets for future ecological restoration efforts. (2) The distribution of bird habitat corridors exhibits significant spatial heterogeneity, with primary corridors predominantly spanning the southwestern and eastern parts of the study area, while secondary corridors are more abundant in the western and northern parts, forming a denser network, whereas the central area shows fewer and more isolated corridors. (3) The decline in structural and functional resilience was notably more rapid under targeted attacks than under random attacks, underscoring the need to prioritize crucial bird habitat sources on the city's periphery, especially near highly urbanized areas, in urban planning and biodiversity conservation efforts to sustain ecological balance and biodiversity. These insights provide a crucial scientific basis for urban planners, emphasizing the integration of biodiversity conservation into urban development strategies by optimizing ecological sources and corridors to balance development with ecological preservation.

Deployment of tethered gene drive for confined suppression in continuous space requires avoiding drive wave interference.

Gene drives have great potential for suppression of pest populations and removal of exotic invasive species. CRISPR homing suppression drive is a powerful but unconfined drive, posing risks of uncontrolled spread. Thus, developing methods for confining a gene drive is of great significance. Tethered drive combines a confined system such as Toxin-Antidote Recessive Embryo drive with a strong drive such as a homing suppression drive. It can prevent the homing drive from spreading beyond the confined drive and can be constructed readily, giving it good prospects for future development. However, we have found that care must be taken when deploying tethered drive systems in some scenarios. Simulations of tethered drive in a panmictic population model reveal that successful deployment requires a proper release ratio between the two components, tailored to prevent the suppression drive from eliminating the confined system before it has the chance to spread. Spatial models where the population moves over a one-dimensional landscape display a more serious phenomenon of drive wave interference between the two tethered drive components. If the faster suppression drive wave catches up to the confined drive wave, success is still possible, but it is dependent on drive performance and ecological parameters. Two-dimensional simulations further restrict the parameter range for drive success. Thus, careful consideration must be given to drive performance and ecological conditions, as well as specific release proposals for potential application of tethered drive systems.

Similarity and economy of scale in urban transportation networks and optimal transport-based infrastructures

Designing and optimizing the structure of urban transportation networks is a challenging task. In this study, we propose a method inspired by optimal transport theory and the principle of economy of scale that uses little information in input to generate structures that are similar to those of public transportation networks. Contrarily to standard approaches, it does not assume any initial backbone network infrastructure but rather extracts this directly from a continuous space using only a few origin and destination points, generating networks from scratch. Analyzing a set of urban train, tram and subway networks, we find a noteworthy degree of similarity in several of the studied cases between simulated and real infrastructures. By tuning one parameter, our method can simulate a range of different subway, tram and train networks that can be further used to suggest possible improvements in terms of relevant transportation properties. Outputs of our algorithm provide naturally a principled quantitative measure of similarity between two networks that can be used to automatize the selection of similar simulated networks.

Improved event-based fault detection filter for networked fuzzy systems under DoS attacks

This paper investigates an improved event-based fault detection method for networked fuzzy systems under denial-of-service (DoS) attacks. In order to solve the bandwidth occupation problem of communication network, a resilient event-triggered transmission strategy is developed. Additionally, a fault detection filter is designed to estimate the time of fault occurrence by using the residual signal. Under this framework, a novel Lyapunov functional related to attack parameters is established to analyze the exponential convergence of the error signals, and the filter gains and event-triggered parameters are obtained by solving linear matrix inequalities. The designed functional reduces the conservatism of the stability criteria significantly in contrast with the previous discontinuous Lyapunov functionals. Finally, a simulation example is provided to verify the effectiveness of the proposed method.

Film hole layout optimization for multi-row film cooling plate in continuous parameter space under non-uniform inlet temperature distribution

Film cooling is an advanced external cooling technology for protecting gas turbine blades from excessive temperatures. To counteract non-uniform heat loads caused by hot streak and swirl effects at the combustion chamber outlet, fine design of film hole positions is necessary. However, the complexity due to the numerous film holes and their positional parameters presents challenges in the independent optimization of film hole positions in a continuous parameter space. This study proposed an optimization method grounded in the divide and conquer approach to address optimization for film hole layout. The film hole layout was decomposed into multiple single rows and optimized systematically by iteratively refining the single-row film hole layout along the streamwise direction using the expectation maximization algorithm. Optimized layouts for five different inlet temperature distributions were obtained using the proposed method and adiabatic wall temperature distributions were compared and analyzed for optimized and staggered layouts. Additionally, the cooling performance of the optimized layout was evaluated against the staggered layout under conjugate heat transfer conditions and the robustness of the optimized layout under various blowing ratios and peak temperature positions was tested. The results demonstrated that the proposed method can optimize the positions of 52 film holes within 0.3h, and it was generalized to various inlet temperature distributions. By enhancing lateral interactions among downstream film jets, the lifting effect of kidney vortex in the optimized layout was weakened, bringing the film jets closer to the wall and significantly increasing coolant coverage during streamwise development. Comparative analysis reveals the superior cooling performance of the optimized layout at the blowing ratio of 1.0, evidenced by a 15.1% reduction in total input heat transfer rate and a 12.3% enhancement in overall cooling efficiency relative to the staggered layout. Under conditions with blowing ratios ranging from 0.5 to 1.5 and peak temperature position deviations, the average wall temperature of the optimized layout consistently remained lower than the staggered layout, indicating the robustness of the optimized layout to variations in blowing ratio and hot streak position.

The GFMxP and the basic extrapolation of the ghost values to solve the Poisson equation for discontinuous functions

In a recent paper, a novel coding of the Ghost Fluid Method for the variable coefficient Poisson equation with discontinuous functions (named GFMxP) was proposed. A lot of numerical tests, with all the required quantities available in a analytic form, were used to demonstrate the ability of the new procedure in modeling a sharp interface and to check the accuracy order of the solutions. In practical applications, however, the real difficulty stands in the estimation of the so-called “ghost values”, that is the values at points where the function is not only unknown, but even not defined. These values allow to compute the corrective terms enabling the use of standard finite difference formulas in presence of a singularity and/or a discontinuity, and can be only determined through some extrapolation procedure, whose truthfulness is essential to achieve a reliable result. The paper deals with such a basic issue, by testing different numerical strategies and demonstrating the strict relationship between the order of the adopted fit-model, the order of the solving scheme for the Poisson equation and the accuracy of the final solution.

Oscillatory contractile forces refine endothelial cell-cell interactions for continuous lumen formation governed by Heg1/Ccm1.

The formation and organization of complex blood vessel networks rely on various biophysical forces, yet the mechanisms governing endothelial cell-cell interactions under different mechanical inputs are not well understood. Using the dorsal longitudinal anastomotic vessel (DLAV) in zebrafish as a model, we studied the roles of multiple biophysical inputs and cerebral cavernous malformation (CCM)-related genes in angiogenesis. Our research identifies heg1 and krit1 (ccm1) as crucial for the formation of endothelial cell-cell interfaces during anastomosis. In mutants of these genes, cell-cell interfaces are entangled with fragmented apical domains. A Heg1 live reporter demonstrated that Heg1 is dynamically involved in the oscillatory constrictions along cell-cell junctions, whilst a Myosin live reporter indicated that heg1 and krit1 mutants lack actomyosin contractility along these junctions. In wild-type embryos, the oscillatory contractile forces at junctions refine endothelial cell-cell interactions by straightening junctions and eliminating excessive cell-cell interfaces. Conversely, in the absence of junctional contractility, the cell-cell interfaces become entangled and prone to collapse in both mutants, preventing the formation of a continuous luminal space. By restoring junctional contractility via optogenetic activation of RhoA, contorted junctions are straightened and disentangled. Additionally, haemodynamic forces complement actomyosin contractile forces in resolving entangled cell-cell interfaces in both wild-type and mutant embryos. Overall, our study reveals that oscillatory contractile forces governed by Heg1 and Krit1 are essential for maintaining proper endothelial cell-cell interfaces and thus for the formation of a continuous luminal space, which is essential to generate a functional vasculature.

Adaptive strategy for achieving fast synchronization between two memristor chaotic circuits without and with noisy perturbation

This paper presents an innovative approach for achieving rapid synchronization between two memristor chaotic circuits (MCCs), both with and without noise perturbations. The proposed adaptive control strategy effectively handles the uncertainty in control gains by adhering to predesigned update law. Additionally, this protocol is non-chattering and differentiable, avoiding the use of conventional discontinuous functions such as signum and absolute value functions. This method successfully mitigates the tremors caused by discontinuous functions. We derive two sufficient criteria using finite-time Lyapunov and stochastic finite-time Lyapunov stability methods. Numerical results validate the theoretical analysis and demonstrate the influence of noise intensity on convergence speed. Furthermore, the results have an application in image encryption transmission.

A survey on temporal knowledge graph embedding: Models and applications

Knowledge graph embedding (KGE), as a pivotal technology in artificial intelligence, plays a significant role in enhancing the logical reasoning and management efficiency of downstream tasks in knowledge graphs (KGs). It maps the intricate structure of a KG to a continuous vector space. Conventional KGE techniques primarily focus on representing static data within a KG. However, in the real world, facts frequently change over time, as exemplified by evolving social relationships and news events. The effective utilization of embedding technologies to represent KGs that integrate temporal data has gained significant scholarly interest. This paper comprehensively reviews the existing methods for learning KG representations that incorporate temporal data. It offers a highly intuitive perspective by categorizing temporal KGE (TKGE) methods into seven main classes based on dynamic evolution models and extensions of static KGE. The review covers various aspects of TKGE, including the background, problem definition, symbolic representation, training process, commonly used datasets, evaluation schemes, and relevant research. Furthermore, detailed descriptions of related embedding models are provided, followed by an introduction to typical downstream tasks in temporal KG scenarios. Finally, the paper concludes by summarizing the challenges faced in TKGE and outlining future research directions.

Seasonal habitat use and diel vertical migration in female spurdog in Nordic waters

BackgroundStudying habitat use and vertical movement patterns of individual fish over continuous time and space is innately challenging and has therefore largely remained elusive for a wide range of species. Amongst sharks, this applies particularly to smaller-bodied and less wide-ranging species such as the spurdog (Squalus acanthias Linnaeus, 1758), which, despite its importance for fisheries, has received limited attention in biologging and biotelemetry studies, particularly in the North-East Atlantic.MethodsTo investigate seasonal variations in fine-scale niche use and vertical movement patterns in female spurdog, we used archival data from 19 pregnant individuals that were satellite-tagged for up to 365 days in Norwegian fjords. We estimated the realised niche space with kernel densities and performed continuous wavelet analyses to identify dominant periods in vertical movement. Triaxial acceleration data were used to identify burst events and infer activity patterns.ResultsPregnant females frequently utilised shallow depths down to 300 m at temperatures between 8 and 14 °C. Oscillatory vertical moments revealed persistent diel vertical migration (DVM) patterns, with descents at dawn and ascents at dusk. This strict normal DVM behaviour dominated in winter and spring and was associated with higher levels of activity bursts, while in summer and autumn sharks predominantly selected warm waters above the thermocline with only sporadic dive and bursts events.ConclusionsThe prevalence of normal DVM behaviour in winter months linked with elevated likely foraging-related activity bursts suggests this movement behaviour to be foraging-driven. With lower number of fast starts exhibited in warm waters during the summer and autumn months, habitat use in this season might be rather driven by behavioural thermoregulation, yet other factors may also play a role. Individual and cohort-related variations indicate a complex interplay of movement behaviour and habitat use with the abiotic and biotic environment. Together with ongoing work investigating fine-scale horizontal movement as well as sex- and age-specific differences, this study provides vital information to direct the spatio-temporal distribution of a newly reopened fishery and contributes to an elevated understanding of the movement ecology of spurdog in the North-East Atlantic and beyond.Graphical

HMM for discovering decision-making dynamics using reinforcement learning experiments.

Major depressive disorder (MDD), a leading cause of years of life lived with disability, presents challenges in diagnosis and treatment due to its complex and heterogeneous nature. Emerging evidence indicates that reward processing abnormalities may serve as a behavioral marker for MDD. To measure reward processing, patients perform computer-based behavioral tasks that involve making choices or responding to stimulants that are associated with different outcomes, such as gains or losses in the laboratory. Reinforcement learning (RL) models are fitted to extract parameters that measure various aspects of reward processing (e.g. reward sensitivity) to characterize how patients make decisions in behavioral tasks. Recent findings suggest the inadequacy of characterizing reward learning solely based on a single RL model; instead, there may be a switching of decision-making processes between multiple strategies. An important scientific question is how the dynamics of strategies in decision-making affect the reward learning ability of individuals with MDD. Motivated by the probabilistic reward task within the Establishing Moderators and Biosignatures of Antidepressant Response in Clinical Care (EMBARC) study, we propose a novel RL-HMM (hidden Markov model) framework for analyzing reward-based decision-making. Our model accommodates decision-making strategy switching between two distinct approaches under an HMM: subjects making decisions based on the RL model or opting for random choices. We account for continuous RL state space and allow time-varying transition probabilities in the HMM. We introduce a computationally efficient Expectation-maximization (EM) algorithm for parameter estimation and use a nonparametric bootstrap for inference. Extensive simulation studies validate the finite-sample performance of our method. We apply our approach to the EMBARC study to show that MDD patients are less engaged in RL compared to the healthy controls, and engagement is associated with brain activities in the negative affect circuitry during an emotional conflict task.

Towards a configurable and non-hierarchical search space for NAS

Neural Architecture Search (NAS) outperforms handcrafted Neural Network (NN) design. However, current NAS methods generally use hard-coded search spaces, and predefined hierarchical architectures. As a consequence, adapting them to a new problem can be cumbersome, and it is hard to know which of the NAS algorithm or the predefined hierarchical structure impacts performance the most. To improve flexibility, and be less reliant on expert knowledge, this paper proposes a NAS methodology in which the search space is easily customizable, and allows for full network search. NAS is performed with Gaussian Process (GP)-based Bayesian Optimization (BO) in a continuous architecture embedding space. This embedding is built upon a Wasserstein Autoencoder, regularized by both a Maximum Mean Discrepancy (MMD) penalization and a Fully Input Convex Neural Network (FICNN) latent predictor, trained to infer the parameter count of architectures. This paper first assesses the embedding’s suitability for optimization by solving 2 computationally inexpensive problems: minimizing the number of parameters, and maximizing a zero-shot accuracy proxy. Then, two variants of complexity-aware NAS are performed on CIFAR-10 and STL-10, based on two different search spaces, providing competitive NN architectures with limited model sizes.

Fast generation of spectrally shaped disorder.

Media with correlated disorder display unexpected transport properties, but it is still a challenge to design structures with desired spectral features at scale. In this work, we introduce an optimal formulation of this inverse problem by means of the nonuniform fast Fourier transform, thus arriving at an algorithm capable of generating systems with arbitrary spectral properties, with a computational cost that scales O(NlogN) with system size. The method is extended to accommodate arbitrary real-space interactions, such as short-range repulsion, to simultaneously control short- and long-range correlations. We thus generate the largest-ever stealthy hyperuniform configurations in 2d (N=10^{9}) and 3d (N>10^{7}) and demonstrate the flexibility of the approach by generating structures with designed spectral features at scale. By an Ewald sphere construction we link the spectral and optical properties at the single-scattering level and show that stealthy hyperuniform structures generically display transmission gaps, providing a concrete example of fine-tuning of a physical property. We also show that large 3d power-law hyperuniformity in particle packings leads to single-scattering properties nearly identical to those of simple hard spheres. Finally, we demonstrate generalizations of the approach to impose features in either continuous or discrete real space, using constraints in either continuous or discrete reciprocal space. In particular, enforcing large spectral power at peaks with the right symmetry leads to the nondeterministic generation of quasicrystalline structures in 2d and 3d. This technique should become an essential tool to embed, and understand the role of, long-range correlations in disordered metamaterials.

Drug-Target Binding Affinity Prediction in a Continuous Latent Space Using Variational Autoencoders.

Accurate prediction of Drug-Target binding Affinity (DTA) is a daunting yet pivotal task in the sphere of drug discovery. Over the years, a plethora of deep learning-based DTA models have emerged, rendering promising results in predicting the binding affinities between drugs and their target proteins. However, in contrast to the conventional approach of modeling binding affinity in vector spaces, we propose a more nuanced modeling process in a continuous space to account for the diversity of input samples. Initially, the drug is encoded using the Simplified Molecular Input Line Entry System (SMILES), while the target sequences are characterized via a pretrained language model. Subsequently, highly correlative information is extracted utilizing residual gated convolutional neural networks. In a departure from existing deep learning-based models, our model learns the hidden representations of the drugs and targets jointly. Instead of employing two vectors, our hidden representations consist of two Gaussian distributions. To validate the effectiveness of our proposal, we conducted evaluations on commonly utilized benchmark datasets. The experimental outcomes corroborated that our method surpasses the state-of-the-art vectorial representation methods in terms of performance. This approach, therefore, offers potential enhancements in the precision of DTA predictions, potentially contributing to more efficient drug discovery processes.

ScNODE : generative model for temporal single cell transcriptomic data prediction.

Measurement of single-cell gene expression at different timepoints enables the study of cell development. However, due to the resource constraints and technical challenges associated with the single-cell experiments, researchers can only profile gene expression at discrete and sparsely sampled timepoints. This missing timepoint information impedes downstream cell developmental analyses. We propose scNODE, an end-to-end deep learning model that can predict in silico single-cell gene expression at unobserved timepoints. scNODE integrates a variational autoencoder with neural ordinary differential equations to predict gene expression using a continuous and nonlinear latent space. Importantly, we incorporate a dynamic regularization term to learn a latent space that is robust against distribution shifts when predicting single-cell gene expression at unobserved timepoints. Our evaluations on three real-world scRNA-seq datasets show that scNODE achieves higher predictive performance than state-of-the-art methods. We further demonstrate that scNODE's predictions help cell trajectory inference under the missing timepoint paradigm and the learned latent space is useful for in silico perturbation analysis of relevant genes along a developmental cell path. The data and code are publicly available at https://github.com/rsinghlab/scNODE.

Marginalized Augmented Few-Shot Domain Adaptation.

Domain adaptation (DA) has recently drawn a lot of attention, as it facilitates unlabeled target learning by borrowing knowledge from an external source domain. Most existing DA solutions seek to align feature representations between the labeled source and unlabeled target data. However, the scarcity of target data easily results in negative transfer, as it misleads the cross DA to the dominance of the source. To address the challenging few-shot domain adaptation (FSDA) problem, in this article, we propose a novel marginalized augmented FSDA (MAF) approach to address the cross-domain distribution disparity and insufficiency of target data simultaneously. On the one hand, cross-domain continuity augmentation (CCA) synthesizes abundant intermediate patterns across domains leading to a continuous domain-invariant latent space. On the other hand, sufficient source-supervised semantic augmentation (SSA) is explored to progressively diversify the conditional distribution within and across domains. Moreover, the proposed augmentation strategies are implemented efficiently via an expected transferable cross-entropy (CE) loss over the augmented distribution instead of explicit data synthesis, and minimizing the upper bound of the expected loss introduces negligible extra computing cost. Experimentally, our method outperforms the state of the art in various FSDA benchmarks, which demonstrates the effectiveness and contribution of our work. Our source code is provided at https://github.com/scottjingtt/MAF.git.

Mobile robot navigation based on intrinsic reward mechanism with TD3 algorithm

Deep reinforcement learning methods have been applied to mobile robot navigation to find the optimal path to the target. The rewards are usually given when the task is completed, which may lead to the local optima during the training procedure. It seriously affects the training efficiency and navigation performance of the mobile robot. To this end, this paper proposes an intrinsic reward mechanism with intrinsic curiosity module and randomness enhanced module, combining the TD3 (twin-delayed deep deterministic policy gradient) reinforcement learning algorithm for mobile robot navigation. It effectively resolves the issue of slow convergence caused by sparse rewards in continuous action spaces. It also encourages mobile robots to explore unknown areas and reduces the occurrence of local optima. The experimental results show that the proposed navigation method significantly improves the training efficiency of mobile robots. Out of 1000 test episodes, only 3 exceeded the maximum step limit. This approach significantly reduces the occurrence of local optima. Furthermore, it increases the success rate to an impressive 83.5%, outperforms the existing navigation methods.