Exploration Exploitation Research Articles

Exploring action-oriented models via active inference for autonomous vehicles

Being able to robustly interact with and navigate a dynamic environment has been a long-standing challenge in intelligent transportation systems. Autonomous agents can use models that mimic the human brain to learn how to respond to other participants’ actions in the environment and proactively coordinate with the dynamics. Modeling brain learning procedures is challenging for multiple reasons, such as stochasticity, multimodality, and unobservant intents. Active inference may be defined as the Bayesian modeling of a brain with a biologically plausible model of the agent. Its primary idea relies on the free energy principle and the prior preference of the agent. It enables the agent to choose an action that leads to its preferred future observations. An exploring action-oriented model is introduced to address the inference complexity and solve the exploration–exploitation dilemma in unobserved environments. It is conducted by adapting active inference to an imitation learning approach and finding a theoretical connection between them. We present a multimodal self-awareness architecture for autonomous driving systems where the proposed techniques are evaluated on their ability to model proper driving behavior. Experimental results provide the basis for the intelligent driving system to make more human-like decisions and improve agent performance to avoid a collision.

Mixed Thermal and Renewable Energy Generation Optimization in Non-Interconnected Regions via Boolean Mapping

Global efforts aiming to shift towards renewable energy and smart grid configurations require accurate unit commitment schedules to guarantee power balance and ensure feasible operation under different complex constraints. Intelligent systems utilizing hybrid and high-level techniques have arisen as promising solutions to provide optimum exploration–exploitation trade-offs at the expense of computational complexity. To ameliorate this requirement, which is extremely expensive in non-interconnected renewable systems, radically different approaches based on enhanced priority schemes and Boolean encoding/decoding have to take place. This compilation encompasses various mappings that convert multi-valued clausal forms into Boolean expressions with equivalent satisfiability. Avoiding any need to introduce prior parameter settings, the solution utilizes state-of-the-art advancements in the field of artificial intelligence models, namely Boolean mapping. It allows for the efficient identification of the optimal configuration of a non-convex system with binary and discontinuous dynamics in the fewest possible trials, providing impressive performance. In this way, Boolean mapping becomes capable of providing global optimum solutions to unit commitment utilizing fully tractable procedures without deteriorating the computational time. The results, considering a non-interconnected power system, show that the proposed model based on artificial intelligence presents advantageous performance in terms of generating cost and complexity. This is particularly important in isolated networks, where even a-not-so great deviation between production and consumption may reflect as a major disturbance in terms of frequency and voltage.

The Effect of Reward Magnitude on Different Types of Exploration in Human Reinforcement Learning

AbstractHow humans resolve the explore–exploit dilemma in complex environments is an important open question. Previous studies suggested that environmental richness may affect the degree of exploration in a type-specific manner and reduce random exploration while increasing uncertainty-based exploration. Our study examined this possibility by extending a recently developed two-armed bandit task that can dissociate the uncertainty and novelty of stimuli. To extract the pure effect of environmental richness, we manipulated the reward by its magnitude, not its probability, across blocks because reward probability affects outcome controllability. Participants (N = 198) demonstrated increased optimal choices when the relative reward magnitude was higher. A behavioral analysis with computational modeling revealed that a higher reward magnitude reduced the degree of random exploration but had little effect on the degree of uncertainty- and novelty-based exploration. These results suggest that humans modulate their degree of random exploration depending on the relative level of environmental richness. Combined with findings from previous studies, our findings indicate the possibility that outcome controllability also influences the exploration–exploitation balance in human reinforcement learning.

Deep Learning–Based Enhanced Optimization for Automated Rice Plant Disease Detection and Classification

ABSTRACTEnsuring global food security necessitates innovative solutions for early detection and precise classification of diseases in staple crops like rice. This study introduces an advanced approach for automated rice plant disease detection and classification by integrating deep learning and metaheuristic optimization techniques. Specifically, a deep dense neural network (DNN) is employed for its capacity to capture intricate patterns in images and extreme learning machine (ELM) for classification. To enhance the optimization process, an innovative variant of the Shuffled Shepherd Optimization (SSO) algorithm, known as Enhanced Artificial Shuffled Shepherd Optimization (EASSO), is introduced. EASSO incorporates adaptive strategies and enhanced exploration–exploitation mechanisms, enabling more efficient convergence during the training of the DNN. The proposed system operates by processing high‐resolution images of rice plants through the DNN, extracting nuanced features indicative of various diseases, including blast, bacterial blight, and brown spots. EASSO optimizes the DNN's parameters, maximizing its accuracy in disease classification. The synergy between DNN and EASSO ensures a robust and adaptive model capable of handling diverse and complex disease patterns. This automated approach significantly reduces the reliance on manual inspection, enabling timely intervention and improving overall agricultural productivity. Experimental results demonstrate the superiority of the DNN‐EASSO framework over traditional methods, showcasing higher accuracy rates and faster convergence. The incorporation of Enhanced Artificial Shuffled Shepherd Optimization enhances the precision and reliability of disease classification, making this integrated system a valuable tool for farmers and agricultural practitioners. This research represents a significant stride toward sustainable agriculture, showcasing the potential of advanced technologies in ensuring food security worldwide.

MISAO: Ultra-Short-Term Photovoltaic Power Forecasting with Multi-Strategy Improved Snow Ablation Optimizer

The increase in installed PV capacity worldwide and the intermittent nature of solar resources highlight the importance of power prediction for grid integration of this technology. Therefore, there is an urgent need for an effective prediction model, but the choice of model hyperparameters greatly affects the prediction performance. In this paper, a multi-strategy improved snowmelt algorithm (MISAO) is proposed for optimizing intrinsic computing-expressive empirical mode decomposition with adaptive noise (ICEEMDAN) and weighted least squares support vector machine for PV power forecasting. Firstly, a cyclic chaotic mapping initialization strategy is used to generate a uniformly distributed high-quality population, which facilitates the algorithm to enter the appropriate search domain quickly. Secondly, the Gaussian diffusion strategy enhances the local exploration ability of the intelligences and extends their search in the solution space, effectively preventing them from falling into local optima. Finally, a stochastic follower search strategy is employed to reserve better candidate solutions for the next iteration, thus achieving a robust exploration–exploitation balance. With these strategies, the optimization performance of MISAO is comprehensively improved. In order to comprehensively evaluate the optimization performance of MISAO, a series of numerical optimization experiments were conducted using IEEE CEC2017 and test sets, and the effectiveness of each improvement strategy was verified. In terms of solution accuracy, convergence speed, robustness, and scalability, MISAO was compared with the basic SAO, various state-of-the-art optimizers, and some recently developed improved algorithms. The results showed that the overall optimization performance of MISAO is excellent, with Friedman average rankings of 1.80 and 1.82 in the two comparison experiments. In most of the test cases, MISAO delivered more accurate and reliable solutions than its competitors. In addition, the altered algorithm was applied to the selection of hyperparameters for the ICEEMDAN-WLSSVM PV prediction model, and seven neural network models, including WLSSVM, ICEEMDAN-WLSSVM, and MISAO-ICEEMDAN-WLSSVM, were used to predict the PV power under three different weather types. The results showed that the models have high prediction accuracy and stability. The MAPE, MAE and RMSE of the proposed model were reduced by at least 25.3%, 17.8% and 13.3%, respectively. This method is useful for predicting the output power, which is conducive to the economic dispatch of the grid and the stable operation of the power system.

An improved Harris hawks optimizer with enhanced logarithmic spiral and dynamic factor and its application for predicting molten iron temperature in the blast furnace

AbstractIn response to the problem of poor search performance and difficulty in escaping from the local optimum in the Harris hawks optimizer, an improved Harris hawks optimizer with enhanced logarithmic spiral and dynamic factor (IHHO‐ELSDF) is proposed in this paper. The enhanced logarithmic spiral mechanism is adopted in the exploration phase, and its main feature is the use of an improved opposite‐learning hybrid logarithmic spiral mechanism to search for more promising regions. The dynamic factor is used to replace the escaping energy to improve the global search capability of the algorithm, and it can better balance exploration and exploitation. In addition, a random distribution strategy is proposed for the exploitation phase to avoid falling into the local optimum. Based on 23 classical test functions, the influence of the distribution probability, the three improved mechanisms, and the exploration–exploitation ratio in IHHO‐ELSDF are analyzed. Subsequently, IHHO‐ELSDF is subjected to a comparative analysis with 17 algorithms on the IEEE CEC2022 benchmark suite. These tests show that IHHO‐ELSDF outperforms most competitors in numerical optimization. Furthermore, to assess its applicability in real‐world problems, IHHO‐ELSDF is employed to optimize parameters in the wavelet neural network used for molten iron temperature prediction. The simulation results based on real production data show that the proposed prediction model achieves a high prediction precision with , , and .

Control over self and others’ face: exploitation and exploration

The face serves as a crucial cue for self-identification, while the sense of agency plays a significant role in determining our influence through actions in the environment. The current study investigates how self-identification through facial recognition may influence the perception of control via motion. We propose that self-identification might engender a belief in having control over one’s own face, leading to a more acute detection and greater emphasis on discrepancies between their actions and the sensory feedback in control judgments. We refer to the condition governed by the belief in having control as the exploitation mode. Conversely, when manipulating another individual’s face, the belief in personal control is absent. In such cases, individuals are likely to rely on the regularity between actions and sensory input for control judgments, exhibiting behaviors that are exploratory in nature to glean such information. This condition is termed the explorative mode. The study utilized a face-motion mixing paradigm, employing a deep generative model to enable participants to interact with either their own or another person’s face through facial and head movements. During the experiment, participants observed either their own face or someone else’s face (self-face vs. other-face) on the screen. The motion of the face was driven either purely by their own facial and head motion or by an average of the participant’s and the experimenter’s motion (full control vs. partial control). The results showed that participants reported a higher sense of agency over the other-face than the self-face, while their self-identification rating was significantly higher for the self-face. More importantly, controlling someone else’s face resulted in more movement diversity than controlling one’s own face. These findings support our exploration–exploitation theory: When participants had a strong belief in control triggered by the self-face, they became highly sensitive to any sensorimotor prediction errors, leading to a lower sense of agency. In contrast, when the belief of control was absent, the exploration mode triggered more explorative behaviors, allowing participants to efficiently gather information to establish a sense of agency.

Exploration-exploitation model of moth-inspired olfactory navigation.

Navigation of male moths towards females during the mating search offers a unique perspective on the exploration-exploitation (EE) model in decision-making. This study uses the EE model to explain male moth pheromone-driven flight paths. Wind tunnel measurements and three-dimensional tracking using infrared cameras have been leveraged to gain insights into male moth behaviour. During the experiments in the wind tunnel, disturbance to the airflow has been added and the effect of increased fluctuations on moth flights has been analysed, in the context of the proposed EE model. The exploration and exploitation phases are separated using a genetic algorithm to the experimentally obtained dataset of moth three-dimensional trajectories. First, the exploration-to-exploitation rate (EER) increases with distance from the source of the female pheromone is demonstrated, which can be explained in the context of the EE model. Furthermore, our findings reveal a compelling relationship between EER and increased flow fluctuations near the pheromone source. Using an olfactory navigation simulation and our moth-inspired navigation model, the phenomenon where male moths exhibit an enhanced EER as turbulence levels increase is explained. This research extends our understanding of optimal navigation strategies based on general biological EE models and supports the development of bioinspired navigation algorithms.

Examining the interplay between managerial ties, dynamic capabilities and innovation climate in driving balanced and combined ambidextrous innovation

PurposeThis paper aims to adopt the dynamic capabilities view to investigate the relationship between managerial ties (i.e. business and political ties), dynamic capabilities and innovation climate on ambidextrous innovation (i.e. balanced and combined ambidextrous innovation), in the renewable and sustainable energy context. It also examines the mediating effects of dynamic capabilities between managerial ties and ambidextrous innovation (i.e. balanced and combined ambidextrous innovation), and moderating effects between dynamic capabilities and ambidextrous innovation relationships.Design/methodology/approachMultilevel analyses conducted using AMOS 26 on 288 employees working in 47 UAE energy firms.FindingsResults found that business ties influences balanced and combined ambidextrous innovation indirectly, whereas political ties only impact combined ambidextrous innovation indirectly through dynamic capabilities. Dynamic capabilities insignificantly mediated managerial ties–ambidextrous innovation and political ties–balanced ambidextrous innovation relationships, with stronger indirect effect on combined than on the balanced dimension. Findings also indicate that innovation climate is the crucial moderator between dynamic compatibilities and ambidextrous innovation, as well as balanced and combined ambidextrous innovation, with stronger effect on balanced dimension than the combined.Originality/valueThis study addresses recent calls by highlighting the role of dynamic capabilities, an important yet underexplored organizational capabilities in the innovation and ambidexterity literature. Also, this study advances insight into how balanced and combined exploration–exploitation innovation and dynamic capabilities are connected and enhances the understanding into how organizational factors stimulate dynamic capabilities leading to superior innovation.

An Enhanced Multiple Unmanned Aerial Vehicle Swarm Formation Control Using a Novel Fractional Swarming Strategy Approach

This paper addresses the enhancement of multiple Unmanned Aerial Vehicle (UAV) swarm formation control in challenging terrains through the novel fractional memetic computing approach known as fractional-order velocity-pausing particle swarm optimization (FO-VPPSO). Existing particle swarm optimization (PSO) algorithms often suffer from premature convergence and an imbalanced exploration–exploitation trade-off, which limits their effectiveness in complex optimization problems such as UAV swarm control in rugged terrains. To overcome these limitations, FO-VPPSO introduces an adaptive fractional order β and a velocity pausing mechanism, which collectively enhance the algorithm’s adaptability and robustness. This study leverages the advantages of a meta-heuristic computing approach; specifically, fractional-order velocity-pausing particle swarm optimization is utilized to optimize the flying path length, mitigate the mountain terrain costs, and prevent collisions within the UAV swarm. Leveraging fractional-order dynamics, the proposed hybrid algorithm exhibits accelerated convergence rates and improved solution optimality compared to traditional PSO methods. The methodology involves integrating terrain considerations and diverse UAV control parameters. Simulations under varying conditions, including complex terrains and dynamic threats, substantiate the effectiveness of the approach, resulting in superior fitness functions for multi-UAV swarms. To validate the performance and efficiency of the proposed optimizer, it was also applied to 13 benchmark functions, including uni- and multimodal functions in terms of the mean average fitness value over 100 independent trials, and furthermore, an improvement at percentages of 29.05% and 2.26% is also obtained against PSO and VPPSO in the case of the minimum flight length, as well as 16.46% and 1.60% in mountain terrain costs and 55.88% and 31.63% in collision avoidance. This study contributes valuable insights to the optimization challenges in UAV swarm-formation control, particularly in demanding terrains. The FO-VPPSO algorithm showcases potential advancements in swarm intelligence for real-world applications.

Exploration–Exploitation Mechanisms in Recurrent Neural Networks and Human Learners in Restless Bandit Problems

A key feature of animal and human decision-making is to balance the exploration of unknown options for information gain (directed exploration) versus selecting known options for immediate reward (exploitation), which is often examined using restless bandit tasks. Recurrent neural network models (RNNs) have recently gained traction in both human and systems neuroscience work on reinforcement learning, due to their ability to show meta-learning of task domains. Here we comprehensively compared the performance of a range of RNN architectures as well as human learners on restless four-armed bandit problems. The best-performing architecture (LSTM network with computation noise) exhibited human-level performance. Computational modeling of behavior first revealed that both human and RNN behavioral data contain signatures of higher-order perseveration, i.e., perseveration beyond the last trial, but this effect was more pronounced in RNNs. In contrast, human learners, but not RNNs, exhibited a positive effect of uncertainty on choice probability (directed exploration). RNN hidden unit dynamics revealed that exploratory choices were associated with a disruption of choice predictive signals during states of low state value, resembling a win-stay-loose-shift strategy, and resonating with previous single unit recording findings in monkey prefrontal cortex. Our results highlight both similarities and differences between exploration behavior as it emerges in meta-learning RNNs, and computational mechanisms identified in cognitive and systems neuroscience work.

Slender loris optimization algorithm for solving optimization problems

Over the years, numerous optimization problems have been addressed utilizing meta-heuristic algorithms. Continuing initiatives have always been to create and develop new, practical algorithms. This work proposes a novel meta-heuristic approach employing the slender Loris optimization algorithm (SLOA), miming slender Loris behavior. The behavior includes foraging, hunting, migration and communication with each other. The ultimate goal of the devised algorithm is to replicate the food-foraging behaviour of Slender Loris (SL) and the quick movement of SL when threatened (i.e.) their escape from predators and also mathematically modelled the special communication techniques of SL using their urine scent smell. SLOA modelled SL’s slow food foraging behaviour as the exploitation phase, and moving between the tree and escaping from a predator is modelled as the exploration phase. The Eyesight of slender Loris plays a vital role in food foraging during nighttime in dim light. The operator’s Eyesight is modelled based on the angle of inclination of SL. The urine scent intensity is used here to be instrumental in preventing already exploited territory activities, which improves algorithm performance. The suggested algorithm is assessed and tested against nineteen benchmark test operations and evaluated for effectiveness with standard widely recognized meta-heuristics algorithms. The result shows SLOA performing better and achieving near-optimal solutions and dominance in exploration–exploitation balance in most cases than the existing state-of-the-art algorithms.

The Long March: The quest for valid text-based indicators of exploration and exploitation

Since March outlined the importance of balancing exploration and exploitation in organizational learning, the exploration–exploitation paradigm has received substantial attention in the management literature. Recent studies have used computer-aided text analysis to construct measures of firms’ inclination towards exploration or exploitation, using the original set of keywords proposed by James G. March. We propose a structured series of tests to assess the validity of computer-aided text analysis-based measures and demonstrate that the approach used in prior studies is unlikely to deliver valid indicators. We demonstrate that an alternative approach, which relies on a larger library of keywords, including synonyms of the March keywords and selects only those keywords that are informative and that pass validity tests, delivers valid computer-aided text analysis indicators – both for unstructured (news articles) and structured text-bases (annual reports). Our study contributes to the literature on construct validity and has broader implications for the development of computer-aided text analysis-based indicators in strategy and organization research.

Dealing with uncertainty: Balancing exploration and exploitation in deep recurrent reinforcement learning

Incomplete knowledge of the environment leads an agent to make decisions under uncertainty. One of the major dilemmas in Reinforcement Learning (RL) where an autonomous agent has to balance two contrasting needs in making its decisions is: exploiting the current knowledge of the environment to maximize the cumulative reward as well as exploring actions that allow improving the knowledge of the environment, hopefully leading to higher reward values (exploration–exploitation trade-off). Concurrently, another relevant issue regards the full observability of the states, which may not be assumed in all applications. For instance, when 2D images are considered as input in an RL approach used for finding the best actions within a 3D simulation environment. In this work, we address these issues by deploying and testing several techniques to balance exploration and exploitation trade-off on partially observable systems for predicting steering wheels in autonomous driving scenarios. More precisely, the final aim is to investigate the effects of using both adaptive and deterministic exploration strategies coupled with a Deep Recurrent Q-Network. Additionally, we adapted and evaluated the impact of a modified quadratic loss function to improve the learning phase of the underlying Convolutional Recurrent Neural Network. We show that adaptive methods better approximate the trade-off between exploration and exploitation and, in general, Softmax and Max-Boltzmann strategies outperform ϵ-greedy techniques.

A closer look at reinforcement learning-based automatic speech recognition

Reinforcement learning (RL) has demonstrated effectiveness in improving model performance and robustness for automatic speech recognition (ASR) tasks. Researchers have employed RL-based training strategies to enhance performance beyond conventional supervised or semi-supervised learning. However, existing approaches treat RL as a supplementary tool, leaving the untapped potential of RL training largely unexplored. In this paper, we formulate a novel pure RL setting where an ASR model is trained exclusively through RL via human feedback metrics, e.g., word error rate (WER) or binary reward. This approach promises to significantly simplify the annotation process if we could replace the conventional onerous annotation with a single numeric value in a human–computer interaction (HCI) way. Our experiments demonstrate the feasibility of this new setting and also identify two main inherent issues in conventional RL-based ASR training that may lead to performance degradation: (1) the mismatch issue between the action and reward has been commonly overlooked in Connectionist Temporal Classification (CTC) based models, which is attributed to the inherent CTC alignment mapping issue; (2) the classic exploration–exploitation trade-off still exists in the sampling stage of RL-based ASR, and finding the balance between them becomes a challenge. To address these issues, we first propose a new RL-based approach named CTC-aligned Policy Gradient (CTC-PG), which provides a unified formulation for different sampling strategies and alleviates the mismatch issue of action and reward in CTC-based models. Moreover, we propose Focal Sampling to balance the trade-off between exploration and exploitation with a flexible temperature parameter. Experiment results on LibriSpeech dataset showcase the effectiveness and robustness of our methods by harnessing the full potential of RL in training ASR models.

Dynamic Exploration–Exploitation Pareto Approach for high-dimensional expensive black-box optimization

Surrogate Optimization (SO) plays a vital role in optimizing performance parameters for computationally expensive simulations. However, SO encounters significant challenges in high-dimensional spaces due to the curse of dimensionality, hampering effective point sampling around global optima. In this paper, we introduce “Dynamic Exploration–Exploitation Pareto Approach (DEEPA),” a novel SO method that combines Pareto sampling with a dynamic discretization schema to optimize high-dimensional black-box functions. Unlike traditional SO methods that heavily rely on specific surrogate models, Pareto sampling offers a more adaptable approach. Improvement-based acquisition functions, frequently employed in black-box optimization, are sensitive to model accuracy and tend to prioritize exploitation, potentially missing valuable regions of interest in complex landscapes. Furthermore, they can encounter challenges when dealing with high-dimensional problems due to the curse of dimensionality. DEEPA leverages dynamic coordinate importance to generate samples effectively, providing a solution for addressing high-dimensionality and complex functions. We employ feature selection strategies to assign importance probabilities to perturb each coordinate, demonstrating the impact of importance-based perturbation on convergence to a near-optimal region. We showcase DEEPA’s versatility in fixed-batch evaluation environments using complex global optimization test problems with various topological properties. We compare DEEPA’s performance with state-of-the-art black-box optimization algorithms, and our experimental results demonstrate DEEPA’s superior performance, particularly in complex problems with multiple local minima.

MSAO: A multi-strategy boosted snow ablation optimizer for global optimization and real-world engineering applications

Snow Ablation Optimizer (SAO) is a cutting-edge nature-inspired meta-heuristic technique that mimics the sublimation and melting processes of snow in its quest for optimal solution to complex problems. While SAO has demonstrated competitive performance in comparison to classical algorithms in early research, it still exhibits certain limitations including low convergence accuracy, a lack of population diversity, and premature convergence, particularly when addressing high-dimensional intricate challenges. To mitigate the above-mentioned adverse factors, this paper introduces a novel variant of SAO with featuring four enhancement strategies collectively referred as MSAO. Firstly, the good point set initialization strategy is employed to generate a uniformly distributed high-quality population, which facilitates the algorithm to enter the appropriate search domain rapidly. Secondly, the greedy selection method is adopted to reserve better candidate solutions for the next iteration, thus striking a robust exploration–exploitation balance. Then, the Differential Evolution (DE) scheme is introduced to expand the search range and enhance the exploitation capability of the algorithm for higher convergence accuracy. Finally, to reduce the risk of falling into local optima, a Dynamic Lens Opposition-Based Learning (DLOBL) strategy is developed to operate on the current optimal solution dimension by dimension. With the blessing of these strategies, the optimization performance of MSAO is comprehensively improved. To comprehensively evaluate the optimization performance of MSAO, a series of numerical optimization experiments are conducted using the IEEE CEC2017 & CEC2022 test sets. In the IEEE CEC2017 experiments, the optimal crossover probability CR=0.8 is determined and the effectiveness of each improvement strategy is ablatively verified. MSAO is compared with the basic SAO, various state-of-the-art optimizers, and CEC2017 champion algorithms in terms of solution accuracy, convergence speed, robustness, and scalability. In the IEEE CEC2022 experiments, MSAO is compared with some recently developed improved algorithms to further validate its superiority. The results demonstrate that MSAO has excellent overall optimization performance, with the smallest Friedman mean rankings of 1.66 and 1.25 on both test suites, respectively. In the majority of test cases, MSAO can provide more accurate and reliable solutions than other competitors. Furthermore, six realistic constrained engineering design challenges and one photovoltaic model parameter estimation issue are employed to demonstrate the practicality of MSAO. Our findings suggest that MSAO has excellent optimization capacity and broad application potential.

Innovative search and imitation heuristics: an agent-based simulation study

AbstractProminent research in strategic imitation, exploration, exploitation, and organizational learning identifies imitation as a less costly alternative to experimentation. Yet, its role in the exploration–exploitation dilemma remains underexplored in the literature. This study employs an agent-based model to examine how two distinct agent types—those who imitate and those who experiment—interact and influence each other. The model incorporates the concept of “satisficing” derived from the behavioral theory of the firm, along with insights from research on imitative heuristics. The findings reveal that overcrowding affects both agent types negatively. Imitators suffer from diminished performance due to intensified competition, which increases as more imitators join the system. Meanwhile, explorers are hindered in their attempts at radical innovation due to the presence of other explorers and clusters of imitators. This paper contributes to the field as the first to model individual agents as ‘satisficers’ within a competitive exploration–exploitation framework. By incorporating imitation, it provides novel insights into the dynamics of organizational learning and strategic decision-making.

Groundwater Exploration in Carbonate Reservoirs Using Borehole Investigations: A Case Study from South Dobrogea, Romania

The Late Jurassic–Early Cretaceous (J3–K1) transboundary aquifer is the most important groundwater body in southern–southeastern Romania, shared with Bulgaria and hosted in karstic–fractured carbonates. We conducted an integrated evaluation of this aquifer by analyzing three 700 m deep groundwater exploration–exploitation boreholes, which intercepted it in the Cernavodă area (South Dobrogea region). The evaluation was based on geophysical wireline logging, drilling information, and borehole production tests. A K-means clustering of the logging data was performed for lithology typing, formation boundaries identification, and the delineation of probable producing intervals associated with secondary porosity development. Petrophysical interpretation was carried out via depth-constrained (zonal) inversion, using multimineral models, the estimated formation boundaries, and variable uncertainties for the main input logs. The optimal interpretation models were correlated with borehole testing results to gain insight into the hydrogeological properties of the aquifer complex. The fractured–vuggy interval with the highest water-producing potential was identified in the lower section of the J3-age Rasova Formation (639–700 m depth), comprising mainly undolomitized limestones. A southeast-to-northwest trend of increasing productivity of the boreholes, correlated with an increasing lateral dolomitization intensity within the Rasova Formation, suggests a highly heterogeneous character of the aquifer. The differences in productivity are due not only to local porosity variations but also to various degrees of pore space connectivity that are related to the amount of fracturing or karstification. The novel findings of this study have important practical implications for the optimal placement, design, and drilling program of future groundwater exploitation boreholes in the Cernavodă area and neighboring sectors.

A novel two-stage dynamic pricing model for logistics planning using an exploration–exploitation framework: A multi-armed bandit problem

Dynamic pricing is utilized as a tool for determining pricing strategies in different sectors, such as the food industries, airline companies, and transportation services. Dynamic pricing aims to maximize revenue based on learning the price-demand relationship. It is imperative to perform price change testing within a short time frame since rapid changes can negatively impact the market. Therefore, there are two conflicting objectives: maximizing revenue and learning the demand function, which is called the exploration–exploitation concept. In this study, we propose an exploration–exploitation framework, focused on the domestic delivery service of Tipax Express Pars Company (TEPC), to determine the optimal selling price. In the exploration process, the initial demand function is proposed based on previous sales data using weighted linear regression. Then, the selling price and delivery lead-time parameters are incorporated into the demand function hypotheses. After that, the value of the optimal demand function parameters is determined based on the Z-Price Changes Algorithm (ZPCA). In the exploitation process, price optimization is designed as a Multi-armed Bandit (MAB) problem and solved using (i) Upper Confidence Bound Method (UCBM) (i.e., deterministic approach), and (ii) Thompson Sampling Method (TSM) (i.e., stochastic approach). The proposed exploration–exploitation framework is evaluated in 361 days, of which the exploration and exploitation durations are 31 and 330 days, respectively. In this study, 10,000 demand functions are investigated, which leads to the complexity of the developed model. The cross-validation approach, based on Mean Squared Error (MSE), is developed to decrease the complexity of the proposed model. A simulation study of 500 simulations demonstrates that TSM is more useful than UCBM because its solution is more robust. Furthermore, the results demonstrate that TEPC can gain maximum total expected revenue, maximum total number of selections during the exploitation stage, and maximum actual demand using a gaining market share strategy.