Myopic Strategy Research Articles

Isoperimetric Control of an Expanding Set

Consider a contaminated region in a 2-dimensional plane that is expanding with unit normal speed. Our goal is to apply a control function (e.g. pesticide) along the boundary of this region to reduce the contaminated area to zero in the shortest possible time, while consistently clearing a fixed amount of land per unit time. It has been proven that when the initial contamination forms a convex set, an optimal eradication strategy requires applying pesticide to the boundary segments with the highest curvature at each moment. This is equivalent to adopting a myopic strategy that minimizes the perimeter of the contaminated region at each time step. This result provided insight for developing numerical methods to compute approximations of these optimal strategies for convex sets. Namely, our simulation consists in alternating between computing the isoperimetric (perimeter-minimizing) profile with a convex constraint and then dilating the resulting set by a radius equal to the time mesh. This process is repeated until the shape reaches a circle, which can then be trivially scaled down to a point. The isoperimetric profile is obtained through morphological opening, an erosion-dilation operator commonly used in image processing for noise removal. A major open problem remains: a full characterization of optimal eradication strategies for initial contaminations of general shapes. We have applied the above numerical scheme with non-convex constraints to gain some intuition in this broader context. This may entail replacing the morphological opening operator with a more general algorithm relying on the medial axis of a shape. It turns out that, in some configurations, it is more effective to split the contamination into several components and shrink them separately.

Alleviating bus bunching via modular vehicles

The notorious phenomenon of bus bunching prevailing in uncontrolled bus systems produces irregular headways and downgrades the level of service by increasing passengers’ expected waiting time. Modular autonomous vehicles (MAVs), due to their ability to split and merge en route, have the potential to help both late and early buses recover from schedule deviation while providing continuous service. In this paper, we propose a novel bus bunching alleviation strategy for MAV-aided transit systems. We first consider a soft vehicle capacity constraint and establish a continuum approximation (CA) model (Model I) to capture the system dynamics intertwined with the MAV splitting and merging operations, and then establish an infinite-horizon stochastic optimization model to determine the optimal splitting and merging strategy. To capture the reality that passengers may fail to board an overcrowded bus, we propose a second model (Model II) by extending Model I to accommodate a hard vehicle capacity constraint. Based on the characteristics of the problem, we develop a customized deep Q-network (DQN) algorithm with multiple relay buffers and a penalized ruin state applicable for both models to optimize the strategy for each MAV. Numerical results show that the strategy obtained via the DQN algorithm is an effective bunch-proof strategy and has a better performance than the myopic strategy for MAV-aided systems and the two-way-looking strategy for conventional bus systems. Sensitivity analyses are also conducted to examine the effectiveness and benefits of the proposed strategy across different operation scenarios.

Technician routing and scheduling with employees’ learning through implicit cross-training strategy

With record high talent shortages and skill mismatches around the world, this paper investigates a variant of multi-period dynamic technician and routing problem that can be modeled as a Markov decision process. To deal with the double tradeoffs between the routing and service time costs, as well as the current and future costs, we propose an approximate dynamic programming (ADP)-based cost function approximation (CFA) algorithm — the implicit cross-training strategy (ICT). A two-phase routing and scheduling heuristic is developed to account for both employees’ learning and future information, and to facilitate an efficient implementation of CFA. Extensive computational results show that ICT can provide a better solution in the current decision with a global view in comparison with the myopic strategy. In depth analysis demonstrates that ICT trains the workforce with more balanced skillsets and workloads, which ensures the flexibility of the workforce and helps buffer against the future uncertainties with substantial routing cost savings. Additionally, ICT has much more advantages in large-scale problems with more diversified service requests and randomly distributed customers.

Navigating the green shipping: Stochastic hydrogen hub deployment in inland waterways

The maritime industry is shifting toward sustainability to help combat climate change, given the sector’s significant contribution to global greenhouse gas emissions. This study focuses on the strategic integration of hydrogen fuel—a zero-emission alternative—into inland waterway transport systems. We recognize the operational characteristics of inland shipping networks and the complexities of two-stage financial planning. Consequently, we propose a two-stage stochastic framework for the deployment of hydrogen refueling hubs, taking into account financial uncertainties in the future. This approach contrasts with a myopic strategy and a two-stage deterministic framework, addressing both immediate financial constraints and future uncertainties. We focus on the Yangtze River’s shipping network as a case study, and our analysis validates the superior performance of our proposed two-stage stochastic strategy. This suggests its potential to accommodate volatile future financial uncertainties while ensuring the economic and operational viability of hydrogen integration.

Optimal strategy for Bayesian two-armed bandit problem with an arched reward function

In the classical two-armed bandit (TAB) problems, the goal of maximizing the total return specifies the optimal decision rule and fosters the development of abundant practicable methods, such as $ \epsilon $-greedy and upper confidence bound for TAB procedures. However, the existence of an arched reward function for the total return breaks the previously updated algorithms, like traditional myopic strategy, and prompts the generation of novel computation and theory for maximizing its expectation. Here we develop a special class of Bayesian two-armed bandit problems by imposing a prior probability on which arm has a greater expectation of the returns and propose a myopic strategy to specify the decision rule. We prove that the proposed myopic strategy is optimal under the structure of the arched reward function by exploring the dynamic programming principle. This article also demonstrates that arched functions are extensions of the monotone functions including the classical linear functions. Meanwhile, we establish a corresponding law of large numbers for our myopic strategy. Simulation studies pose supportive evidence that the newly proposed strategy performs well.

Rolling horizon strategies for a dynamic and stochastic ridesharing problem with rematches

We study a dynamic matching and rematching problem with applications in ridesharing systems. Transportation requests for riders and passengers arrive dynamically and are represented as nodes of a bipartite graph, connected by edges that correspond to compatible requests. Matching requests by selecting the corresponding edge earns a profit. We further allow for unmatching previously matched requests. While this increases the system’s flexibility to adjust to new matching opportunities, unmatching may degrade customer experience and therefore implies penalty costs. We evaluate myopic and stochastic multi-period mixed-integer programming models in a rolling horizon framework. All models are compared against a static model that has perfect knowledge about future requests, using a novel and extensive benchmark set of realistic instances. Our results demonstrate the value of being able to unmatch, as well as the benefits of the stochastic strategies over the myopic strategy.

The Perils of Trial-and-Error Reward Design: Misdesign through Overfitting and Invalid Task Specifications

In reinforcement learning (RL), a reward function that aligns exactly with a task's true performance metric is often necessarily sparse. For example, a true task metric might encode a reward of 1 upon success and 0 otherwise. The sparsity of these true task metrics can make them hard to learn from, so in practice they are often replaced with alternative dense reward functions. These dense reward functions are typically designed by experts through an ad hoc process of trial and error. In this process, experts manually search for a reward function that improves performance with respect to the task metric while also enabling an RL algorithm to learn faster. This process raises the question of whether the same reward function is optimal for all algorithms, i.e., whether the reward function can be overfit to a particular algorithm. In this paper, we study the consequences of this wide yet unexamined practice of trial-and-error reward design. We first conduct computational experiments that confirm that reward functions can be overfit to learning algorithms and their hyperparameters. We then conduct a controlled observation study which emulates expert practitioners' typical experiences of reward design, in which we similarly find evidence of reward function overfitting. We also find that experts' typical approach to reward design---of adopting a myopic strategy and weighing the relative goodness of each state-action pair---leads to misdesign through invalid task specifications, since RL algorithms use cumulative reward rather than rewards for individual state-action pairs as an optimization target. Code, data: github.com/serenabooth/reward-design-perils

Redesign of sharing charging system for electric vehicles using blockchain technology

Electric vehicles (EVs) have great potential in improving environmental sustainability. However, current charging infrastructures could not meet the heightening demand due to the rapid expansion of EVs. Sharing private charge posts with others during idle time, as a win–win solution for charge post owners and non-owner EV drivers, is a proactive response to the dilemma. To build a trusted environment for the peer-to-peer charge post rental service, we explore a blockchain-enabled sharing charging system with redesigned operating procedures and a tailored non-myopic charge post match strategy. The new system can leverage the advantages of blockchain in privacy-preserving and misbehavior-auditing while fully utilizing shared charging facilities. The proposed system model is validated on Ethereum, and test results suggest that the redesigned system can provide high quality services with affordable computational resource consumption. Moreover, the results also report that, compared to the myopic strategy, the non-myopic strategy can improve the overall matching successful rate of charging requests by considering the potential impact of each assignment, especially when charge post resources are scarce.

A confirmation of a conjecture on Feldman’s two-armed bandit problem

AbstractThe myopic strategy is one of the most important strategies when studying bandit problems. In 2018, Nouiehed and Ross put forward a conjecture about Feldman’s bandit problem (J. Appl. Prob. (2018) 55, 318–324). They proposed that for Bernoulli two-armed bandit problems, the myopic strategy stochastically maximizes the number of wins. In this paper we consider the two-armed bandit problem with more general distributions and utility functions. We confirm this conjecture by proving a stronger result: if the agent playing the bandit has a general utility function, the myopic strategy is still optimal if and only if this utility function satisfies reasonable conditions.

Sequential claim games

AbstractWe consider the estate division or bankruptcy problem and assume that players sequentially put claims on the estate. Each part of the estate is then divided proportionally with respect to the number of claims on it. We focus on myopic play: players first claim the hitherto least claimed parts, and on subgame perfect equilibria. Our main result is that myopic strategies constitute a subgame perfect equilibrium if punishments for deviators are included.

Improving throughput in SWIPT-based wireless multirelay networks with relay selection and rateless codes

This paper studies a dual-hop Simultaneous Wireless Information and Power Transfer (SWIPT)-based multi-relay network with a direct link. To achieve high throughput in the network, a novel protocol is first developed, in which the network can switch between a direct transmission mode and a Single-Relay-Selection-based Cooperative Transmission (SRS-CT) mode that employs dynamic decode-and-forward relaying accomplished with Rateless Codes (RCs). Then, under this protocol, an optimization problem is formulated to jointly optimize the network operation mode and the resource allocation in the SRS-CT mode. The formulated problem is difficult to solve because not only does the noncausal Channel State Information (CSI) cause the problem to be stochastic, but also the energy state evolution at each relay is complicated by network operation mode decision and resource allocation. Assuming that noncausal CSI is available, the stochastic optimization issue is first to be addressed by solving an involved deterministic optimization problem via dynamic programming, where the complicated energy state evolution issue is addressed by a layered optimization method. Then, based on a finite-state Markov channel model and assuming that CSI statistical properties are known, the stochastic optimization problem is solved by extending the result derived for the noncausal CSI case to the causal CSI case. Finally, a myopic strategy is proposed to achieve a tradeoff between complexity and performance without the knowledge of CSI statistical properties. The simulation results verify that our proposed SRS-and-RC-based design can achieve a maximum of approximately 40% throughput gain over a simple SRS-and-RC-based baseline scheme in SWIPT-based multi-relay networks.

Can Global Strategy Outperform Myopic Strategy in Bayesian Sequential Design?

Can Global Strategy Outperform Myopic Strategy in Bayesian Sequential Design?

New technology product introduction strategy with considerations for consumer-targeted policy intervention and new market entrant

The emergence of new technology products brings systematic changes to the market, and appropriate responses to such changes are essential for the survival and success of firms. This study investigated producers' optimal new product introduction (and positioning) strategies in various situations that require different sets of decisions. Specifically, we considered the diffusion of hybrid electric vehicles and battery electric vehicles (BEVs) in the South Korean automotive market. A model incorporating the strategic, forward-looking behaviors of both producers and consumers was used to derive producer's optimal new product introduction plan based on long-term market forecasting. Then, we conducted simulation analyses to investigate the benefit of forward-looking planning and how optimal strategies change depending on the policy interventions and market entrance of the new producer. The results show that a forward-looking strategy could improve the market performance of new products by up to 40 % compared to the myopic strategy, and the plan adjustment of the incumbent in response to competition could reduce competition-related losses by up to 9 %. Finally, stronger policy intervention and new market competition will motivate producers to introduce more, less-expensive BEVs in their product portfolio. Such responses significantly expand the BEV market, which is also a socially desirable outcome.

Informational Cascades With Nonmyopic Agents

We consider an environmentwhere players need to decide whether to buy a certain product (or adopt a technology) or not. The product is either good or bad, but its true value is unknown to the players. Instead, each player has her own private information on its quality. Each player can observe the previous actions of other players and estimate the quality of the product. A classic result in the literature shows that in similar settings, informational cascades occur, where learning stops for the whole network and players repeat the actions of their predecessors. In contrast to this literature, in this paper, players get more than one opportunity to act. In each turn, a player is chosen uniformly at random from all the players and can decide to buy the product and leave the market or wait. Her utility is the total expected discounted reward, and thus, myopic strategies may not constitute equilibria. We provide a characterization of perfect Bayesian equilibria (PBEs) with forward-looking strategies through a fixed-point equation of dimensionality that grows only quadratically with the number of players. Using this tractable fixed-point equation, we show the existence of a PBE and characterize PBEs with threshold strategies. Based on this characterization, we study informational cascades in two regimes. First, we show that for a discount factor <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\delta$</tex-math></inline-formula> strictly smaller than 1, informational cascades happen with high probability as the number of players <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$N$</tex-math></inline-formula> increases. Furthermore, only a small portion of the total information in the system is revealed before a cascade occurs. Second, and more surprisingly, we show that for a fixed <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$N$</tex-math></inline-formula> , and for a sufficiently large <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\delta &lt; 1$</tex-math></inline-formula> , when the product is bad, there exists an equilibrium where an informational cascade can happen only after at least half of the players revealed their private information, and consequently, the probability for a “bad cascade” where all the players buy the product vanishes exponentially with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$N$</tex-math></inline-formula> . Finally, when <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\delta =1$</tex-math></inline-formula> and the product is bad, there exists an equilibrium where informational cascades do not happen at all.

Impact of strategy conformity on vaccination behaviors

In most of the studies focusing on the conformity of voluntary vaccination decisions, the conformity was always directly modeled as a conformity-driven strategy-updating rule. However, the utility of an individual can also be influenced by the group identity or discrimination behaviors associated with strategy conformity in realities. Thus, a novel utility model of the vaccination game is first formulated in which the influence of strategy conformity is considered. Then, we use the spatial evolutionary game theory to study the dynamics of individual vaccination strategies under the influence of strategy conformity on the scale-free network. The results show that moderate strategy conformity and a high herd immunity threshold have a significant positive effect on vaccination behaviors when the initial vaccination fraction is low. Moreover, for a high initial vaccination fraction, the strong strategy conformity and high herd immunity threshold are more conducive to encourage vaccination behavior. To analyze the model sensitivity, experiments are conducted in the small world network and square lattice network. In addition, we performed the sensitivity analysis on vaccination effectiveness. Finally, the generality of strategy conformity effect is investigated when the myopic strategy updating rule is adopted in the whole population. The result shows that vaccination behaviors can also be promoted under the condition of moderate strategy conformity and low initial vaccination fraction.

Purposeful Disownership: Paying Contract Manufacturers to Reduce Ownership and Termination Cost

Firms often persist with failing products due to excessive organizational ownership and relational commitments to suppliers. We propose that firms make strategic use of contract manufacturing to purposefully reduce ownership and termination costs. Using detailed data on 249,705 products (or designs) in the fashion industry, we show that firms are more likely to contract manufacture new products that have a high risk of commercial failure even when they can be readily produced in-house. In exchange, firms pay production costs that are 10 to 16 percent higher. Organizational hierarchy, which reduces ownership, significantly moderates the increase in contract manufacturing. Our findings suggest that firms actively manage escalating commitment and path-dependence through make vs. buy decisions and revise the prevailing view that contract manufacturing is a myopic strategy that trades learning opportunities with short-term cost savings. We show that our theory generalizes and successfully explains decisions between direct capital investment and rentals by US public firms.

Polynomial affine approach to HARA utility maximization with applications to OrnsteinUhlenbeck [formula omitted] models.

This paper designs a numerical methodology, named PAMH, to approximate an investor’s optimal portfolio strategy in the contexts of expected utility theory (EUT) and mean-variance theory (MVT). Thanks to the use of hyperbolic absolute risk aversion utilities (HARA), the approach produces optimal solutions for decreasing relative risk aversion (DRRA) investors, as well as for increasing relative risk aversion (IRRA) agents. The accuracy and efficiency of the approximation is examined in a comparison to known closed-form solutions for a one dimensional (n=1) geometric Brownian motion with a CIR stochastic volatility model (i.e. GBM 1/2 or Heston model), and a high dimensional (up to n=35) stochastic covariance model. The former confirms the method works even when the theoretical candidate is not well-defined, while the latter illustrates low errors (up to 8% in certainty equivalent rate (CER)) and feasible computational time (less than one hour in a PC).Given the potential of this method, we investigate a relevant practical setting with no closed-form solution, namely when assets follow an OrnsteinUhlenbeck 4/2 stochastic volatility (SV, i.e. OU 4/2) model. We conduct sensitivity analyses of the optimal strategies for DRRA and IRRA investors with respect to key parameters; (e.g. risk aversion, minimum capital guarantee and 4/2’s parameters). In particular, the efficient frontier for the IRRA case is presented. A comparison to important sub-optimal strategies in terms of CER is performed, indicating low CER performances due to ignorance of stochastic volatility for CRRA investors, i.e. a myopic strategy would be even better than ignoring SV. The analyses highlight the importance of efficient and precise numerical methods to obtain substantially higher CERs.

Performance persistence and optimal asset allocation strategies

This study explores whether optimal asset allocation strategies, defined by permutations and combinations of different predictor variables, produce consistently superior performance for investors. We extend the literature by exploring whether such strategies benefit investors over the entire investment period or whether investors are forced to switch among alternative strategies over time. As benchmarks, we employ the 1/N (equally weighted) and the myopic (no predictability) strategies. Persistence tests suggest that no single optimal strategy outperforms the remaining optimal and benchmark strategies over the entire sample. However, in two out of three subsample periods, some optimal strategies persistently outperform the benchmarks.

Early classification of time series

An increasing number of applications require to recognize the class of an incoming time series as quickly as possible without unduly compromising the accuracy of the prediction. In this paper, we put forward a new optimization criterion which takes into account both the cost of misclassification and the cost of delaying the decision. Based on this optimization criterion, we derived a family of non-myopic algorithms which try to anticipate the expected future gain in information in balance with the cost of waiting. In one class of algorithms, unsupervised-based, the expectations use the clustering of time series, while in a second class, supervised-based, time series are grouped according to the confidence level of the classifier used to label them. Extensive experiments carried out on real datasets using a large range of delay cost functions show that the presented algorithms are able to solve the earliness vs. accuracy trade-off, with the supervised partition based approaches faring better than the unsupervised partition based ones. In addition, all these methods perform better in a wide variety of conditions than a state of the art method based on a myopic strategy which is recognized as being very competitive. Furthermore, our experiments show that the non-myopic feature of the proposed approaches explains in large part the obtained performances.

Highly cooperative individuals’ clustering property in myopic strategy groups

Highly cooperative individuals’ clustering property in myopic strategy groups