Follower Action Research Articles

Signaling-based Robust Incentive Designs with Randomized Monitoring

Incentive design problems entail hierarchical decision-making where a leader crafts a strategy to induce a desired response from a follower. Such dynamic games with decentralized information structures have been well-studied under three assumptions-the leader must have access to the follower's observations, actions, and the objective function. Lack of knowledge on any of these can potentially lead to performance loss for the leader. In this paper, we first study a setup where the leader observes the follower's action through a random monitoring channel and learns about the follower's observation through a follower-designed signal. In this setup, we establish the existence of a signaling-based incentive equilibrium strategy for the leader that induces honest reporting and desired control response from the follower. Then, we study a setting, where the follower's costs are parametric, but the parameters are not known to the leader. We construct an incentive strategy that reduces the sensitivity of the leader's performance to uncertainty in the parameter, close to an initial estimate. More generally, for the case when the leader's knowledge about the follower's cost and distributions of cost-relevant random variables is inaccurate, we establish the existence of a robust incentive equilibrium strategy that bounds the performance loss from the inaccuracy in the model.

Local Optimality of Almost Piecewise-Linear Quantizers for Witsenhausen's Problem

We pose Witsenhausen's problem as a leader-follower game of incomplete information. The follower makes a noisy observation of the leader's action (who moves first) and chooses an action minimizing her expected deviation from the leader's action. Knowing this, the leader who observes the realization of the state chooses an action that minimizes her distance to the state of the world and the ex-ante expected deviation from the follower's action. We study the perfect Bayesian equilibria of the game and identify a class of “near-piecewise-linear equilibria” when leader cares much more about being close to the follower than the state, and the state is highly volatile. As a major consequence of this result, we prove the existence of a set of local minima for Witsenhausen's problem in form of slopey quantizers, which are at most a constant factor away from the optimal cost.

Reciprocal information flow and role distribution support joint action coordination

Many joint actions require task partners to temporally coordinate actions that follow different spatial patterns. This creates the need to find trade-offs between temporal coordination and spatial alignment. To study coordination under incongruent spatial and temporal demands, we devised a novel coordination task that required task partners to synchronize their actions while tracing different shapes that implied conflicting velocity profiles. In three experiments, we investigated whether coordination under incongruent demands is best achieved through mutually coupled predictions or through a clear role distribution with only one task partner adjusting to the other. Participants solved the task of trading off spatial and temporal coordination demands equally well when mutually perceiving each other’s actions without any role distribution, and when acting in a leader-follower configuration where the leader was unable to see the follower’s actions. Coordination was significantly worse when task partners who had been assigned roles could see each other’s actions. These findings make three contributions to our understanding of coordination mechanisms in joint action. First, they show that mutual prediction facilitates coordination under incongruent demands, demonstrating the importance of coupled predictive models in a wide range of coordination contexts. Second, they show that mutual alignment of velocity profiles in the absence of a leader-follower dynamic is more wide-spread than previously thought. Finally, they show that role distribution can result in equally effective coordination as mutual prediction without role assignment, provided that the role distribution is not arbitrarily imposed but determined by (lack of) perceptual access to a partner’s actions.

$Q$-Learning in a Stochastic Stackelberg Game between an Uninformed Leader and a Naive Follower

We consider a game between a leader and a follower, where the players' actions affect the stochastic dynamics of the state process $x_t$, $t\in\mathbb Z_+$. The players observe their rewards and the system state $x_t$. The transition kernel of the process $x_t$ and the opponent rewards are unobservable. At each stage of the game, the leader selects action $a_t$ first. When selecting the action $b_t$, the follower knows the action $a_t$. The follower's actions are unknown to the leader (an uniformed leader). Each player tries to maximize the discounted criterion by applying the $Q$-learning algorithm. The players' randomized strategies are uniquely determined by Boltzmann distributions depending on the $Q$-functions, which are updated in the course of learning. The specific feature of the algorithm is that when updating the $Q$-function, the follower believes that the action of the leader in the next state is the same as in the current one (a naive follower). It is shown that the convergence of the algorithm is secured by the existence of deterministic stationary strategies that generate an irreducible Markov chain. The limiting large time behavior of the players' $Q$-functions is described in terms of controlled Markov processes. The distributions of the players' actions converge to Boltzmann distributions depending on the limiting $Q$-functions.