Linear Regret Research Articles

Improved Bandits in Many-to-One Matching Markets with Incentive Compatibility

Two-sided matching markets have been widely studied in the literature due to their rich applications. Since participants are usually uncertain about their preferences, online algorithms have recently been adopted to learn them through iterative interactions. An existing work initiates the study of this problem in a many-to-one setting with responsiveness. However, their results are far from optimal and lack guarantees of incentive compatibility. We first extend an existing algorithm for the one-to-one setting to this more general setting and show it achieves a near-optimal bound for player-optimal regret. Nevertheless, due to the substantial requirement for collaboration, a single player's deviation could lead to a huge increase in its own cumulative rewards and a linear regret for others. In this paper, we aim to enhance the regret bound in many-to-one markets while ensuring incentive compatibility. We first propose the adaptively explore-then-deferred-acceptance (AETDA) algorithm for responsiveness setting and derive an upper bound for player-optimal stable regret while demonstrating its guarantee of incentive compatibility. This result is a significant improvement over existing works. And to the best of our knowledge, it constitutes the first player-optimal guarantee in matching markets that offers such robust assurances. We also consider broader substitutable preferences, one of the most general conditions to ensure the existence of a stable matching and cover responsiveness. We devise an online DA (ODA) algorithm and establish an upper bound for the player-pessimal stable regret for this setting.

On the regret of online edge service hosting

We consider the problem of service hosting where a service provider can dynamically rent edge resources via short term contracts to ensure better quality of service to its customers. The service can also be partially hosted at the edge, in which case, customers’ requests can be partially served at the edge. The total cost incurred by the system is modeled as a combination of the rent cost, the service cost incurred due to latency in serving customers, and the fetch cost incurred as a result of the bandwidth used to fetch the code/databases of the service from the cloud servers to host the service at the edge. In this paper, we compare multiple hosting policies with regret as a metric, defined as the difference in the cost incurred by the policy and the optimal policy over some time horizon T. In particular we consider the Retro Renting (RR) and Follow The Perturbed Leader (FTPL) policies proposed in the literature and provide performance guarantees on the regret of these policies. We show that under i.i.d stochastic arrivals, RR policy has linear regret while FTPL policy has constant regret. Next, we propose a variant of FTPL, namely Wait then FTPL (W-FTPL), which also has constant regret while demonstrating much better dependence on the fetch cost. We also show that under adversarial arrivals, RR policy has linear regret while both FTPL and W-FTPL have regret O(T) which is order-optimal.

On the Regret of Online Edge Service Hosting

We consider the problem of service hosting where a service provider can dynamically rent edge resources via short term contracts to ensure better quality of service to its customers. The total cost incurred by the system is modeled as a combination of the rent cost, the service cost incurred due to latency in serving customers, and the fetch cost incurred as a result of the bandwidth used to fetch the code/databases of the service from the cloud servers to host the service at the edge. In this paper, we compare multiple hosting policies with regret as a metric, defined as the difference in the cost incurred by the policy and the optimal policy over some time horizon T. In particular we consider the Retro Renting (RR) and Follow The Perturbed Leader (FTPL) policies proposed in the literature and provide performance guarantees on the regret of these policies. We show that under i.i.d Bernoulli arrivals, RR policy has linear regret while FTPL policy has constant regret. Next, we propose a variant of FTPL, namely Wait then FTPL (W-FTPL), which also has constant regret while demonstrating much better dependence on the fetch cost. We also show that under adversarial arrivals, RR policy has linear regret while both FTPL and W-FTPL have regret O(pT) which is order-optimal.

Implications of Regret on Stability of Linear Dynamical Systems

The setting of an agent making decisions under uncertainty and under dynamic constraints is common for the fields of optimal control, reinforcement learning, and recently also for online learning. In the online learning setting, the quality of an agent's decision is often quantified by the concept of regret, comparing the performance of the chosen decisions to the best possible ones in hindsight. While regret is a useful performance measure, when dynamical systems are concerned, it is important to also assess the stability of the closed-loop system for a chosen policy. In this work, we show that for linear state feedback policies and linear systems subject to adversarial disturbances, linear regret implies asymptotic stability in both time-varying and time-invariant settings. Conversely, we also show that bounded input bounded state stability and summability of the state transition matrices imply linear regret.

Influence of Trust, Quality of Service and Value Perception of User Confidence on Mobile Banking Services (Case Study on BNI Palembang Branch Customers)

This study was conducted to determine the impact of trust, service quality, and value perception of user confidence in mobile banking services for BNI customers in Palembang. The population in this study is all customers who use mobile banking services at PT. Bank Negara Indonesia (Persero), Tbk Palembang Branch, with a sample of 100 people, will be distributed to active customers at Bank BNI Cabang Palembang in 2021-2022 using the purposive sampling method. Based on the results of multiple linear regret analysis that has been carried out shows that trust, service quality, and value perception have a positive and significant effect on confidence in users of mobile banking services for BNI Customers in Abang Palembang. Service features n mobile banking need to be added, especially to the increasingly diverse top e-wallet services. Display the balance amount when making transactions such as transfers or e-wallet top-ups. Quality of service, there is a crash when topping up an e-wallet and top-up credit. BNI needs to increase the capacity of visits and establish cooperative relationships with e-wallet providers or service providers to provide better services. Perception of value, there needs to be follow-up or evaluation regarding changes in mpin and transaction codes if mobile banking users move applications by logging in and resetting on other devices because the BNI mobile banking service feature has many regulations that are a bit difficult to feel for users who are around >40 years.

Robust Multi-Agent Bandits Over Undirected Graphs

We consider a multi-agent multi-armed bandit setting in which n honest agents collaborate over a network to minimize regret but m malicious agents can disrupt learning arbitrarily. Assuming the network is the complete graph, existing algorithms incur O((m + K/n) łog (T) / Δ ) regret in this setting, where K is the number of arms and Δ is the arm gap. For m łl K, this improves over the single-agent baseline regret of O(Kłog(T)/Δ). In this work, we show the situation is murkier beyond the case of a complete graph. In particular, we prove that if the state-of-the-art algorithm is used on the undirected line graph, honest agents can suffer (nearly) linear regret until time is doubly exponential in K and n . In light of this negative result, we propose a new algorithm for which the i -th agent has regret O(( dmal (i) + K/n) łog(T)/Δ) on any connected and undirected graph, where dmal(i) is the number of i 's neighbors who are malicious. Thus, we generalize existing regret bounds beyond the complete graph (where dmal(i) = m), and show the effect of malicious agents is entirely local (in the sense that only the dmal (i) malicious agents directly connected to i affect its long-term regret).

A Simple and Optimal Policy Design for Online Learning with Safety against Heavy-tailed Risk

We design simple and optimal policies that ensure safety against heavy-tailed risk in the classical multi-armed bandit problem. Recently, \cite{fan2021fragility} showed that information-theoretically optimized bandit algorithms suffer from serious heavy-tailed risk; that is, the worst-case probability of incurring a linear regret slowly decays at a rate of $1/T$, where $T$ is the time horizon. Inspired by their results, we further show that widely used policies such as the standard Upper Confidence Bound policy and the Thompson Sampling policy also incur heavy-tailed risk; and this heavy-tailed risk actually exists for all "instance-dependent consistent" policies. To ensure safety against such heavy-tailed risk, for the two-armed bandit setting, we provide a simple policy design that (i) has the worst-case optimality for the expected regret at order $\tilde O(\sqrt{T})$ and (ii) has the worst-case tail probability of incurring a linear regret decay at an exponential rate $\exp(-\Omega(\sqrt{T}))$. We further prove that this exponential decaying rate of the tail probability is optimal across all policies that have worst-case optimality for the expected regret. Finally, we improve the policy design and analysis to the general setting with an arbitrary $K$ number of arms. We provide detailed characterization of the tail probability bound for any regret threshold under our policy design. Namely, the worst-case probability of incurring a regret larger than $x$ is upper bounded by $\exp(-\Omega(x/\sqrt{KT}))$. Numerical experiments are conducted to illustrate the theoretical findings. Our results reveal insights on the incompatibility between consistency and light-tailed risk, whereas indicate that worst-case optimality on expected regret and light-tailed risk are compatible.

Ballooning multi-armed bandits

In this paper, we introduce ballooning multi-armed bandits (BL-MAB), a novel extension of the classical stochastic MAB model. In the BL-MAB model, the set of available arms grows (or balloons) over time. In contrast to the classical MAB setting where the regret is computed with respect to the best arm overall, the regret in a BL-MAB setting is computed with respect to the best available arm at each time. We first observe that the existing stochastic MAB algorithms result in linear regret for the BL-MAB model. We prove that, if the best arm is equally likely to arrive at any time instant, a sub-linear regret cannot be achieved. Next, we show that if the best arm is more likely to arrive in the early rounds, one can achieve sub-linear regret. Our proposed algorithm determines (1) the fraction of the time horizon for which the newly arriving arms should be explored and (2) the sequence of arm pulls in the exploitation phase from among the explored arms. Making reasonable assumptions on the arrival distribution of the best arm in terms of the thinness of the distribution's tail, we prove that the proposed algorithm achieves sub-linear instance-independent regret. We further quantify explicit dependence of regret on the arrival distribution parameters. We reinforce our theoretical findings with extensive simulation results. We conclude by showing that our algorithm would achieve sub-linear regret even if (a) the distributional parameters are not exactly known, but are obtained using a reasonable learning mechanism or (b) the best arm is not more likely to arrive early, but a large fraction of arms is likely to arrive relatively early.

Data-Driven Online Recommender Systems with Costly Information Acquisition

In numerous recommender systems, collecting useful information from users is costly, implying that the recommender system has to make active choices by simultaneously learning the observations of the features' states to make useful recommendations to users among available products and services. This paper integrates information acquisition decisions into recommender system. To solve the aforementioned dual learning problem, we propose two different algorithms, namely Sim-OOS and Seq-OOS, where observations are made simultaneously and sequentially, respectively. We prove that both algorithms guarantee a sub linear regret. The developed recommender system can be applied to a variety of real-world applications, including medical informatics, smart transportation, finance, and cyber-security where collecting information before making decisions results in an excessive cost. We validate and evaluate our proposed policies in a medical decision-support system that recommends tests and treatments for breast cancer patients.

Distributed Optimal State Consensus for Multiple Circuit Systems With Disturbance Rejection

This paper investigates the distributed optimal state consensus problem for an electronic system with a group of circuit units. The dynamics of each unit is modeled by a Chua's circuit in the presence of disturbance generated by an external system. By means of the internal model approach and feedback control, a compensator-based continuous-time algorithm is proposed to minimize the sum of all cost functions associated with each individual unit in a cooperative manner. Supported by convex analysis, graph theory and Lyapunov theory, it is proved that the proposed algorithm is exponentially convergent. Compared with the centralized algorithms, the proposed protocol possesses remarkable superiority in improving scalability and reliability of multiple circuit systems. Moreover, we also study the distributed uncertain optimal state consensus problem and a linear regret bound is obtained in this case. Finally, a state synchronization example is provided to validate the effectiveness of the proposed algorithms.

Satisficing Search and Price Competition

The rising prevalence of algorithmic pricing in industry calls for a closer study of market outcomes that arise when these algorithms interact in a competitive environment. In this paper, we study the impact of boundedly rational customer choice on the landscape of joint demand curves in competition, and its consequences on the properties of competing pricing algorithms derived from the online convex optimization framework. In particular, we study price competition in a duopoly under a new customer behavior model motivated by online platforms offering perfectly substitutable services, e.g., ride-hailing. In this model, a customer samples the firms in an idiosyncratically preferred order until she finds one priced below her willingness-to-pay. We exhaustively characterize the equilibrium profile of the resulting pricing game when customers' willingness-to-pay distribution has a monotone hazard rate. We show that these games are plagued by a particular strictly-local Nash equilibrium, in which the price of the firm with a smaller market share is only a local best-response to the competitor's price, when a globally optimal response with a potentially unboundedly higher payoff is available. Through numerical experiments, we show that price dynamics resulting from distributed online gradient descent may often converge to this undesirable outcome, causing the smaller firm to incur linear regret relative to its best-response price. Our results thus suggest that algorithms that adaptively rely on local information for pricing, despite faring well in monopolistic settings, may suffer from serious drawbacks in competitive environments. We finally propose and test a practical resolution of this concern in the context of our model.

Incentivizing High Quality User Contributions: New Arm Generation in Bandit Learning

We study the problem of incentivizing high quality contributions in user generated content platforms, in which users arrive sequentially with unknown quality. We are interested in designing a content displaying strategy which decides which content should be chosen to show to users, with the goal of maximizing user experience (i.e., the likelihood of users liking the content).This goal naturally leads to a joint problem of incentivizing high quality contributions and learning the unknown content quality. To address the incentive issue, we consider a model in which users are strategic in deciding whether to contribute and are motivated by exposure, i.e., they aim to maximize the number of times their contributions are viewed. For the learning perspective, we model the content quality as the probability of obtaining positive feedback (e.g., like or upvote) from a random user. Naturally, the platform needs to resolve the classical trade-off between exploration (collecting feedback for all content) and exploitation (displaying the best content). We formulate this problem as a multi-arm bandit problem, where the number of arms (i.e., contributions) is increasing over time and depends on the strategic choices of arriving users. We first show that applying standard bandit algorithms incentivizes a flood of low cost contributions, which in turn leads to linear regret. We then propose Rand_UCB which adds an additional layer of randomization on top of the UCB algorithm to address the issue of flooding contributions. We show that Rand_UCB helps eliminate the incentives for low quality contributions, provides incentives for high quality contributions (due to bounded number of explorations for the low quality ones), and achieves sub-linear regrets with respect to displaying the current best arms.

Finite budget analysis of multi-armed bandit problems

In the budgeted multi-armed bandit (MAB) problem, a player receives a random reward and needs to pay a cost after pulling an arm, and he cannot pull any more arm after running out of budget. In this paper, we give an extensive study of the upper confidence bound based algorithms and a greedy algorithm for budgeted MABs. We perform theoretical analysis on the proposed algorithms, and show that they all enjoy sublinear regret bounds with respect to the budget B. Furthermore, by carefully choosing the parameters, they can even achieve log linear regret bounds. We also prove that the asymptotic lower bound for budgeted Bernoulli bandits is Ω(ln B). Our proof technique can be used to improve the theoretical results for fractional KUBE [26] and Budgeted Thompson Sampling [30].

Misspecified Linear Bandits

We consider the problem of online learning in misspecified linear stochastic multi-armed bandit problems. Regret guarantees for state-of-the-art linear bandit algorithms such as Optimism in the Face of Uncertainty Linear bandit (OFUL) hold under the assumption that the arms expected rewards are perfectly linear in their features. It is, however, of interest to investigate the impact of potential misspecification in linear bandit models, where the expected rewards are perturbed away from the linear subspace determined by the arms features. Although OFUL has recently been shown to be robust to relatively small deviations from linearity, we show that any linear bandit algorithm that enjoys optimal regret performance in the perfectly linear setting (e.g., OFUL) must suffer linear regret under a sparse additive perturbation of the linear model. In an attempt to overcome this negative result,we define a natural class of bandit models characterized by a non-sparse deviation from linearity. We argue that the OFUL algorithm can fail to achieve sublinear regret even under models that have non-sparse deviation. We finally develop a novel bandit algorithm, comprising a hypothesis test for linearity followed by a decision to use either the OFUL or Upper Confidence Bound (UCB) algorithm. For perfectly linear bandit models, the algorithm provably exhibits OFULs favorable regret performance, while for misspecified models satisfying the non-sparse deviation property, the algorithm avoids the linear regret phenomenon and falls back on UCBs sublinear regret scaling. Numerical experiments on synthetic data, and on recommendation data from the public Yahoo! Learning toRank Challenge dataset, empirically support our findings.

Pengaruh Budaya Kerja Terhadap Kinerja Pegawai di Kantor Kementerian Agama Kabupaten Gunung Mas Provinsi Kalimantan Tengah

This research aims to determine the influence of work culture on the performance of employees in the Office of the Ministry of Religious Affairs of Gunung Mas Regency of Central Kalimantan, by applying hypothesis as follows: "There is a real positive influence or influence Working culture to employees ' performance. " The method used in this study is quantitative. The samples in this study amounted to 45 people, using a total sampling technique. Technical data collection using questionnaires that are distributed to the employees of the Ministry of Religious Affairs at Gunung Mas District office. The data analysis technique in this study uses the simple linear Regret method

On Solving Some Stochastic Discrete Optimization Problems Under General Regret Function

In this paper we consider stochastic discrete optimization problems (DOP) in which feasible solutions remain feasible irrespective of the randomness of the problem parameters. We introduce the concept of the risk associated with a solution and define optimal solution in terms of having least possible risk. We show that a least risk solution can be obtained by solving a non-stochastic discrete optimization problem similar to the stochastic problem in certain problems and present results regarding the generation of the non-stochastic problem in terms of finding the parameter of the distribution which may act as surrogate for the random element in its no-stochastic counterpart. While this surrogate in the mean for a linear regret function, the situation is complex under general regret. Our results show that the above result continues to hold (in general) if the DOP has only one random element having symmetric distribution. We obtain some bounds for this parameter for certain group of asymmetric distributions and study its limiting behavior under two asymptotic setup. We establish through various examples that the results from uni-dimensional case cannot be extended to stochastic DOP with multiple random element with any reasonable generality. However, we characterize a finite number of solutions which will include the optimal solution in this case. An heuristic based on local search type algorithm is also devised when the number of random elements is too high, and we study the performance of this algorithm through simulation.