Non-parametric Learning Algorithms Research Articles

Competitive Demand Learning: A Noncooperative Pricing Algorithm with Coordinated Price Experimentation

We consider a periodical equilibrium pricing problem for multiple firms over a planning horizon of [Formula: see text] periods. At each period, firms set their selling prices and receive stochastic demand from consumers. Firms do not know their underlying demand curve, but they wish to determine the selling prices to maximize total revenue under competition. Hence, they have to do some price experiments such that the observed demand data are informative to make price decisions. However, uncoordinated price updating can render the demand information gathered by price experimentation less informative or inaccurate. We design a nonparametric learning algorithm to facilitate coordinated dynamic pricing, in which competitive firms estimate their demand functions based on observations and adjust their pricing strategies in a prescribed manner. We show that the pricing decisions, determined by estimated demand functions, converge to underlying equilibrium as time progresses. We obtain a bound of the revenue difference that has an order of [Formula: see text] and a regret bound that has an order of [Formula: see text] with respect to the number of the competitive firms [Formula: see text] and [Formula: see text]. We also develop a modified algorithm to handle the situation where some firms may have the knowledge of the demand curve.

Decentralized diffusion-based learning under non-parametric limited prior knowledge

We study the problem of diffusion-based network learning of a nonlinear phenomenon, m, from local agents’ measurements collected in a noisy environment. For a decentralized network and information spreading merely between directly neighboring nodes, we propose a non-parametric learning algorithm, that avoids raw data exchange and requires only mild a priori knowledge about m. Non-asymptotic estimation error bounds are derived for the proposed method. Its potential applications are illustrated through simulation experiments.

Network revenue management with online inverse batch gradient descent method

We consider a general class of price‐based network revenue management problems that a firm aims to maximize revenue from multiple products produced with multiple types of resources endowed with limited inventory over a finite selling season. A salient feature of our problem is that the firm does not know the underlying demand function that maps prices to demand rate, which must be learned from sales data. It is well known that for almost all classes of demand functions, such as linear, exponential, multinomial logit, and nested logit models, the revenue rate function is not concave in the products' prices but is concave in products' market shares (or price‐controlled demand rates). This creates challenges in adopting any stochastic gradient descent based methods in the price space. We propose a novel nonparametric learning algorithm termed the online inverse batch gradient descent (IGD) algorithm. This algorithm proceeds in batches. In each batch, the firm implements each product's perturbed prices and then uses the sales information to estimate the market shares. Leveraging these estimates, the firm carries out a stochastic gradient descent step in the market share space that takes into account the relative inventory scarcity for the entire horizon and then inversely maps the updated market shares back to the price space to obtain the prices for the next batch. Moreover, we also propose an inventory‐adjusted algorithm (IGD‐I) that the feasible market share set is dynamically adjusted to capture the real‐time relative inventory scarcity for the remaining season. For the large‐scale systems wherein all resources' inventories and the length of the horizon are proportionally scaled by a parameter k, we establish a dimension‐independent regret bound of [Formula: see text]. This result is independent of the number of products and resources and works for a continuum action‐set prices and the demand functions that are only once differentiable. Our theoretical result guarantees the efficacy of both algorithms in the high‐dimensional systems where the number of products or resources is large and the prices are continuous. Our algorithms also numerically outperform the existing algorithms in the literature.

Enhancing PRNU-based image forensics with a non-parametric correlation predictor based on locally weighted regression

For PRNU-based image manipulation localization, the cor- relation predictor plays a crucial role to reduce false positives considerably, as well as increasing accuracy of manipulation local- ization. In this paper, we propose a novel correlation predictor with a non-parametric learning algorithm, which is Locally Weighted Regression. Instead of fitting a global set of model parameters, a non-parametric learning algorithm fits a model dynamically by sampling the training set based on the pixel in the query image at which the correlation needs to be predicted. Our experimental results suggest that building a model dynamically based on the distance of training examples from the query pixel in the feature space helps to predict the correlation more accurately. Experimental results on benchmark data indicate that integrating the new predictor significantly improves the accuracy of predicted correlation, as well as image manipulation localization performance of PRNU-based forensic detectors.

Learning new physics efficiently with nonparametric methods

We present a machine learning approach for model-independent new physics searches. The corresponding algorithm is powered by recent large-scale implementations of kernel methods, nonparametric learning algorithms that can approximate any continuous function given enough data. Based on the original proposal by D’Agnolo and Wulzer (Phys Rev D 99(1):015014, 2019, arXiv:1806.02350 [hep-ph]), the model evaluates the compatibility between experimental data and a reference model, by implementing a hypothesis testing procedure based on the likelihood ratio. Model-independence is enforced by avoiding any prior assumption about the presence or shape of new physics components in the measurements. We show that our approach has dramatic advantages compared to neural network implementations in terms of training times and computational resources, while maintaining comparable performances. In particular, we conduct our tests on higher dimensional datasets, a step forward with respect to previous studies.

Data‐Driven Inventory Control with Shifting Demand

We consider an inventory control problem with lost‐sales in a shifting demand environment. Over a planning horizon of T periods, demand distributions can change up to O( log T) times, but the firm does not know the demand distributions before or after each change, the time periods when changes occur, or the number of changes. Therefore, the firm needs to detect changes and learn the demand distributions only from historical sales data. We show that with censored demand, active exploration in the inventory space is needed for a reasonable detecting and learning algorithm. We provide a theoretical lower bound by partitioning all admissible policies into either exploration‐heavy or exploitation‐heavy, and for both categories we prove that the convergence rate cannot be better than [Formula: see text]. We then develop a nonparametric learning algorithm for this problem and prove that it achieves a convergence rate that (almost) matches the theoretical lower bound.

Nonparametric Learning Algorithms for Joint Pricing and Inventory Control with Lost Sales and Censored Demand

We consider a joint pricing and inventory control problem in which the customer’s response to selling price and the demand distribution are not known a priori. Unsatisfied demand is lost and unobserved, and the only available information for decision making is the observed sales data (also known as censored demand). Conventional approaches, such as stochastic approximation, online convex optimization, and continuum-armed bandit algorithms, cannot be employed, because neither the realized values of the profit function nor its derivatives are known. A major challenge of this problem lies in that the estimated profit function constructed from observed sales data is multimodal in price. We develop a nonparametric spline approximation–based learning algorithm. The algorithm separates the planning horizon into a disjoint exploration phase and an exploitation phase. During the exploration phase, a spline approximation of the demand-price function is constructed based on sales data, and then the corresponding surrogate optimization problem is solved on a sparse grid to obtain a pair of recommended price and target inventory level. During the exploitation phase, the algorithm implements the recommended strategies. We establish a (nearly) square-root regret rate, which (almost) matches the theoretical lower bound.

Traffic-Aware Backscatter Communications in Wireless-Powered Heterogeneous Networks

With the emerging Internet-of-Things services, massive machine-to-machine (M2M) communication will be deployed on top of human-to-human (H2H) communication in the near future. Due to the coexistence of M2M and H2H communications, the performance of M2M (i.e., secondary) networks depends largely on the H2H (i.e., primary) network. In this paper, we propose ambient backscatter communication for the M2M networks which exploits the energy (signal) sources of the H2H network, referring to its traffic applications and popularity. In order to maximize the harvesting and transmission opportunities offered by varying traffic sources of the H2H network, we adopt a Bayesian nonparametric (BNP) learning algorithm to classify traffic applications (patterns) for secondary transmitters (STs). We then analyze the performance of STs using the stochastic geometric approach, based on a criterion for optimal traffic selection. Because of the mathematical intractability of the optimal criterion, we have attained a suboptimal traffic selection criterion which provides more tractable analysis. Results are presented to validate the performance of the proposed BNP classification algorithm and the criterion, as well as the impact of traffic sources and popularity.

Data-Driven Inventory Control in a Changing Environment

We consider an inventory control problem with lost-sales in a shifting demand environment. Over a planning horizon of T periods, demand distributions can change up to O(log T) times, but the firm does not know the demand distributions before or after each change, the time periods when changes occur, or the number of changes. Therefore, the firm needs to detect changes and learn the demand distributions only from historical sales data. We show that with censored demand, active exploration in the inventory space is needed for a reasonable detecting and learning algorithm. We provide a theoretical lower bound by partitioning all admissible policies into either exploration-heavy or exploitation-heavy, and for both categories we prove that the convergence rate cannot be better than Ω(√T). We then develop a nonparametric learning algorithm for this problem and prove that it achieves a convergence rate that (almost) matches the theoretical lower bound.

Tailored Base-Surge Policies in Dual-Sourcing Inventory Systems with Demand Learning

We consider a periodic-review dual-sourcing inventory system, in which the expedited supplier is faster and more costly, while the regular supplier is slower and cheaper. Under full demand distributional information, it is well-known that the optimal policy is extremely complex but the celebrated Tailored Base-Surge (TBS) policy performs near optimally. Under such a policy, a constant order is placed at the regular source in each period, while the order placed at the expedited source follows a simple order-up-to rule. In this paper, we assume that the firm does not know the demand distribution a priori, and makes adaptive inventory ordering decisions in each period based only on the past sales (a.k.a. censored demand) data. The standard performance measure is regret, which is the cost difference between a feasible learning algorithm and the clairvoyant (full-information) benchmark. When the benchmark is chosen to be the (full-information) optimal Tailored Base-Surge policy, we develop the first nonparametric learning algorithm that admits a regret bound of O(T^{1/2} (log T)^{3} loglog T), which is provably tight up to a logarithmic factor. Leveraging the structure of this problem, our approach combines the power of bisection search and stochastic gradient descent and also involves a delicate high probability coupling argument between our and the clairvoyant optimal system dynamics. We also develop several technical results that are of independent interest.

Cooking Is Creating Emotion: A Study on Hinglish Sentiments of Youtube Cookery Channels Using Semi-Supervised Approach

The success of Youtube has attracted a lot of users, which results in an increase of the number of comments present on Youtube channels. By analyzing those comments we could provide insight to the Youtubers that would help them to deliver better quality. Youtube is very popular in India. A majority of the population in India speak and write a mixture of two languages known as Hinglish for casual communication on social media. Our study focuses on the sentiment analysis of Hinglish comments on cookery channels. The unsupervised learning technique DBSCAN was employed in our work to find the different patterns in the comments data. We have modelled and evaluated both parametric and non-parametric learning algorithms. Logistic regression with the term frequency vectorizer gave 74.01% accuracy in Nisha Madulika’s dataset and 75.37% accuracy in Kabita’s Kitchen dataset. Each classifier is statistically tested in our study.

Network Revenue Management with Online Inverse Batch Gradient Descent Method

We consider a general class of price-based network revenue management problems that a firm aims to maximize revenue from multiple products produced with multiple types of resources endowed with limited inventory over a finite selling season. A salient feature of our problem is that the firm does not know the underlying demand function that maps prices to demand rate, which must be learned from sales data. It is well known that for almost all classes of demand functions, such as linear, exponential, multinomial logit and nested logit models, the revenue rate function is not concave in the products' prices but is concave in products' market shares (or price-controlled demand rates). This creates challenges in adopting any stochastic gradient descent based methods in the price space. We propose a novel nonparametric learning algorithm termed online inverse batch gradient descent (IGD) algorithm. This algorithm proceeds in batches. In each batch, the firm implements each product's perturbed prices, and then uses the sales information to estimate the market shares. Leveraging these estimates, the firm carries out a stochastic gradient descent step in the market share space that takes into account the relative inventory scarcity for the entire horizon, and then inversely maps the updated market shares back to the price space to obtain the prices for the next batch. Moreover, we also propose an inventory adjusted algorithm (IGD-I) that the feasible market share set is dynamically adjusted to capture the real-time relative inventory scarcity for the remaining season. For the large scale systems wherein all resources' inventories and the length of the horizon are proportionally scaled by a parameter $k$, we establish a dimension-independent regret bound of $O( k^{4/5} \log k)$. This result is independent of the number of products and resources and works for continuum action-set prices and the demand functions that are only once differentiable. Our theoretical result guarantees the efficacy of both algorithms in the high dimensional systems where the number of products or resources is large and the prices are continuous. Our algorithms also numerically outperform the existing algorithms in the literature.

Dynamic Pricing and Learning: An Application of Gaussian Process Regression

We consider the problem of offering an optimal price when demand is unknown and must be learned by price experimentation - a variant of the multi-armed bandit problem. In each period, the retailer must decide on a price while using knowledge already acquired to weigh the benefits of exploring other prices versus maximizing revenue. Other algorithms proposed for such problems either learn the price-demand relation for each price separately or determine the parameters of an assumed parametric demand function. We instead combine Gaussian process regression with Thompson sampling as a nonparametric learning algorithm that can learn any functional relation between price and demand. This GP-TS algorithm scales significantly better with the number of price vectors than do existing approaches, yet it makes no restrictive assumptions on the price-demand relationship's functional form. We show how to apply the algorithm to finite inventory settings that consider both single and multiple products and also to settings in which exogenous contextual information can affect prices. One advantage of the algorithm is that its performance depends not on the number of price vectors over all products but only on the number of products considered. For each setting considered here, we benchmark our approach to existing algorithms. The GP-TS algorithm's learning performance is far superior to that of its peers, especially when there are increases in the number of products to learn concurrently. Finally, we propose extensions that enable the application of our algorithm when demand follows a Bernoulli or Poisson distribution.

A Predictive Modeling for Detecting Fraudulent Automobile Insurance Claims

Fraudulent automobile insurance claims are not only a loss for insurance companies, but also for their policyholders. The goal of this research is to develop, first, a decision-making algorithm to classify whether a claim is classified as fraudulent or not; and, second, what types of variables should be focused to detect fraudulent claims. To achieve this goal, highly accurate prediction models are built by discovering important sets of features via variable selection algorithms, which can in turn help prevent future loss. In this research, parametric and nonparametric statistical learning algorithms are considered to reduce uncertainty and increase the chances of detecting the appropriate claims. An important set of features for a model is determined by measuring variable importance based on the observed characteristics of a claim via a cross-validation and by testing improvement of the performance at which automobile fraudulent claims are accurately classified using Akaike Information Criterion. We could achieve accuracy above 95% with a set of features selected via a cross-validation. This research would offer some benefit to the insurance industry for their fraud detection research in order to prevent insurance abuse from escalating any further.

Technical Note—Perishable Inventory Systems: Convexity Results for Base-Stock Policies and Learning Algorithms Under Censored Demand

We develop the first nonparametric learning algorithm for periodic-review perishable inventory systems. In contrast to the classical perishable inventory literature, we assume that the firm does not know the demand distribution a priori and makes replenishment decisions in each period based only on the past sales (censored demand) data. It is well known that even with complete information about the demand distribution a priori, the optimal policy for this problem does not possess a simple structure. Motivated by the studies in the literature showing that base-stock policies perform near optimal in these systems, we focus on finding the best base-stock policy. We first establish a convexity result, showing that the total holding, lost sales and outdating cost is convex in the base-stock level. Then, we develop a nonparametric learning algorithm that generates a sequence of order-up-to levels whose running average cost converges to the cost of the optimal base-stock policy. We establish a square-root convergence rate of the proposed algorithm, which is the best possible. Our algorithm and analyses require a novel method for computing a valid cycle subgradient and the construction of a bridging problem, which significantly departs from previous studies. The e-companion is available at https://doi.org/10.1287/opre.2018.1724

Object tracking by transitive learning using perspective transformation with asymptotic stability

Object tracking is a core subject in computer vision and has significant meaning in both theory and practice. We propose a tracking method in which a robust discriminative classifier is built based on both object and context information. In this method, we consider multiple frames of local invariant features on and around the object and construct the object template and context template. To overcome the limitation of the invariant representations, we also design a nonparametric learning algorithm using transitive matching perspective transformation. This learning algorithm can keep adding object appearance and can avoid improper updating when occlusions appear. We also analyze the asymptotic stability of our method and prove its drift-free capability in long-term tracking. Extensive experiments using challenging publicly available video sequences that cover most of the critical conditions in tracking demonstrate the enhanced strength and robustness of our method.

Closing the Gap: A Learning Algorithm for Lost-Sales Inventory Systems with Lead Times

We consider a periodic-review, single-product inventory system with lost sales and positive lead times under censored demand. In contrast to the classical inventory literature, we assume the firm does not know the demand distribution a priori and makes an adaptive inventory-ordering decision in each period based only on the past sales (censored demand) data. The standard performance measure is regret, which is the cost difference between a learning algorithm and the clairvoyant (full-information) benchmark. When the benchmark is chosen to be the (full-information) optimal base-stock policy, Huh et al. [Huh WT, Janakiraman G, Muckstadt JA, Rusmevichientong P (2009a) An adaptive algorithm for finding the optimal base-stock policy in lost sales inventory systems with censored demand. Math. Oper. Res. 34(2):397–416.] developed a nonparametric learning algorithm with a cubic-root convergence rate on regret. An important open question is whether there exists a nonparametric learning algorithm whose regret rate matches the theoretical lower bound of any learning algorithms. In this work, we provide an affirmative answer to this question. More precisely, we propose a new nonparametric algorithm termed the simulated cycle-update policy and establish a square-root convergence rate on regret, which is proven to be the lower bound of any learning algorithm. Our algorithm uses a random cycle-updating rule based on an auxiliary simulated system running in parallel and also involves two new concepts, namely the withheld on-hand inventory and the double-phase cycle gradient estimation. The techniques developed are effective for learning a stochastic system with complex system dynamics and lasting impact of decisions. This paper was accepted by Yinyu Ye, optimization.

Closing the Gap: A Learning Algorithm for the Lost-Sales Inventory System with Lead Times

We consider a periodic-review single-product inventory system with lost-sales and positive lead times under censored demand. In contrast to the classical inventory literature, we assume the firm does not know the demand distribution a priori, and makes adaptive inventory ordering decision in each period based only on the past sales (censored demand) data. The standard performance measure is regret, which is the cost difference between a feasible learning algorithm and the clairvoyant (full-information) benchmark. When the benchmark is chosen to be the (full-information) optimal base-stock policy, Huh et al. [Mathematics of Operations Research 34(2): 397-416 (2009)] developed a nonparametric learning algorithm with a cubic-root convergence rate on regret. An important open question is whether there exists a nonparametric learning algorithm whose regret rate matches the theoretical lower bound of any learning algorithms. In this work, we provide an affirmative answer to the above question. More precisely, we propose a new nonparametric algorithm termed the simulated cycle-update policy, and establish a square-root convergence rate on regret, which is proven to be the lower bound of any learning algorithms. Our algorithm uses a random cycle-updating rule based on an auxiliary simulated system running in parallel, and also involves two new concepts, namely, the withheld on-hand inventory and the double-phase cycle gradient estimation. The techniques developed are effective for learning a stochastic system with complex systems dynamics and lasting impact of decisions.

PAC Algorithms for Detecting Nash Equilibrium Play in Social Networks: From Twitter to Energy Markets

The detection of agents whose responses satisfy equilibrium play is useful for predicting the dynamics of information propagation in social networks. Using Afriat’s theorem of revealed preferences, we construct a non-parametric detection test to detect if the responses of a group of agents is consistent with play from the Nash equilibrium of a concave potential game. For agents that satisfy the detection test, it is useful to learn the associated concave potential function of the game. In this paper, a non-parametric learning algorithm is provided to estimate the concave potential function of agents with necessary and sufficient conditions on the response class for the algorithm to be a probably approximately correct learning algorithm. In the case of response signals measured in noise, a statistical test to detect agents playing a concave potential game that has a pre-specified Type-I error probability is provided. The detection tests and learning algorithm are applied to real-world data sets from the Twitter social network and the Ontario power grid.

Short-Term Load Forecasting Based on Sequential Relevance Vector Machine

This paper proposes a dynamic short-term load forecasting method that utilizes a new sequential learning algorithm based on Relevance Vector Machine (RVM). The method performs general optimization of weights and hyperparameters using the current relevance vectors and newly arriving data. By doing so, the proposed algorithm is trained with the most recent data. Consequently, it extends the RVM algorithm to real-time and nonstationary learning processes. The results of application of the proposed algorithm to prediction of electrical loads indicate that its accuracy is comparable to that of existing nonparametric learning algorithms. Further, the proposed model reduces computational complexity.