Unobserved Demand Research Articles

Dynamic pricing of regulated field services using reinforcement learning

Resource flexibility and dynamic pricing are effective strategies in mitigating uncertainties in production systems; however, they have yet to be explored in relation to the improvement of field operations services. We investigate the value of dynamic pricing and flexible allocation of resources in the field service operations of a regulated monopoly providing two services: installations (paid-for) and maintenance (free). We study the conditions under which the company can improve service quality and the profitability of field services by introducing dynamic pricing for installations and the joint management of the resources allocated to paid-for (with a relatively stationary demand) and free (with seasonal demand) services when there is an interaction between quality constraints (lead time) and the flexibility of resources (overtime workers at extra cost). We formalize this problem as a contextual multi-armed bandit problem to make pricing decisions for the installation services. A bandit algorithm can find the near-optimal policy for joint management of the two services independently of the shape of the unobservable demand function. The results show that (i) dynamic pricing and resource management increase profitability; (ii) regulation of the service window is needed to maintain quality; (iii) under certain conditions, dynamic pricing of installation services can decrease the maintenance lead time; (iv) underestimation of demand is more detrimental to profit contribution than overestimation.

Prediction of electricity prices for non‐regulated markets based on a power transformed mean reverting process

AbstractThe electricity price time series for non‐regulated markets presents two basic properties: (a) a mean reversion trend around a constant or deterministic function and (b) a price process with high volatility and marginal, conditional, and asymptotic distributions skewed to the right. We propose a microeconomic‐based model for the dynamics of electricity prices that provides a satisfactory explanation of the stylized features observed in non‐regulated electricity markets. The suggested model is based on a power transformation of the Ornstein–Uhlenbeck process, which accounts for the unobserved demand process. This means that the model requires only the spot price of the electricity price to be available. Parameter estimates were obtained using the maximum likelihood method. This approach was implemented for data from the Alberta electricity market in Canada, with satisfactory results in terms of residual analysis and forecasting. The success rate of predicting the price jump sign (upward and downward) was approximately . Moreover, the statistical properties for the maximum likelihood parameter estimators are shown via a Monte Carlo simulation of the electricity price time series.

A Fast and Efficient Markov Chain Monte Carlo Method for Market Microstructure Model

The non-linear market microstructure (MM) model for financial time series modeling is a flexible stochastic volatility model with demand surplus and market liquidity. The estimation of the model is difficult, since the unobservable surplus demand is a time-varying stochastic variable in the return equation, and the market liquidity arises both in the mean term and in the variance term of the return equation in the MM model. A fast and efficient Markov Chain Monte Carlo (MCMC) approach based on an efficient simulation smoother algorithm and an acceptance-rejection Metropolis–Hastings algorithm is designed to estimate the non-linear MM model. Since the simulation smoother algorithm makes use of the band diagonal structure and positive definition of Hessian matrix of the logarithmic density, it can quickly draw the market liquidity. In addition, we discuss the MM model with Student-t heavy tail distribution that can be utilized to address the presence of outliers in typical financial time series. Using the presented modeling method to make analysis of daily income of the S&P 500 index through the point forecast and the density forecast, we find clear support for time-varying volatility, volatility feedback effect, market microstructure theory, and Student-t heavy tails in the financial time series. Through this method, one can use the estimated market liquidity and surplus demand which is much smoother than the strong stochastic return process to assist the transaction decision making in the financial market.

Schedule-Constrained Demand Management in Two-Region Urban Networks

Demand management aiming to optimize system cost while ensuring user compliance in an urban traffic network is a challenging task. This paper introduces a cooperative demand redistribution strategy to optimize network performance through the retiming of departure times within a limited time window. The proposed model minimizes the total time spent in a two-region urban network by incurring minimal disruption to travelers’ departure schedules. Two traffic models based on the macroscopic fundamental diagram (MFD) are jointly implemented to redistribute demand and analyze travelers’ reaction. First, we establish equilibrium conditions via a day-to-day assignment process, which allows travelers to find their preferred departure times. The trip-based MFD model that incorporates individual traveler attributes is implemented in the day-to-day assignment, and it is conjugated with a network-level detour ratio model to incorporate the effect of congestion in individual traveler route choice. This allows us to consider travelers with individual preferences on departure times influenced by desired arrival times, trip lengths, and earliness and lateness costs. Second, we develop a nonlinear optimization problem to minimize the total time spent considering both observed and unobserved demand—that is, travelers opting in and out of the demand management platform. The accumulation-based MFD model that builds on aggregated system representation is implemented as part of the constraints in the nonlinear optimization problem. The results confirm the resourcefulness of the model to address complex two-region traffic dynamics and to increase overall performance by reaching a constrained system optimum scenario while ensuring the applicability at both full and partial user compliance conditions.

Dirichlet Sampled Capacity and Loss Estimation for LV Distribution Networks With Partial Observability

With low voltage (LV) distribution networks increasingly being re-purposed beyond their original design specifications to accommodate low carbon technologies, the ability to accurately calculate their actual spare capacity is critical. Traditionally, within the Great Britain (GB) power system, there has been limited monitoring of LV distribution networks, making this difficult. This paper proposes a method for estimating spare capacity of unmonitored LV networks using demand data from customer Smart Meters. In particular, the proposed method infers existing LV network capacity, as well as losses, across scenarios where only a limited number of customers have Smart Meters installed. Typical daily load profiles across customers with Smart Meters are learned using a Dirichlet sampled Gaussian mixture model (GMM). Learned profiles are then applied to all unmetered customers to estimate network parameters. Method accuracy is assessed by comparing estimations with simulated, fully observed, LV network models. The method is also compared to benchmark models for establishing unobserved demand profiles. Overall, results in the paper show that the proposed method outperforms benchmark models in terms of accurately assessing substation headroom, particularly in scenarios where only 10–50% of customers have Smart Meters installed.

The Chinese warrant bubble: A fundamental analysis

AbstractWe investigate the information content in Chinese warrant prices based on an option pricing framework that incorporates short‐selling and margin‐trading constraints in the underlying stock market. We show that Chinese warrant prices can be explained under this pricing framework. On the basis of this new model, we develop a price deviation measure to quantify stock market investors' unobserved demand for short selling or margin trading due to market constraints. We find that warrant‐price deviations are driven by underlying stock valuation to a great extent. Chinese warrant prices, save for the time around expiration dates, are better characterized as derivatives than as pure bubbles.

Dynamic soft sensor based parameters and demand curve estimation for Water Distribution System: Theoretical and Experimental cross validation

The ostensive process of each Water Distribution System (WDS) is manipulated by the hydraulic parameters such as dimensional features, working state of the system components and the demand profile of the end user. In general cases, the access of those factors is unobtainable due to the shortage of sensors and more capital expenditure on the measurement methods. In such circumstances, the crucial parameters have been foreseen with the obtainable measurements of WDS, which are often contaminated with measurement noises. In this study, a hybridized version of metaheuristic based Grey Wolf Optimization (GWO) is proposed using static Kalman Bucy (KB) mechanism as Hybridized Grey Wolf Optimization (HGWO). This proposed hybridization suppresses the effect of measurement noises over the parameter estimation. Further, to make the HGWO adaptable for any fault occurrence or dynamical changes, a statistical based fault diagnosis is also proposed. To assess the efficiency of the proposed algorithm under fault occurrence situation a synthetic WDS system considered as the first case study, where faults purposely injected into the system and performance of the estimation algorithm studied. The second case study is from eastern WDS section from Peroorkada town, Trivandrum City, India given by Kerala Water Authority (KWA) consists of 110 unknown parameters and 6 unobservable demand profiles. The credibility of estimation algorithm also tested in the Hardware in Loop platform (HIL) for field applications. For both the case studies, parameter estimation is carried out using and compared with the related algorithms, viz., Particle Swarm Optimization (PSO) and GWO. The obtained result shows that the proposed HGWO provides better estimates of the factors and the unobservable states for both the case studies.

Measuring the Impact of a Single Negative Customer Review on Online Search and Purchase Decisions

How much does a single negative customer review impact online search and purchase behavior? To answer this question, the authors compare choices made by consumers searching for a product that had a negative review posted on its product-page with those made when the same review had “moved” to a second page as additional reviews were posted. By focusing on where the review appears rather than whether it was submitted, our approach tackles concerns of spurious correlation between customer reviews and unobserved demand shocks. Using data from a large online retailer, we find that the discovery of a negative review – one that has 3-stars or less out of 5 – leads to an average drop in purchase probability of 51.4%, an increase in continuing search for a competing item of 11.4%, and a 15.8% increase in the price paid if a purchase occurs. Our findings are presented on a two-dimensional product map – an innovative way to evaluate the vulnerability of products to a single negative review.

Frictions in a Competitive, Regulated Market: Evidence from Taxis

This paper presents a dynamic equilibrium model of a taxi market. The model is estimated using data from New York City yellow cabs. Two salient features by which most taxi markets deviate from the efficient market ideal are, first, matching frictions created by the need for both market sides to physically search for trading partners, and second, regulatory limitations to entry. To assess the importance of these features, we use the model to simulate the effect of changes in entry, alternative matching technologies, and different market density. We use the geographical features of the matching process to back out unobserved demand through a matching simulation. The matching function exhibits increasing returns to scale, which is important to understand the impact of changes in this market and has welfare implications. For instance, although alternative dispatch platforms can be more efficient than street-hailing, platform competition is harmful because it reduces effective density. (JEL C78, L51, L84, L92, L98, R48)

Industrial Espionage in Duopoly Games

In the context of a duopoly market, firms can learn about unobservable demand from two sources: conducting market research, and spying on their competitor's market research results. In the unique linear equilibrium firms pick an action which is a weighted average of the market research signal and the espionage signal. A firm's preferred information source depends on its technological advantage over the competitor (in terms of spying precision), and on the degree of differentiation between their products. If the former is larger than the latter, the firm trusts its espionage more than its market research (i.e. places a larger weight on the respective signal), and vice versa. Spying under price competition is shown to be beneficial for the industry: aggregate profit increases as the precision of the spying device improves. The same is true under quantity competition if and only if the espionage technology is already more accurate than the alternative market research technology.

Instrument-Free Demand Estimation

We consider the identification of empirical models of supply and demand with imperfect competition. As is well known, a supply-side instrument can resolve price endogeneity in demand estimation. We show that, under common assumptions, two other approaches also yield consistent estimates of the joint model: (i) a demand-side instrument, or (ii) a covariance restriction between unobserved demand and cost shocks. Covariance restrictions can obtain identification even the absence of instruments. Further, supply and demand assumptions alone may bound the structural parameters without additional restrictions. We illustrate the covariance restriction approach with applications to ready-to-eat cereal, cement, and airlines.

Deterministic Versus Random Utility: Implied Patterns of Vertical Product Differentiation in a Multi-Product Monopoly

In this article we study patterns of vertical product differentiation in a multi-product monopoly using a random utility model. Prior research shows that applying such a model in a multi-product setting implies symmetric patterns of product differentiation in which all product variants of a single firm have the same characteristics. Assuming that preferences differ across consumers and allowing for unobserved demand heterogeneity, we numerically show the existence of asymmetric, fully differentiated, patterns of vertical product differentiation in which the monopolist maximizes profits by setting prices and qualities. In particular, we show that the patterns of vertical product differentiation depend crucially on the level of unobserved demand heterogeneity and the observed dispersion of willingness to pay for quality. Only if unobserved demand heterogeneity is small relative to the observed dispersion, asymmetric, fully differentiated, equilibriums exist. Furthermore, we find in our model that the level of unobserved heterogeneity and the dispersion of willingness to pay for quality do not affect the relative welfare efficiency of the monopolist.

Mortgage Supply and the US Housing Boom: The Role of the Community Reinvestment Act

This paper studies the role of the Community Reinvestment Act (CRA) in the recent US housing boom-bust cycle. Using a difference-in-differences matching estimation, I find that the enhancement of CRA enforcement in 1998 caused a 7.7 percentage points increase in annual growth rate of mortgage lending by CRA-regulated banks to CRA-eligible census tracts relative to a group of similar-income CRA-ineligible census tracts within the same state. Financial institutions which are not subject to the CRA, however, do not show any change in their mortgage supply between these two types of census tracts after 1998. I take advantage of this exogenous shift in mortgage supply within an instrumental variable framework to identify the causal effect of mortgage supply on housing prices. I find that every 1 percentage point higher annual growth rate of mortgage supply leads to 0.3 percentage points higher annual growth rate of housing prices. Reduced form regressions show that CRA-eligible neighborhoods experienced higher house price growth during the boom and sharper decline during the bust period. I use placebo tests to confirm that this effect is in fact channeled through the shift in mortgage supply by CRA-regulated banks and not by unobserved demand factors. Furthermore, my results indicate that CRA-induced mortgages went to borrowers with lower FICO scores, carried higher interest rates, and encountered more frequent delinquencies.

Dynamic Oligopoly with Incomplete Information

We consider learning and signalling in a dynamic Cournot oligopoly where firms have private information about their production costs and only observe the market price, which is subject to unobservable demand shocks. An equilibrium is Markov if play depends on the history only through the firms’ beliefs about costs and calendar time. We characterize symmetric linear Markov equilibria as solutions to a boundary value problem. In every such equilibrium, given a long enough horizon, play converges to the static complete information outcome for the realized costs, but each firm only learns its competitors’ average cost. The weights assigned to costs and beliefs under the equilibrium strategies are non-monotone over time. We explain this by decomposing incentives into signalling and learning, and discuss implications for prices, quantities, and welfare.

The Chinese Warrant Bubble: A Fundamental Analysis

Based on a rational option pricing framework that incorporates short-selling and margin-trading constraints in the stock market, we present evidence that Chinese warrant prices, which are regarded as bubbles in the previous literature, can be explained by a new option pricing model. Based on the new model, we develop a warrant-price deviation measure to quantify unobserved demand for short-selling or margin trading due to market constraints. We empirically show that warrant-price deviation is driven mainly by underlying stock valuation. We conclude that the Chinese warrant market is better characterized by derivatives rather than a pure bubble detached from underlying assets.

Do Lending Relationships Affect Corporate Financial Policies?

This paper provides new evidence on how lending relationships impact firms’ financing and investment decisions. I find that lending relationships have a significant impact on leverage ratios, issuance choices, and the investment structures of relationship borrowers. The influence of relationships is heightened for financially constrained firms. I find a significant decrease in leverage, net debt issuing, and investment activity in the aftermath of lender‐specific shocks to lending relationships, including announcements of bank write‐downs and downgrades in banks’ credit ratings. My findings are robust to controlling for confounding effects that might arise due to unobserved demand and relationship changes.

Frictions in a Competitive, Regulated Market Evidence from Taxis

This paper presents a dynamic general equilibrium model of a taxi market. The model is estimated using data from New York City yellow cabs. Two salient features by which most taxi markets deviate from the efficient market ideal is the need of both market sides to physically search for trading partners in the product market as well as prevalent regulatory limitations on entry in the capital market. To assess the relevance of these features we use the model to simulate the effect of changes in entry and an alternative search technology. The results are contrasted with a policy that improves the intensive margin of medallion utilization through a transfer of medallions to more efficient ownership. We use the geographical features of New York City to back out unobserved demand through a matching simulation.

Capacity and Utilization in Health Care: The Effect of Empty Beds on Neonatal Intensive Care Admission.

Because geographic variation in medical care utilization is jointly determined by both supply and demand, it is difficult to empirically estimate whether capacity itself has a causal impact on utilization in health care. In this paper, I exploit short-term variation in Neonatal Intensive Care Unit (NICU) capacity that is unlikely to be correlated with unobserved demand determinants. I find that available NICU beds have little to no effect on NICU utilization for the sickest infants, but do increase utilization for those in the range of birth weights where admission decisions are likely to be more discretionary.

Capacity and Utilization in Health Care: The Effect of Empty Beds on Neonatal Intensive Care Admission

Because geographic variation in medical care utilization is jointly determined by both supply and demand, it is difficult to empirically estimate whether capacity itself has a causal impact on utilization in health care. In this paper, I exploit short-term variation in Neonatal Intensive Care Unit (NICU) capacity that is unlikely to be correlated with unobserved demand determinants. I find that available NICU beds have little to no effect on NICU utilization for the sickest infants, but do increase utilization for those in the range of birth weights where admission decisions are likely to be more discretionary.

Behavioural model uncertainty in estimation of structural oligopoly models

The focus of this paper is on developing a methodology for dealing with behavioural model uncertainty in structural oligopoly models. It is well recognised that being an essential part of the identification strategy, the particular choice of a behavioural model embodies a highly influential, yet largely arbitrary, set of assumptions in the structural framework. The methods developed here are founded in Bayesian model averaging techniques and provide a practically and conceptually desirable way of accommodating behavioural model uncertainty in structural estimation. Moreover, a substantial feature of this approach is that it yields straightforward model comparison through the model posterior distribution. These methods are applied to estimate the parameters of the industry demand curve and firms' cost functions in oligopoly markets (e.g., marginal costs, markups, etc.). Three models of oligopoly behaviour are considered: one non-cooperative and two variations of cooperative with unobserved demand shocks. The specific industry analysed is the 1800s railroad cartel, commonly known as the Joint Executive Committee, which is widely familiar to industrial organisations economists. The results indicate that the algorithm performs quite well in correctly identifying cooperative behaviour, in additional to offering a clear view of the way in which model averaging resolves conflicts in inference arising from competing behavioural models.