Wrong Model Research Articles

Measuring model risk

Model risk as part of the operational risk is a serious problem for financial institutions. As the pricing of derivatives as well as the computation of the market or credit risk of an institution depend on statistical models the application of a wrong model can lead to a serious overor underestimation of the institution’s risk. Because the underlying data generating process is unknown in practice evaluating the model risk is a challenge. So far, definitions of model risk are either application-oriented including risk induced by the statistician rather than by the statistical model or research-oriented and too abstract to be used in practice. Especially, they are not data-driven. We introduce a data driven notion of model risk which includes the features of the research-oriented approach by extending it by a statistical model building procedure and therefore compromises between the two definitions at hand. We furthermore suggest the application of robust estimates to reduce the model risk and advocate the application of stress tests with respect to the valuation of the portfolio. JEL-numbers: C50, G32

Exploring some issues in acoustic full waveform inversion

ABSTRACTThe least‐squares error measures the difference between observed and modelled seismic data. Because it suffers from local minima, a good initial velocity model is required to avoid convergence to the wrong model when using a gradient‐based minimization method. If a data set mainly contains reflection events, it is difficult to update the velocity model with the least‐squares error because the minimization method easily ends up in the nearest local minimum without ever reaching the global minimum.Several authors observed that the model could be updated by diving waves, requiring a wide‐angle or large‐offset data set. This full waveform tomography is limited to a maximum depth. Here, we use a linear velocity model to obtain estimates for the maximum depth. In addition, we investigate how frequencies should be selected if the seismic data are modelled in the frequency domain. In the presence of noise, the condition to avoid local minima requires more frequencies than needed for sufficient spectral coverage.We also considered acoustic inversion of a synthetic marine data set created by an elastic time‐domain finite‐difference code. This allowed us to validate the estimates made for the linear velocity model. The acoustic approximation leads to a number of problems when using long‐offset data. Nevertheless, we obtained reasonable results. The use of a variable density in the acoustic inversion helped to match the data at the expense of accuracy in the inversion result for the density.

Detecting Outliers in Factor Analysis Using the Forward Search Algorithm

In this article we extend and implement the forward search algorithm for identifying atypical subjects/observations in factor analysis models. The forward search has been mainly developed for detecting aberrant observations in regression models (Atkinson, 1994) and in multivariate methods such as cluster and discriminant analysis (Atkinson, Riani, & Cerioli, 2004). Three data sets and a simulation study are used to illustrate the performance of the forward search algorithm in detecting atypical and influential cases in factor analysis models. The first data set has been discussed in the literature for the detection of outliers and influential cases and refers to the grades of students on 5 exams. The second data set is artificially constructed to include a cluster of contaminated observations. The third data set measures car's characteristics and is used to illustrate the performance of the forward search when the wrong model is specified. Finally, a simulation study is conducted to assess various aspects of the forward search algorithm.

Calculations of the energy distribution from perturbation peak data—A new tool for characterization of chromatographic phases

The calculation of the adsorption energy distribution (AED) was recently introduced as an important tool for the chromatographic community for characterization of modern phases. The AED-calculations, provides model-independent information about the numbers of different adsorption sites and their respective energy-levels, prior to the selection of an adsorption isotherm model which narrows the number of possible rival models. The selection of a proper model for the fitting of the determined raw data is crucial; if the wrong model is selected misleading information about the retention mechanism may be drawn. The AED-calculations require raw adsorption isotherm data (i.e. data points) which is unfortunately not obtained by the newly validated perturbation peak method. In this study, we developed mathematical expression allowing the use of the raw tangential slope provided by the perturbation peak method for AED calculations. The approach worked excellently and was verified against both computer-generated adsorption isotherm data as well as experimentally determined data, using three different experimental systems. It was found that the calculations of the AED, as based on perturbation peak data, converts faster and are not more sensitive to experimental noise as compared to the classical AED calculations using raw adsorption isotherm data.

Correcting for spatial autocorrelation in sequential sampling

1 Sequential sampling is attractive because it permits the user to choose, and efficiently achieve, desired confidence interval lengths. Sequential sampling has been broadly applied in the inventory of ecological resources. 2 Using case studies and simulations, we demonstrate that estimates of the population mean that are derived from sequential sampling can be overconfident if the population is autocorrelated, that is, that the confidence intervals are too short. We test a model-based correction designed to ameliorate this effect of autocorrelation upon the estimates of confidence intervals from sequential sampling. 3 The correction is useful in realistic situations. Among the scenarios we tested, better confidence interval coverage was achieved with larger sample sizes, and coverage rates were poor at smaller sample sizes. Nominal coverage could be attained even when the wrong model was used, although only at the cost of requiring a much higher average sample size. 4 Synthesis and applications. If sequential sampling is naively applied in a population that has autocorrelation, then confidence intervals for population parameters will be too short, and the usefulness of the sample will be overestimated. We recommend using a correction to lengthen the estimated confidence intervals. Our results suggest that this correction requires a substantial sample size, up to several hundred units, in order to provide nominal coverage. Sequential sampling seems risky in autocorrelated populations if the realized sample size is small.

Viewpoint: Response to ‘Fifty Years Using the Wrong Model of Advertising’

Viewpoint: Response to ‘Fifty Years Using the Wrong Model of Advertising’

Impact of an error in the column hold-up time for correct adsorption isotherm determination in chromatography: II. Can a wrong column porosity lead to a correct prediction of overloaded elution profiles?

The adsorption isotherm was determined for phenol in methanol/water on a C-8 stationary phase using frontal analysis in staircase mode, assuming different total column porosities, from 1 to 87%. Each set of adsorption isotherm data, with a certain column porosity, was fitted to various adsorption models and the generated parameters were used to calculate overloaded elution band profiles that were compared with experiments. It was found that the bi-Langmuir model had an optimum fit for a porosity that corresponds well with the value found experimentally. The adsorption energy distribution (AED) calculations and error analysis confirmed a bimodal energy distribution. It was also found that band profiles can be accurately predicted with a quite arbitrary chosen porosity, under prerequisite that a wrong but flexible adsorption model is chosen instead of the correct one. The latter result is very useful for quick optimizations of preparative separations where the exact value of the column porosity is not available.

Fifty Years Using the Wrong Model of Advertising

This paper investigates the dominance of the information processing model in TV advertising. Despite theoretical and empirical evidence that supports the importance of factors such as emotional content and creativity, the authors show that a rational information-based persuasion model, which pre-dates the development of formal marketing, persists in its domination of almost all TV advertising development and evaluation. It is postulated that this persistence derives from a sociological desire to maintain a positivist worldview of simplistic, well-ordered value systems operated by rational and predictable consumers. The authors suggest that both advertisers and researchers need to adopt a Critical Realism perspective in order to move beyond the philosophical straitjacket of this information processing model, and they summarise the implications that this has for current research practice.

Use of alternative conceptual models to assess the impact of a buried valley on groundwater vulnerability

A buried valley incised into a sequence of pre-Quaternary sediments is shown to seriously affect the vulnerability of groundwater. Often the existence of buried valleys is not known or is not described explicitly in a hydrogeological model. In the present study, two numerical groundwater models, representing two alternative conceptual models, were produced to help quantify the effect of the valley on groundwater vulnerability. One model included the buried valley and the other did not. Both models were subjected to calibration and were found to describe hydraulic head and river discharge equally well. Even though the two models showed similar calibration statistics; fluxes, travel paths and travel times were affected by the inclusion of the buried valley. The recharge area and the groundwater age of potential abstraction wells placed in the pre-Quaternary deep aquifers surrounding the buried valley were different for the two models, with significantly higher vulnerability when the valley was included in the model. Based on the results of the present study, it is concluded that a buried valley may not always be detectable when calibrating a wrong conceptual model. If reliable results should be obtained a good geological model has to be constructed.

Distributional effects of educational improvements: Are we using the wrong model?

Measuring the incidence of public spending in education requires an intergenerational framework distinguishing between what current and future generations—that is, parents and children—give and receive. In standard distributional incidence analysis, households are assumed to receive a benefit equal to what is spent on their children enrolled in the public schooling system and, implicitly, to pay a fee proportional to their income. We show that, in an intergenerational framework, this is equivalent to assuming perfectly altruistic individuals, in the sense of the dynastic model, and perfect capital markets. But in practice, credit markets are imperfect and poor households cannot borrow against the future income of their children. We show that under such circumstances, standard distributional incidence analysis may greatly over-estimate the progressivity of public spending in education: educational improvements that are progressive in the long-run steady state may actually be regressive for the current generation of poor adults. This is especially true where service delivery in education is highly inefficient—as it is in poor districts of many developing countries—so that the educational benefits received are relatively low in comparison with the cost of public spending. Our results have implications for both policy measures and analytical approaches.

Journals — the wrong model for Africa?

Journals — the wrong model for Africa?

Optimal portfolios when volatility can jump

We consider an asset allocation problem in a continuous-time model with stochastic volatility and jumps in both the asset price and its volatility. First, we derive the optimal portfolio for an investor with constant relative risk aversion. The demand for jump risk includes a hedging component, which is not present in models without volatility jumps. We further show that the introduction of derivative contracts can have substantial economic value. We also analyze the distribution of terminal wealth for an investor who uses the wrong model, either by ignoring volatility jumps or by falsely including such jumps, or who is subject to estimation risk. Whenever a model different from the true one is used, the terminal wealth distribution exhibits fatter tails and (in some cases) significant default risk.

Statistical Analysis of Visual Fits: Answer to J. Ninio

REPLY: We thank Dr. Ninio for his interest in our 25-yr-old publications and for his attempt to analyze our data and interpretations using a strategy different from the one we used for this pioneering work. However, his approach is exclusively concerned with curve fitting. It avoids the more demanding problem of the recovery of parameters of explicit statistical models of biological processes. That is, he decided to discriminate between fits of our published histograms visually, or “by eye.” The field long ago progressed to the point where the standard for confronting data with models is statistical analysis, not visual fitting. Reevaluation of old methods can be a good thing (Lowen et al. 1997), provided there is a sound scientific questions at stake. Yet, we are surprised that Dr. Ninio raises concerns about the validity of aspects of these seminal papers but does not want to review the subsequent literature that appeared about our results and conclusions. This is difficult to understand because the aim of a meaningful scientific approach is to progress in the understanding of natural phenomena and processes. In this respect, it is worth noting that subsequent research carried out independently by a number of eminent scientists supported our proposal of the “One Vesicle Hypothesis” of transmitter release at several synapses in the CNS. In this response we address the flaws in the logic of Ninio’s analysis and his criticisms and misunderstandings of our published work. Because of the progress in this field, and given that Dr. Ninio is not an expert in this area, we believe our response here will be of interest to other investigators who are concerned with using rigorous methods to extract information from experimental distributions. BACKGROUND Experimental context We performed simultaneous intracellular recordings and dye injections, in vivo, to collect data sets of fluctuating inhibitory postsynaptic potentials (IPSPs) evoked in the Mauthner cell of teleost fish by single presynaptic action potentials and used an optimization procedure to determine the best-fitting binomial and Poisson descriptions of the data. We then compared those results with histological reconstructions of the presynaptic cells, including the axonal branching pattern and the identification of contacts with the postsynaptic neuron. This was one of the earliest attempts to obtain such structure‐function correlations. We were particularly interested in two questions that had been raised previously: 1) Were the fluctuations in PSP amplitude better described by a binomial or a Poisson distribution? and 2) Could the statistical parameters be correlated with physical structures?

Case Study

Three major reviews of the assessment center (AC) construct-validity literature have disagreed as to the most appropriate analytic model for AC postexercise dimension ratings. We report a Monte Carlo study addressing the following questions: (a) To what extent does the “true” model (i.e., the model that generated the data) actually appear to fit the data well? (b) To what extent can a model appear to fit the data well even though it is the wrong model? and (c) Is model fit actually a useful empirical criterion for judging which model is most likely the population model? Results suggest that “true” models may not always appear as the best fitting models, whereas “false” models sometimes appear to offer better fit than the true models. Implications for the study of AC construct validity are discussed.

Americans Sweep Nobel Prizes: Thinking Inside and Outside the Box Wonder — is not precisely Knowing And not precisely Knowing not—

Americans Sweep Nobel Prizes: Thinking Inside and Outside the Box Wonder — is not precisely Knowing And not precisely Knowing not—

Quality control of risk measures: backtesting VAR models

measures: quality control of risk measures (QCRM). The approach is applied to the problem of backtesting value-at-risk (VAR) models. VAR models are used to predict the maximum likely losses in a bank’s portfolio at a specified confidence level and time horizon. The widely accepted VAR backtesting procedure outlined by the Basel Committee for Banking Supervision controls the probability of rejecting the model when the model is correct. A drawback of the Basel approach is its limited power to control the probability of accepting an incorrect VAR model. By exploiting the binomial structure of the testing problem, QCRM provides a more balanced testing procedure, which results in a uniform reduction of the probability of accepting a wrong model. QCRM consists of three elements: the first is a new hypothesis-testing problem in which the null and alternative hypotheses are exchanged to control the probability of accepting an inaccurate model. The second element is a new approach for comparing the power of the QCRM and Basel tests in terms of the probability of rejecting correct and incorrect models. The third element involves the use of the technique of pivoting the cumulative distribution function to obtain one-sided confidence intervals for the probability of an exception. The use of these confidence intervals results in new acceptance/rejection regions for tests of the VAR model. We compare these to ones commonly used in the financial literature.

The effect of miss-specified baseline characteristics on inference for longitudinal trends in linear mixed models

The main advantage of longitudinal studies is that they can distinguish changes over time within individuals (longitudinal effects) from differences among subjects at the start of the study (baseline characteristics, cross-sectional effects). Often, especially in observational studies, longitudinal trends are studied after correction for many potentially important baseline differences between subjects. We show that, in the context of linear mixed models, inference for longitudinal trends is in general biased if a wrong model for the baseline characteristics is used. However, we will argue that this bias is small in most practical situations and completely vanishes in the special case of a growth curve model for complete balanced data. In the latter case, inference for longitudinal trends is completely independent of additional baseline covariates that might have been omitted from the model.

Estimating the Wrong Graphical Model: Benefits in the Computation-Limited Setting

Consider the problem of joint parameter estimation and prediction in a Markov random field: that is, the model parameters are estimated on the basis of an initial set of data, and then the fitted model is used to perform prediction (e.g., smoothing, denoising, interpolation) on a new noisy observation. Working under the restriction of limited computation, we analyze a joint method in which the same convex variational relaxation is used to construct an M-estimator for fitting parameters, and to perform approximate marginalization for the prediction step. The key result of this paper is that in the computation-limited setting, using an inconsistent parameter estimator (i.e., an estimator that returns the wrong model even in the infinite data limit) is provably beneficial, since the resulting errors can partially compensate for errors made by using an approximate prediction technique. En route to this result, we analyze the asymptotic properties of M-estimators based on convex variational relaxations, and establish a Lipschitz stability property that holds for a broad class of convex variational methods. This stability result provides additional incentive, apart from the obvious benefit of unique global optima, for using message-passing methods based on convex variational relaxations. We show that joint estimation/prediction based on the reweighted sum-product algorithm substantially outperforms a commonly used heuristic based on ordinary sum-product.

Tests of alternate models for the pricing of Korean Treasury bond futures contracts

It is known that KTB futures contracts are significantly underpriced when the model price is calculated using the ad hoc cost-of-carry model employed in industry. This paper examines whether this underpricing phenomenon is caused by using the wrong model to price the futures contracts. This paper documents that the difference between the model price and the market price of KTB futures decreases substantially if the correct term–structure-based model is used to estimate the model price of KTB futures. In addition, even though the underpricing phenomenon can be exploited to generate some trading profits, the profits cannot be regarded as arbitrage profits. Thus, we believe that the underpricing phenomenon is illusory, and that much of it can be attributed to the wrong model being used in the industry.

Does the Choice of Model or Benchmark Affect Inference in Measuring Mutual Fund Performance?

We address the practical question of whether investors and researchers are likely to make invalid inferences about fund manager performance when using the wrong model and/or benchmark. We consider three well-known models, those of Jensen (1968), Treynor and Mazuy (1966), and Henriksson and Merton (1981), and two commonly used timing benchmarks, the SP (2) but biases in measures of overall performance are economically insignificant; (3) benchmark misspecification results in qualitatively similar difficulties, with the addition that overall performance as well can be biased; and (4) model and benchmark misspecification do not appreciably alter the power to detect ability and distinguish a good fund from a bad fund. These results are robust to alternative asset pricing specifications, alternative simulation schemes, varying length of the return series, and periodicity of the simulated series. The use of daily fund returns amplifies our conclusions about the biases induced by model misspecifications. Moreover, the biases we identify appear to be difficult to correct by using standard model selection criteria and misspecification tests. If the benchmark is known but the timing model is not, investors should use measures of overall performance to evaluate funds and managers.