Statistical Analysis Of Historical Data Research Articles

The Implications of Banks’ Credit Risk Modeling Disclosures for Their Loan Loss Provision Timeliness and Loan Origination Procyclicality

We identify two credit risk modeling (CRM) activities from disclosures in banks’ 1995-2009 financial reports: (1) statistical analysis of historical data on underwriting criteria, loan performance statuses, and relevant economic variables (MODEL); and (2) stress testing of credit losses to possible adverse future events (STRESS). We expect MODEL to discipline banks’ loan origination and loan loss provisions (LLPs) for homogeneous loans during stable economic times, but to be limited for heterogeneous loans and for all loans during sharp economic downturns, when STRESS is essential. We predict and find that banks engaging in MODEL exhibit timelier LLPs during stable economic times, particularly for homogeneous loans, and also late in the financial crisis after data on elevated credit losses had accumulated, but not early in the financial crisis. We further predict and find that these banks exhibit less procyclical loan originations, particularly for homogeneous loans. We predict and find that banks engaging in STRESS exhibit timelier LLPs for both homogeneous and heterogeneous loans during recessions, including early in the financial crisis. We further predict and find that these banks exhibit less procyclical loan originations for both loan types during recessions.

Virtual machine consolidation enhancement using hybrid regression algorithms

Cloud computing data centers are growing rapidly in both number and capacity to meet the increasing demands for highly-responsive computing and massive storage. Such data centers consume enormous amounts of electrical energy resulting in high operating costs and carbon dioxide emissions. The reason for this extremely high energy consumption is not just the quantity of computing resources and the power inefficiency of hardware, but rather lies in the inefficient usage of these resources. VM consolidation involves live migration of VMs hence the capability of transferring a VM between physical servers with a close to zero down time. It is an effective way to improve the utilization of resources and increase energy efficiency in cloud data centers. VM consolidation consists of host overload/underload detection, VM selection and VM placement. Most of the current VM consolidation approaches apply either heuristic-based techniques, such as static utilization thresholds, decision-making based on statistical analysis of historical data; or simply periodic adaptation of the VM allocation. Most of those algorithms rely on CPU utilization only for host overload detection. In this paper we propose using hybrid factors to enhance VM consolidation. Specifically we developed a multiple regression algorithm that uses CPU utilization, memory utilization and bandwidth utilization for host overload detection. The proposed algorithm, Multiple Regression Host Overload Detection (MRHOD), significantly reduces energy consumption while ensuring a high level of adherence to Service Level Agreements (SLA) since it gives a real indication of host utilization based on three parameters (CPU, Memory, Bandwidth) utilizations instead of one parameter only (CPU utilization). Through simulations we show that our approach reduces power consumption by 6 times compared to single factor algorithms using random workload. Also using PlanetLab workload traces we show that MRHOD improves the ESV metric by about 24% better than other single factor regression algorithms (LR and LRR). Also we developed Hybrid Local Regression Host Overload Detection algorithm (HLRHOD) that is based on local regression using hybrid factors. It outperforms the single factor algorithms.

Perspectives on animal research and its application

Return on investment from animal research in Australia is about half that from investment in crop research. Return on investment in animal research and its application could be enhanced by improving (1) selection of areas for research, (2) application of existing knowledge and (3) adherence to the scientific method. (1) The flat response curve in outputs to changing inputs around the optimum for most agricultural systems means that research investments must be directed towards new technologies rather than refining existing technologies. New technologies have been responsible for advances in all human endeavours. Many new technologies are derived from curiosity research, but processes can also be adopted to identify specific industry-related research priorities. These processes include statistical analysis of historical data, critical evaluation of factors limiting enterprise productivity and profitability, and application of simulation models. These methods can identify changes in industry practice that would have marked effects on productivity. A thorough review of the literature is needed to determine whether the change could be achieved through adoption of existing knowledge or whether new research is required. (2) Adoption of existing knowledge is frequently poor because of the way it is presented to enterprise managers and the perceived risks from changing practices. A method is described to improve adoption, while controlling the risks associated with higher productivity. The method targets the correct and consistent application of a small number of farm practices, which, if not undertaken precisely, would have a major impact on productivity or sustainability. Application of the method has been highly successful in improving enterprise productivity and profitability. However, these improvements tend to wane over time because of the rigour involved and monotony associated with repeated actions. Sustained improvements in productivity should come in the future as more electronic-based technologies are employed to take essential measurements, interpret the information collected and automatically control systems through real-time processing. (3) Other contributors to a poor return on research investment are lack of strict adherence to the scientific method, inadequate experimental design and insufficient application of statistics.

Innovative application of LSS in aircraft maintenance environment

Purpose – The purpose of this paper is to eliminate the wastes and inefficient procedures in the maintenance organization of aircraft so as to reduce its downtime and increase mission availability. Design/methodology/approach – Customized lean Six Sigma (LSS) was applied at the task level and servicing cycle level to reduce the task content, cycle length and resources in servicing. The loading of the servicing facility was simulated through a simulation program developed from a statistical analysis of historical data for validating/simulating/determining optimum loading of servicing facility with refined tasks, reduced cycle length and resources. In simulation, the optimum combination of manpower, resources and infrastructure at the facility level was determined through sensitive analysis and design of experiments (DoE). Findings – Optimization at the task level and its re-organization at the servicing cycle level reduced the cycle length by 55-68 per cent and manpower resources by 26 per cent. This further reduced facility-level manpower by 25 to 40 per cent, capacity requirements by more than 33 per cent and annual aircraft downtime by 78 per cent. The approach reduced the average number of aircraft undergoing servicing at each airbase at any time from 2.35 to just 0.76 and increased the mission availability to 20 per cent. Originality/value – The hallmark of the paper has been the design of LSS approach from structured historical data and its validation through innovative simulation. The multi-pronged bottom-up approach practically bundles all wastes resident in the maintenance organization. The paper provides cursory approach to lean practitioners in the elimination of wastes in the maintenance of capital assets like aircraft.

Compliance with national guidelines for stroke in radiology

Stroke is a medical emergency, and if patient outcomes are to be optimised there should be no delays in accessing treatment. This project focuses on the application of Operational Research methodology to investigate how a hospital can comply with the revised computerised tomography (CT) scanning guidelines for stroke. Such guidelines, released by the Royal College of Physicians recommend a 50% reduction in time from hospital admission to report of a CT head scan to just 12 hours. The results of statistical analyses of historical data were used to populate a discrete event simulation model of patient flow through the CT scanning unit. The model was then used to explore a number of operational modifications to the CT scanning system through a series of scenario analyses. The results of this investigation presented evidence of a number of strategies to support operational improvements in relation to revised stroke guidelines.

An Adaptive Learning Approach for Efficient Resource Provisioning in Cloud Services

The emerging cloud computing service market aims at delivering computing resources as a utility over the Internet with a high quality. It has evolving unknown demand that is typically highly uncertain. Traditional provisioning methods either make idealized assumption of the demand distribution or rely on extensive offline statistical analysis of historical data. In this paper, we present an online adaptive learning approach to address the optimal resource provisioning problem. Based on a stochastic loss model of the cloud services, we formulate the provisioning problem from a revenue management perspective, and present a stochastic gradient-based learning algorithm that adaptively adjusts the provisioning solution as observations of the demand are continuously made. We show that our adaptive learning algorithm guarantees optimality and demonstrate through simulation that they can adapt quickly to non-stationary demand.

A characterization of hierarchical computable distance functions for data warehouse systems

A data warehouse is a huge multidimensional repository used for statistical analysis of historical data. In a data warehouse events are modeled as multidimensional cubes where cells store numerical indicators while dimensions describe the events from different points of view. Dimensions are typically described at different levels of details through hierarchies of concepts. Computing the distance/similarity between two cells has several applications in this domain. In this context distance is typically based on the least common ancestor between attribute values, but the effectiveness of such distance functions varies according to the structure and to the number of the involved hierarchies. In this paper we propose a characterization of hierarchy types based on their structure and expressiveness, we provide a characterization of the different types of distance functions and we verify their effectiveness on different types of hierarchies in terms of their intrinsic discriminant capacity.

Data-driven optimized vehicle-level engineering specifications

Purpose – The purpose of this paper is to propose a methodology that determines vehicle-level specifications for new-car program by balancing market environments and engineering feasibility in the early stages of the vehicle development processes using statistical analysis of historical data. Design/methodology/approach – The proposed methodology effectively captures the interplay among key factors in preliminary vehicle planning: engineering feasibility constraints, market demands, and economic conditions. Engineering design constraints, derived by statistical analysis of historical data, define the strategic feasible space. Within the defined design space, the methodology determines a set of specifications that maximize the customer utility which is built as a function of preferences on each attribute of a vehicle. Findings – The present paper develops an “extrapolation” approach using historical vehicle data, rather than attempt to model a complex system with limited information. In doing so, the proposed approach avoids the difficulties of understanding an entire complex system by determining only the moderate level of specifications. Moreover, its quantification of revealed customer preferences as expressed in sales data resolves the confusions in vehicle planning arising from the translation of customer requirements to engineering specification. Originality/value – The proposed methodology can provide feasible prediction values with a new, historical-data-based statistical approach that effectively surmounts the difficulty of mechanically understanding complex systems. Moreover, through quantification of the target market's customer requirements as well as effects of market-environmental changes, the methodology enables designers to plan complex products for new concept in objective and reasonable manner.

A Method for Improving the Accuracy of Weather Forecasts Based on a Comprehensive Statistical Analysis of Historical Data for the Contiguous United States

Using historical weather forecast data downloaded from the National Oceanic and Atmospheric Administration’s [NOAA] National Weather Service Digital Library, we performed statistical analysis on the forecast accuracies of temperature, probability of precipitation, quantitative precipitation and wind speed. The major findings of this study are: (1) There are significant variations in forecast accuracies at different geographical locations in the United States; (2) The overall accuracies of 3-day or longer temperature forecasts are similar in magnitude to the standard deviations of historical daily changes in temperature; (3) There are statistically significant biases in the forecasts of either large positive or negative changes in temperatures; (4) The observed probabilities of precipitation are significantly lower than forecasted probabilities for 2-day or longer horizons; (5) On the average, the 3-day or longer forecasts for quantitative precipitation tend to significantly under estimate actual amount for periods of heavy precipitation; and (6) Forecasters generally under predict wind speeds by a large margin for days when wind speeds exceed 20 mph. An improved weather forecast model can be constructed based on some of the empirical statistical parameters from this study.

Managing Overloaded Hosts for Dynamic Consolidation of Virtual Machines in Cloud Data Centers under Quality of Service Constraints

Dynamic consolidation of virtual machines (VMs) is an effective way to improve the utilization of resources and energy efficiency in cloud data centers. Determining when it is best to reallocate VMs from an overloaded host is an aspect of dynamic VM consolidation that directly influences the resource utilization and quality of service (QoS) delivered by the system. The influence on the QoS is explained by the fact that server overloads cause resource shortages and performance degradation of applications. Current solutions to the problem of host overload detection are generally heuristic based, or rely on statistical analysis of historical data. The limitations of these approaches are that they lead to suboptimal results and do not allow explicit specification of a QoS goal. We propose a novel approach that for any known stationary workload and a given state configuration optimally solves the problem of host overload detection by maximizing the mean intermigration time under the specified QoS goal based on a Markov chain model. We heuristically adapt the algorithm to handle unknown nonstationary workloads using the Multisize Sliding Window workload estimation technique. Through simulations with workload traces from more than a thousand PlanetLab VMs, we show that our approach outperforms the best benchmark algorithm and provides approximately 88 percent of the performance of the optimal offline algorithm.

Real-Time Tactical and Strategic Sales Management for Intelligent Agents Guided by Economic Regimes

Many enterprises that participate in dynamic markets need to make product pricing and inventory resource utilization decisions in real time. We describe a family of statistical models that addresses these needs by combining characterization of the economic environment with the ability to predict future economic conditions to make tactical (short-term) decisions, such as product pricing, and strategic (long-term) decisions, such as level of finished goods inventories. Our models characterize economic conditions, called economic regimes, in the form of recurrent statistical patterns that have clear qualitative interpretations. We show how these models can be used to predict prices, price trends, and the probability of receiving a customer order at a given price. These “regime” models are developed using statistical analysis of historical data and are used in real time to characterize observed market conditions and predict the evolution of market conditions over multiple time scales. We evaluate our models using a testbed derived from the Trading Agent Competition for Supply Chain Management, a supply chain environment characterized by competitive procurement, sales markets, and dynamic pricing. We show how regime models can be used to inform both short-term pricing decisions and long-term resource allocation decisions. Results show that our method outperforms more traditional short- and long-term predictive modeling approaches.

Planning level tunnel cost estimation based on statistical analysis of historical data

Estimating the construction cost of a tunnel with certain diameter and length during the planning stage of a project is essential and a “must do” to justify the project, but it is a very difficult task. Numerous variables influence the cost of tunnel construction and limited information is available at early stages of design when the possible use of tunnels is being studied. Therefore, there is limited number of models at engineers’ disposal for development of a proper cost estimate in tunneling projects. The focus of this study was to offer a model for estimation of the construction cost of tunnels for various applications in the preliminary stage of a project. The work is based on study of nearly 270 projects and statistical analysis of the recoded construction costs, tunnel size, ground conditions, and tunnel applications. Several cost estimation models are introduced for various tunnel types including waste water, water, and transportation tunnels in soft ground and rock. The model uses commonly available cost indices (CCI and BCI) to adjust the estimated costs for the time and location of the project. The proposed cost models presented in this study can included in a simple program to help users make more accurate and quick cost estimates for their future projects. The models do not deal with construction risk which is an essential part of the study of various methods and should be done through developing a risk registry with related cost of each case before final selection, design, and cost estimation of any tunneling project.

A Bivariate Model for Deman and Spot Price of Electricity

In this paper bivariate modelling methodology, solely applied to the spot price of electricity or demand for electricity in earlier studies, is extended to a bivariate process of spot price of electricity and demand for electricity. The suggested model accommodates common idiosyncrasies observed in deregulated electricity markets such as cyclical trends in price and demand for electricity, occurrence of extreme spikes in prices, and mean-reversion effect seen in settling of prices from extreme values to the mean level over a short period of time. The paper presents detailed statistical analysis of historical data of daily averages of electricity spot prices and corresponding demand for electricity. The data is obtained from the NSW section of Australian Energy Markets.

A Study of the Cascading Effects of Ambulance Diversion among Hospitals

This research effort is undertaken to determine the impact that one hospital’s diversion status has on other hospitals in a region and the strength of these interactions. The conditional probability of one hospital going on diversion given that another is already on diversion is evaluated. Based on this analysis, the strength of interactions among the hospitals is established. Through statistical analyses of historical data, the strength of the mutual effects of diversion among a collection of hospitals is determined. These effects are mutual if one hospital’s diversion status affected another’s, then the reverse was also true. The intensity of these interactions between hospitals is varied, some being stronger than others. The model illustrates an approach to studying the cascading effects of diversion among hospitals in a region. This is important, because the status of any hospital in a region can signal the likelihood of impending diversion in every other hospital in the region. This allows actions that might prevent the occurrence of diversion or mitigate the cascading effects of Emergency Medical Systems diversion.

What can be learnt from an ecotoxicity database in the framework of the REACh regulation?

Since REACh applies in all of EU, special emphasis has been put on the reduction of systematic ecotoxicity testing. In this context, it is important to extract a maximum of information from existing ecotoxicity databases in order to propose alternative methods aimed at replacing and reducing experimental testing. Consequently, we analyzed a database of new chemicals registered in France and Europe during the last twenty years reporting aquatic ecotoxicity data with respect to three trophic levels ( i.e., Algae EC50 72 h, Daphnia EC50 48 h and Fish LC50 96 h). In order to ensure the relevance of the comparison between these three experimental tests, we performed a stringent data selection based on the pertinence and quality of available ecotoxicological information. At the end of this selection, less than 5% of the initial number of chemicals was retained for subsequent analysis. Such an analysis showed that fish was the least sensitive trophic level, whereas Daphnia had the highest sensitivity. Moreover, thanks to an analysis of the relative sensitivity of trophic levels, it was possible to establish that respective correction factors of 50 and 10 would be necessary if only one or two test values were available. From a physicochemical point of view, it was possible to characterize two significant correlations relating the sensitivity of the aforementioned trophic levels with the chemical structure of the retained substances. This analysis showed that algae displayed a higher sensitivity towards chemicals containing acid fragments whereas fish presented a higher sensitivity towards chemicals containing aromatic ether fragments. Overall, our work suggests that statistical analysis of historical data combined with data yielded by the REACh regulation should permit the derivation of robust safety factors, testing strategies and mathematical models. These alternative methods, in turn, could allow a replacement and reduction of ecotoxicological testing.

Management Insights

Management Insights

Practical Application of a Probabilistic Approach to Estimate Reserves Using Production Decline Data

Summary Analysts have increasingly used probabilistic approaches to evaluate the uncertainty in reserves estimates based on decline-curve analysis (DCA). This is because the results represent statistical analysis of historical data that usually possess significant amounts of noise. Probabilistic approaches usually provide a distribution of reserves estimates with three confidence levels (P10, P50, and P90) and a corresponding 80% confidence interval (CI). The question arises: How reliable is this 80% CI? In other words, in a large set of analyses, is the true value of reserves contained within this interval 80% of the time? Our investigation indicates that it is common in practice for true values of reserves to lie outside 80% CIs much more than 20% of the time using traditional statistical analyses. This indicates that uncertainty is being underestimated, often significantly. Thus, the challenge in probabilistic reserves estimation using DCA is not only how to appropriately characterize probabilistic properties of complex production-data sets, but also how to determine and then improve the reliability of the uncertainty quantifications. This paper presents an improved methodology for probabilistic quantification of reserves estimates using DCA and practical application of the methodology to actual individual-well decline curves. Application of our proposed new method to 100 oil and gas wells demonstrates that it provides much wider 80% CIs than methods previously presented, and these CIs contain the true values approximately 80% of the time. In addition, the method yields more-accurate P50 values than previously published methods do. Thus, the new methodology provides more-reliable probabilistic reserves estimation, which has important impacts on economic risk analysis and reservoir management.

Avalanche hazard mapping over large undocumented areas

An innovative methodology to perform avalanche hazard mapping over large undocumented areas is herewith presented and discussed. The method combines GIS tools, computational routines, and statistical analysis in order to provide a “semi-automatic” definition of areas potentially affected by avalanche release and motion. The method includes two main modules. The first module is used to define zones of potential avalanche release, based on the consolidated relations on slope, morphology, and vegetation. For each of the identified zones of potential release, a second module, named Avalanche Flow and Run-out Algorithm (AFRA), provides an automatic definition of the areas potentially affected by avalanche motion and run-out. The definition is generated by a specifically implemented “flow-routing algorithm” which allows for the determination of flow behaviour in the track and in the run-out zone. In order to estimate the avalanche outline in the run-out zone, AFRA uses a “run-out cone”, which is a 3D projection of the angle of reach α. The α-value is evaluated by statistical analysis of historical data regarding extreme avalanches. Pre- and post-processing of the AFRA input/output data is done in an open source GIS environment (GRASS GIS). The method requires only a digital terrain model and an indication of the areas covered by forest as input parameters. The procedure, which allows rapid mapping of large areas, does not in principle require any site-specific historical information. Furthermore, it has proven to be effective in all cases where a preliminary cost-efficient analysis of the territories potentially affected by snow avalanche was needed.

Expected Return and the Value Premium

Empirical research has shown that within a cross-section of stocks, investor return increases with book-to-market ratio. The reason is in dispute. One side argues that the market is inefficient; the other, that value stocks are riskier. Statistical analysis of historical data has not resolved the issue. We take a theoretical approach, which suggests that the higher returns for value stocks can be explained without the introduction of risk. Our theory leads to an equation for expected return, which differs from realized return through elimination of a speculative noise factor. The theory defines fundamental factors that determine investor return, and is confirmed by empirical correlations reported by others.

Frequency of Adverse Events after Vaccination with Different Vaccinia Strains

BackgroundLarge quantities of smallpox vaccine have been stockpiled to protect entire nations against a possible reintroduction of smallpox. Planning for an appropriate use of these stockpiled vaccines in response to a smallpox outbreak requires a rational assessment of the risks of vaccination-related adverse events, compared to the risk of contracting an infection. Although considerable effort has been made to understand the dynamics of smallpox transmission in modern societies, little attention has been paid to estimating the frequency of adverse events due to smallpox vaccination. Studies exploring the consequences of smallpox vaccination strategies have commonly used a frequency of approximately one death per million vaccinations, which is based on a study of vaccination with the New York City Board of Health (NYCBH) strain of vaccinia virus. However, a multitude of historical studies of smallpox vaccination with other vaccinia strains suggest that there are strain-related differences in the frequency of adverse events after vaccination. Because many countries have stockpiled vaccine based on the Lister strain of vaccinia virus, a quantitative evaluation of the adverse effects of such vaccines is essential for emergency response planning. We conducted a systematic review and statistical analysis of historical data concerning vaccination against smallpox with different strains of vaccinia virus.Methods and FindingsWe analyzed historical vaccination data extracted from the literature. We extracted data on the frequency of postvaccinal encephalitis and death with respect to vaccinia strain and age of vaccinees. Using a hierarchical Bayesian approach for meta-analysis, we estimated the expected frequencies of postvaccinal encephalitis and death with respect to age at vaccination for smallpox vaccines based on the NYCBH and Lister vaccinia strains. We found large heterogeneity between findings from different studies and a time-period effect that showed decreasing incidences of adverse events over several decades. To estimate death rates, we then restricted our analysis to more-recent studies. We estimated that vaccination with the NYCBH strain leads to an average of 1.4 deaths per million vaccinations (95% credible interval, 0–6) and that vaccination with Lister vaccine leads to an average of 8.4 deaths per million vaccinations (95% credible interval, 0–31). We combined age-dependent estimates of the frequency of death after vaccination and revaccination with demographic data to obtain estimates of the expected number of deaths in present societies due to vaccination with the NYCBH and Lister vaccinia strains.ConclusionsPrevious analyses of smallpox vaccination policies, which rely on the commonly assumed value of one death per million vaccinations, may give serious underestimates of the number of deaths resulting from vaccination. Moreover, because there are large, strain-dependent differences in the frequency of adverse events due to smallpox vaccination, it is difficult to extrapolate from predictions for the NYCBH-derived vaccines (stockpiled in countries such as the US) to predictions for the Lister-derived vaccines (stockpiled in countries such as Germany). In planning for an effective response to a possible smallpox outbreak, public-health decision makers should reconsider their strategies of when to opt for ring vaccination and when to opt for mass vaccination.