Estimation Of Performance Measures Research Articles

Robust evidence on the similarity of Sharpe ratio and drawdown-based hedge fund performance rankings

In this article, we analyse whether the class of adequately defined drawdown-based performance measures produces hedge fund rankings similar to the one that can be obtained using the Sharpe ratio. Supported by a series of robustness checks, we find that the choice of performance measure does not matter if investors are simply interested in identifying the best hedge funds and if a sufficient return history is used to calculate performance measure estimates. In small time series sample sizes typically used to evaluate hedge funds, the rankings cannot be regarded as strictly identical. However, with an increasing time series dimension, the ranking differences fall considerably.

Estimation of Safety Performance Measures from Smartphone Sensors

Safety performance measures represent an useful tool for evaluating road safety conditions on the basis of objective parameters deducible from the vehicle kinematics. In this context, safety performances are expressed in terms of indicators representing interactions between different pairs of vehicles belonging to the traffic stream. Safety performance is expressed from the perspective of rear-end vehicle interactions. Differences in safety performance are discussed with respect to type of indicator and traffic conditions. When these indicators reach a certain critical value (threshold), a possible accident scenario is identified. Most common approaches used to acquire vehicle tracking data are based on video image processing algorithms and satellite navigation systems. However, many studies are increasingly interested in the emerging smartphone technologies for tracking people, and hence vehicles. Due to the fact that smartphones are becoming a valid alternative to Tablets, PDAs and laptops, offering phone features coupled with multiple mobile internet applications, smartphone sales will more than triple to 491.9 million units by 2012 from 139.3 million in 2008 (Gartner Inc. forecasts). The main goal of this study is to present a procedure for extracting vehicle tracking data from smartphone sensors and to use them in the estimation of safety performance indicators. The accuracy of tracking data from smartphone sensors is evaluated with respect to GPS tracking measurements. The results of this analysis identify interactions potentially dangerous and highlight high risk zones that reflect locations characterized by high vehicular interactions. This study underscores the usefulness of the smartphones for providing meaningful experimental data to assess potential safety problems.

Benchmarking Regions: Estimating the Counterfactual Distribution of Labor Market Outcomes

This paper develops and implements a new benchmarking approach for labor market regions. Based on panel data for regions, we use nonparametric matching techniques to account for observed labor market characteristics and for spatial proximity. As the benchmark, we estimate the counterfactual distribution of labor market outcomes for a region based on outcomes of similar regions. This allows to measure both the rank (relative performance) and the absolute performance based on the actual outcome for a region. Our outcome variable of interest is the hiring rate among the unemployed. We implement different similarity measures to account for differences in labor market conditions and spatial proximity, and we choose the tuning parameters in our matching approach based on a cross-validation procedure. The results show that both observed labor market characteristics and spatial proximity are important features to successfully match regions. Specifically, the modified Zhao (2004) distance measure and geographic distance in logs work best in our applications. Our estimated performance measures remain quite stable over time.

Benchmarking Regions: Estimating the Counterfactual Distribution of Labor Market Outcomes

This paper develops and implements a new benchmarking approach for labor market regions. Based on panel data for regions, we use nonparametric matching techniques to account for observed labor market characteristics and for spatial proximity. As the benchmark, we estimate the counterfactual distribution of labor market outcomes for a region based on outcomes of similar regions. This allows to measure both the rank (relative performance) and the absolute performance based on the actual outcome for a region. Our outcome variable of interest is the hiring rate among the unemployed. We implement different similarity measures to account for differences in labor market conditions and spatial proximity, and we choose the tuning parameters in our matching approach based on a cross-validation procedure. The results show that both observed labor market characteristics and spatial proximity are important features to successfully match regions. Specifically, the modified Zhao (2004) distance measure and geographic distance in logs work best in our applications. Our estimated performance measures remain quite stable over time.

Non-asymptotic end-to-end performance bounds for networks with long range dependent fBm cross traffic

Fractional Brownian motion (fBm) emerged as a useful model for self-similar and long-range dependent aggregate Internet traffic. Asymptotic, respectively, approximate performance measures are known for single queueing systems with fBm through traffic. In this paper end-to-end performance bounds for a through flow in a network of tandem queues under open-loop fBm cross traffic are derived. To this end, a rigorous sample path envelope for fBm is proven that complements previous approximate results. The sample path envelope and the concept of leftover service curves are employed to model the remaining service after scheduling fBm cross traffic at a queuing system. Using composition results for tandem systems from the stochastic network calculus end-to-end statistical performance bounds for individual flows in networks under fBm cross traffic are derived. The discovery is that these bounds grow in O n ( log n ) 1 / ( 2 - 2 H ) for n systems in series where H is the Hurst parameter of the cross traffic. Explicit results on the impact of the variability and the burstiness of through and cross traffic on network performance are shown. Our analysis has direct implications on fundamental questions in network planning and service management.

A Simulation-Based Closed Queueing Network Approximation of Semiconductor Automated Material Handling Systems

Queueing networks have been used to model material handling in flexible manufacturing systems. We explore the use of a closed queueing network model to approximate an intrabay automated material handling system (AMHS) in semiconductor manufacturing. A simulation-based approach is proposed to obtain estimates of system performance measures. Current simulation models in the industry are very complex, require significant development time, and take a long time to run. The proposed simulation-based queueing network approximation can be used as an easy and fast alternative or as a precursor to a full-fledged modeling exercise depending on the type of decision being made and the level of granularity required in the output. Using a simple intrabay configuration, we compare estimates from the approximation approach and a more detailed modeling approach to characterize the performance of the approximation. In addition, a simulation information model is used to compare AMHS models based on the amount and types of information needed to model the system, and to obtain desired output. This classification aids modelers in determining the level of detail to incorporate in a model based on the objectives of the simulation study.

Performance Evaluation and Dimensioning of GIX/M/c/N Systems through Kernel Estimation

We extend the analysis of queueing systems for real‐life situations, where the arrival pattern of customers is unknown. In real systems, we must understand how the choice of a method of estimation influences the configuration of the system. Using kernel smoothing, we evaluate algorithms to estimate performance measures of a GIX/M/c/N system, including the invariant probability distribution of the number of customers in the system, the blocking probability, the average queue size, and the average client queue time. We successfully apply the method to the arrivals to a call center to plan and improve the performance of these important queueing systems.

Analytical performance of soft clustering protocols

The success of a mobile ad hoc network (MANET) is strongly related to the protocol used at the medium access control (MAC) layer. Depending on the requirements and the specific network under concern, the protocol parameters at the MAC layer can be arbitrated to make best use of the channel resources. Typically, extensive simulation studies are used to find the best values for these variables. The problem with this approach is the need for excessive amounts of processing power and time. As the dimensions of the decision space increase, the need for processing power grows exponentially. This paper addresses this problem by developing an analytical model that reflects the relationships between protocol parameters and the overall performance of the protocol under various network conditions. Specifically, we model the MH-TRACE cluster-based protocol, which is capable of supporting real-time data transmission. The model is capable of estimating performance measures such as energy consumption and number of receptions while being simple enough to be run for a large set of parameters. The model can be used to optimize parameters of the protocol (such as the number of frames per superframe) as well as to predict the performance variations as the external conditions (such as data generation rate) vary.

Performance analysis of serial supply chain networks considering system disruptions

The supply chain networks could be very fragile in global environment due to unexpected events such as emergencies, normal disruptions and industrial accidents. The supply chain members may lose their production capacities temporarily, which might have significant impacts on the performance of the whole supply chain network. This article proposes a discrete time model to characterise the unreliable production capacity in serial supply chain networks. Based on the proposed model, the exact probability distributions are available for the performance analysis of a single-stage system in the lost sales scenario. The iterative methods are developed to derive the approximate performance measures for single-stage systems in the backorder scenario and multi-stage systems in both the lost sales and backorder scenarios. The proposed methods are verified through a series of numerical experiments. The analysis results suggest that the performance of the supply chain network suffers more from the downstream-stage unreliability than the upstream-stage unreliability. Furthermore, some application examples are illustrated to show the possible solutions for practical problems.

Modeling and Performance Evaluation of a Series-parallel Flow Line System with Finite Buffers

An approximation method based on discrete state Markov chain is developed to estimate the production rate of a series-parallel flow line with finite buffers. The proposed method's technique replaces a set of parallel machines at a work centre with an equivalent machine in order to obtain a traditional flow line with work centres in series separated by intermediate buffers. We derive equations for the parameters of the equivalent machine when it operates in isolation as well as in flow line. The few published analytical methods for series-parallel systems can tract only lines with a maximum of two work centres in series and a buffer in between them. Moreover, they are limited to estimate performance measures of flow lines composed only of identical machines in parallel. The proposed method, however, extends the aforementioned estimations for flow lines of any length with identical and/or non identical machines in parallel. Numerical examples are presented to show the applicability of the method where identical and non-identical parallel machine cases are considered and tackled.

Approximation of Performance Measures in an M/G/1 Queue with Breakdowns

We present a new perturbation bound in an M/G/I queueing system with breakdowns and repairs. Our analysis is based on bounding the distance of stationary distributions in a suitable functional space. This leads to understand how the breakdowns of the server will affect the system’s level of performance. In order to give an idea about the application of our approach in practice, we give a numerical example which would show the difference between explicit analytical estimates of errors and real simulated errors. It also allows for evaluating the potential of our approach. Eventually, we will point out directions of further research.

Equivalent single-queue–single-server model for a Pentium processor

In this paper, a Pentium processor is represented as a queuing network. The objective of this paper is to deduce an equivalent single-queue–single-server model for the original queuing network. Closed-form expressions for the equivalent service rate, equivalent queue lengths, equivalent response and waiting times of the equivalent single-queue–single-server model are derived and plotted. For large values of arrival rate, queue lengths increase faster than the response times and waiting times for both the cases. Performance measures like, queue lengths, response times and waiting times are higher for lower service rates and lower for higher service rates (which is expected) of the different servers in the original queuing network. Also, the reliability in estimating performance measures for homogeneous workloads is much better than that for heterogeneous workloads.

Development and validation of a prediction model with missing predictor data: a practical approach

Objective To illustrate the sequence of steps needed to develop and validate a clinical prediction model, when missing predictor values have been multiply imputed. Study Design and Setting We used data from consecutive primary care patients suspected of deep venous thrombosis (DVT) to develop and validate a diagnostic model for the presence of DVT. Missing values were imputed 10 times with the MICE conditional imputation method. After the selection of predictors and transformations for continuous predictors according to three different methods, we estimated regression coefficients and performance measures. Results The three methods to select predictors and transformations of continuous predictors showed similar results. Rubin's rules could easily be applied to estimate regression coefficients and performance measures, once predictors and transformations were selected. Conclusion We provide a practical approach for model development and validation with multiply imputed data.

Bayesian estimation of performance measures of screening tests in the presence of covariates and absence of a gold standard

The sensitivity (Se) and the specificity (Sp) are the two most common measures of the performance of a diagnostic test, where Se is the probability of a diseased individual to be correctly identified by the test while Sp is the probability of a healthy individual to be correctly identified by the same test. A problem appears when all individuals cannot be verified by a gold standard. This occurs when there is not a definitive test for detection of the disease or the verification by a gold standard is an impracticable procedure according to its cost, accessibility or risks. In this paper we develop a Bayesian analysis to estimate the disease prevalence, the sensitivity and specificity of screening tests in the presence of a covariate and in the absence of a gold standard. We use the Metropolis–Hastings algorithm to obtain the posterior summaries of interest. We have as motivation for the investigation the LAMS (Latin American Screening) Study, an extensive project designed for comparing screening tools for cervical cancer in Brazil and Argentina. When applied to the analysis of data from LAMS Study, the proposed Bayesian method shows to be a useful alternative to estimate measures of performance of screening tests in the presence of covariates and when a gold standard is not available. An advantage of the method is the fact that the number of parameters to be estimated is not limited by the number of observations, as it happens with several frequentist approaches. However, it is important to point out that the Bayesian analysis requires informative priors in order for the parameters to be identifiable. The method can be easily extended for the analysis of other medical data sets.

Inevitable Disappointment in Projects Selected on the Basis of Forecasts

Summary Investors often select projects whose estimated performance measures meet or exceed a hurdle value. At the time of decision making, the true performance of a project is unknown but uncertain forecasts are available. Decision makers (DMs) often ignore the prediction errors when they use these forecasts to choose projects. To the disappointment of the DMs, many selected projects result in smaller actual yields than those that were forecasted. Some have attributed the cause of this to the optimistic bias of the predictions. This paper shows that this disappointment can occur even if the prediction is unbiased. In this case, a bias can be introduced by the selection process that will allow more unattractive (overestimated) projects to be accepted than attractive (underestimated) ones. Although a similar phenomenon has been noted in statistics and finance research, it is not well understood in the context of project selection by DMs. We present a solution method based on Bayesian updating and demonstrate its effectiveness in eliminating the disappointment in project selection with realistic data from oil exploration and production projects.

Measuring Performance in a Dynamic World: Conditional Mean-Variance Fundamentals

We develop conditional alpha performance measures that are consistent with conditional mean-variance analysis. Commonly employed conditional alpha measures are inconsistent with conditional mean-variance analysis, conditional Sharpe ratio maximization, the magnitude or sign of the true conditional alphas, and reliable time-varying alphas. This conclusion is based on our derivation of the conditional investment opportunity set and its implied conditional alphas for given population parameters. The analysis is not subject to estimation errors and the important issue of whether empirically estimated performance measures violate the bounds on rational stochastic discount factors (Chen and Knez, 1996). An empirical example demonstrates that the differences between existing measures and our proposed measures are substantive for typical parameterizations.

The effect of AntNet parameters on its performance

AntNet is a routing algorithm which takes advantage of the way ants help each other to find food sources faster. Ants’ communication in this respect is indirect by leaving pheromone trails as they move. The richness and thickness of the trail is proportional to the number of ants that have passed. This method of indirect courtesy guidance by affecting the environment is called stigmergy. Many network parameters that affect the performance of AntNet algorithm. However, there is little works on AntNet parameter analysis. In this paper we examine the effect of AntNet parameters on its performance. Simulation is used to estimate performance measures. Results show that changing one specific parameter may not lead to an improvement in performance unless other parameters are also changed, accordingly. Key words: Routing algorithm, AntNet, AntNet performance parameter, agent.

Analysis of nonstationary stochastic simulations using classical time-series models

This article extends the use of classical autoregressive and moving average time-series models to the analysis of a variety of nonstationary discrete-event simulations. A thorough experimental evaluation shows that integrated and seasonal time-series models constitute very promising metamodels, especially for analyzing queueing system simulations under congested or cyclical traffic conditions. In some situations, stationarity-inducing transformations may be required before this methodology can be used. Our approach for efficient estimation of meaningful performance measures of selected responses in the target system is illustrated using a set of case studies taken from the simulation literature.

Partially-coherent receivers architectures for QAM communications in the presence of non-constant phase estimation error

In this paper, we present a new partially coherent receiver architecture motivated by optimum detection of quadrature amplitude modulation (QAM) signals in the presence of time-varying Tikhonov-distributed residual phase estimation error due to phase-locked loop (PLL)-aided phase tracking scheme. Performance is established in terms of bit error rate (BER). In this paper, an approximate performance measure motivated by union bound is presented for the proposed receiver architecture for 8- and 16-QAM constellations. The performance measures are assessed via simulation and analytical means for additive white Gaussian noise (AWGN) as well as for Rayleigh and Rician fading channels. The performance measures are shown to follow those of the optimum receiver over a wide range of signal-to-noise ratio (SNR), while outperforming a standard coherent receiver operating in the presence of residual phase error by as much as 2 dB.

A network queuing approach for evaluation of performance measures in autonomous vehicle storage and retrieval systems

A network queuing approach is used to estimate performance measures for Autonomous Vehicle Storage and Retrieval (AVS/R) Systems using opportunistic interleaving. The model exploits the distribution of cycle types and random storage assumptions to estimate the proportion of single and dual command cycles in a system. Comparable in accuracy and computational efficiency to procedures based on standard queuing models, the proposed technique provides the additional advantage of flexibility for modeling the interfaces between a storage system and the overall material flow system in a facility. The procedure is demonstrated for a range of problem scenarios.