Benchmarking Procedure Research Articles

Applying the Benchmarking Procedure: A Decision Criterion of Choice Under Risk

Modeling risk in a prescriptively plausible way represents a major issue in decision theory. The benchmarking procedure, being based on the satisficing principle and providing a probabilistic interpretation of expected utility (EU) theory, is prescriptive. Because it is a target-based language, the benchmarking procedure can be applied naturally to finance. In finance, the centrality of risk is widely recognized, but the risk measures that are commonly used to assess risk are too poor as a decision making tool. In this paper we propose a two-stage decision criterion of choice under risk that provides an application of benchmarking to finance through a risk measure. We will analyze some nonexpected utility theories, in particular lottery dependent utility, as potential frameworks for our criterion.

Modelling of water flow in a coupled surface-groundwater system. Application of the benchmarking process to the Riera del Carme basin (Barcelona, Spain)

Results of a surface-groundwater modelling application in the Riera del Carme catchment (Barcelona, Spain) following a benchmark procedure application are presented. The case study focuses on groundwater flow understanding as the management objective was to reduce groundwate.r extraction to sustainable levels. As the catchment is a hydraulically connected stream-aquifer system, modelling couples in a chained form Visual MODFLOW 3.0 and a black box code for surface water specifically developed for the case study. Although the poor quality of data made the modelling exercise more difficult, the most important output is the identification of those parameters with the greatest contribution for the hydrologic system understanding, and consequently those which need to be accurately monitored. The application of the BMW model evaluation tool (MET) for the selection and evaluation of simulation models step by step in the case study has been an useful tool.

A Quality Information Benchmarking Methodology: A US Banking Industry Empirical Study

Information is itself a product that customers and staff in any organization consume. Like any product the quality of information can be measured and compared in most industries. This paper presents an application of an information quality measuring methodology adapted and applied for the first time in the US banking industry. An empirical study is used to attest to the informational efficacy of the methodology. Results of the study reveal that the proposed methodology can be used to identify quality differences between differing banking institutions and differing dimensions of information quality as well as a quality benchmarking procedure.

Benchmark problems in nuclear data processing and utilization system for innovative reactors

A nuclear data processing and utilization system has been developed for research and development of innovative reactors such as accelerator-driven systems. The system is able to handle JENDL-3.3, ENDF/B-VI, and JEFF-3.1 to generate pointwise and groupwise cross sections in several formats, and has a capability to perform criticality and shielding benchmarks. Verification of nuclear data is available by using various criticality and shielding benchmark problems selected from database such as ICSBEP and SINBAD. The system performance highly depends on the quality of benchmark problems so that appropriate benchmarks should be selected from existing database. However a number of adequate benchmarks are quite a few to verify nuclear data for application of the accelerator-driven systems. Hence that new measurement should be necessary concerning criticality and shielding benchmarks. In the JENDL high-energy file (JENDL/HE) project, the intermediate-energy nuclear data integral test working group of Japanese Nuclear Data Committee made critical review of existing benchmark problems for accelerator-driven systems and performed verification of nuclear data in JENDL/HE. In this paper, the scheme of benchmark procedure and the selected benchmarks in the nuclear data processing and utilization system have been described.

Productivity Growth in the Lisbon Police Force

This paper estimates total productivity change with regard to the Lisbon Police Force and decomposes it into technically efficient change and technological change with data envelopment analysis (DEA). The benchmarking procedure implemented is an internal benchmarking, in which police precincts are compared against each other. The aim of this procedure is to seek out those best practices that will lead to improved performance throughout the whole force. We rank the precincts according to their total productivity change for the period 2000–2001, concluding that some precincts experienced productivity growth while a small number experienced productivity decrease. Economic implications arising from the study are considered.

Efficiency measurement among hypermarkets and supermarkets and the identification of the efficiency drivers

PurposeThe purpose of this paper is to analyse a representative sample of hypermarkets and supermarkets working in the Portuguese market, using a benchmark procedure to compare companies that compete in the same market and thereby deriving managerial and policy implications.Design/methodology/approachA two‐stage procedure to benchmark the companies was adopted. In the first stage data envelopment analysis (DEA) is used and in the second stage a Tobit model is employed to estimate the efficient drivers.FindingsFirst, that, on average, the efficiency of hypermarket and supermarket retail companies is high compared with that to be found in other sectors. Second, larger retail groups are, on average, more efficient than the smaller retailers, and third, that national retailers are on average more efficient than regional retailers. Finally, scale plays an important role in this market. The efficiency drivers are market share, number of outlets and location. Finally, regulation has a negative effect on efficiency.Research limitations/implicationsThis paper has two limitations. The first is in relation to the data set, and the second in relation to the DEA method. With reference to the data set, the homogeneity of the retailers used in the analysis is questionable, since retailers of different sizes, production characteristics and locations are compared, which may face different restrictions and therefore might not be considered to be directly comparable. However, it can always be claimed that the units are not comparable and that it would consequently be equally impossible to undertake a ratio analysis. Moreover, the data set is short, so that the conclusions are limited. For the conclusions to be generalised, a larger panel data set would be required.Practical implicationsThese can be separated into managerial and competitive – more importantly, the latter with its implications for the retailing industry and its future evolution.Originality/valueClarifies two issues: the changes in the fortunes of the retail companies, as observed in their sales volume ranking, and the need to look for the causes of retail efficiency, apart from internal managerial procedures.

BIOREL: The benchmark resource to estimate the relevance of the gene networks

The progress of high-throughput methodologies in functional genomics has lead to the development of statistical procedures to infer gene networks from various types of high-throughput data. However, due to the lack of common standards, the biological significance of the results of the different studies is hard to compare. To overcome this problem we propose a benchmark procedure and have developed a web resource (BIOREL), which is useful for estimating the biological relevance of any genetic network by integrating different sources of biological information. The associations of each gene from the network are classified as biologically relevant or not. The proportion of genes in the network classified as “relevant” is used as the overall network relevance score. Employing synthetic data we demonstrated that such a score ranks the networks fairly in respect to the relevance level. Using BIOREL as the benchmark resource we compared the quality of experimental and theoretically predicted protein interaction data.

Towards a benchmark simulation model for plant-wide control strategy performance evaluation of WWTPs

The COST/IWA benchmark simulation model has been available for seven years. Its primary purpose has been to create a platform for control strategy benchmarking of activated sludge processes. The fact that the benchmark has resulted in more than 100 publications, not only in Europe but also worldwide, demonstrates the interest in such a tool within the research community In this paper, an extension of the benchmark simulation model no 1 (BSM1) is proposed. This extension aims at facilitating control strategy development and performance evaluation at a plant-wide level and, consequently, includes both pre-treatment of wastewater as well as the processes describing sludge treatment. The motivation for the extension is the increasing interest and need to operate and control wastewater treatment systems not only at an individual process level but also on a plant-wide basis. To facilitate the changes, the evaluation period has been extended to one year. A prolonged evaluation period allows for long-term control strategies to be assessed and enables the use of control handles that cannot be evaluated in a realistic fashion in the one-week BSM1 evaluation period. In the paper, the extended plant layout is proposed and the new suggested process models are described briefly. Models for influent file design, the benchmarking procedure and the evaluation criteria are also discussed. And finally, some important remaining topics, for which consensus is required, are identified.

Average conditional correlation and tree structures for multivariate GARCH models

We propose a simple class of multivariate GARCH models, allowing for time-varying conditional correlations. Estimates for time-varying conditional correlations are constructed by means of a convex combination of averaged correlations (across all series) and dynamic realized (historical) correlations. Our model is very parsimonious. Estimation is computationally feasible in very large dimensions without resorting to any variance reduction technique. We back-test the models on a six-dimensional exchange-rate time series using different goodness-of-fit criteria and statistical tests. We collect empirical evidence of their strong predictive power, also in comparison to alternative benchmark procedures. Copyright © 2006 John Wiley & Sons, Ltd.

Methodology for Evaluating the Quality of ICC Profiles–Scanner, Monitor, and Printer

The concepts and technology for an open loop color management system such as that specified by the International Color Consortium (ICC) have been around for a number of years. The adoption of this workflow by the graphic arts industry has been slow. A major contribution to the lack of popularity is that the quality part of the ICC workflow is unregulated and the average user is unable to independently assess the quality of profiles and profile making software. This paper describes a number of test methods that can be used to evaluate the colorimetric accuracy of ICC scanner, monitor, and printer profiles. ICC profiles are being used in a number of color proofing scenarios. In order to understand the color reproduction abilities of such proofing systems it is necessary to quantify the accuracy of the underlying ICC profiles used in the workflow. A quality metric can be useful to provide feedback on how well a device has been characterized and therefore provide limits on the ability of a color managed system. Further, a universally defined merit figure will allow the comparison of results across manufacturers, allowing the user to make informed choices appropriate for their workflow. If we are able to establish a benchmarking procedure akin to the miles per gallon fuel consumption quoted for motor vehicles this provides a universal quality metric that can help raise the quality of profiling software, assist user choice, and ultimately lead to the greater acceptance of ICC color management in graphic arts and the printing industry. The quality of input profiles is described in terms of a AE calculation. Input profiles from ten different profiling packages were evaluated. Profiles were made for a flatbed scanner using Agfa, Fuji, and Kodak IT8.7/2 targets. The average AE for input profiles is shown to be in the range of ΔE 0.60-2.46. A procedure is described for evaluating the quality of monitor profiles in terms of measured gamma, white point and the color reproduction of 24 specially chosen colors. Eleven monitor systems were evaluated. Monitor profiles were made for an Apple Cinema HD LCD display and measurements were made to verify the accuracy of these profiles using a telespectroradiometer. It is shown that commercial products were able to reproduce 24 patches of a color checker on a monitor with an average AE of 2.92-5.81. A printer profile metric is also described. This research describes three possible metrics for a printer profile-accuracy of the B2A 1 tag (PCS to Device), A2B1 tag (Device to PCS), and a round trip test. Data are presented to show that the average accuracy of the output profile colorimetric intent (average of B2A1 and A2B1) for an Epson ink jet printer can be between AE 1.72-3.49. The accuracy of the printer profile is useful when considering the use of an ink jet printer in color management proofing workflows. This research proposes a ΔE metric system that can be used to evaluate the quality of commercially created ICC input, display, and output profiles. The data presented here are fundamentally a methodology that can be used to estimate the colorimetric accuracy of profiles. The use of commercial profiling products is only to illustrate how the metrics may be determined in practice. The data, however, do show useful information about the state of color management products today and this analysis can be used to track the improvements and evolution in ICC profiling software.

A multi-objective simulated-annealing algorithm for scheduling in flowshops to minimize the makespan and total flowtime of jobs

In this paper, we consider the problem of permutation flowshop scheduling with the objectives of minimizing the makespan and total flowtime of jobs, and present a Multi-Objective Simulated-annealing Algorithm (MOSA). Two initial sequences are obtained by using simple and fast existing heuristics, supplemented by the implementation of three improvement schemes. Each of the two resultant sequences corresponds to a possible non-dominated solution containing the minimum value of one objective function. These sequences, taken one at a time, are given as the starting sequences to the MOSA. The MOSA seeks to obtain non-dominated solutions through the implementation of a simple probability function that attempts to generate solutions on the Pareto-optimal front. The probability function selects probabilistically a particular objective function, considering which the algorithm uncovers non-dominated solutions. Moreover, the probability function is varied in such a way that the entire objective-function space is covered uniformly so as to obtain as many non-dominated and well-dispersed solutions as possible. The parameters in the proposed MOSA are determined after conducting a pilot study. Two variants of the proposed algorithm, called MOSA-I and MOSA-II, with different parameter settings with respect to the temperature and epoch length, are considered in the performance evaluation of algorithms. In order to evaluate MOSA-I and MOSA-II, we have made use of 90 benchmark problems provided by Taillard [Eur. J. Operation. Res. 64 (1993) 278]. After an extensive literature survey, the following flowshop multi-objective scheduling algorithms have been identified as benchmark procedures: (a) MOGLS (Multi-Objective Genetic Local Search) by Ishibuchi and Murata [IEEE Trans. Syst., Man, Cybernet. C: Appl. Rev. 28 (1998) 392]; (b) Elitist Non-dominated sorting Genetic Algorithm (ENGA) by Bagchi [Multi-Objective Scheduling by Genetic Algorithms, Kluwer Academic Publishers, 1999]; (c) GPW (Gradual Priority Weighting) approach by Chang, Hsieh and Lin [Int. J. Prod. Econ. 79 (2002) 171]; and (d) a posteriori approach based heuristic by Framinan, Leisten and Ruiz-Usano [Eur. J. Operation. Res. 141 (2002) 559]. The non-dominated sets obtained from each of the existing benchmark algorithms and the proposed MOSA-I and MOSA-II are compared, and subsequently combined to obtain a net non-dominated front. It is found that most of the solutions in the net non-dominated front are yielded by MOSA-I and MOSA-II. In addition, it is noteworthy that both MOSA-I and MOSA-II require less computational effort than the MOGLS, ENGA and GPW.

Benchmarking real-valued acts

A benchmarking procedure ranks real-valued acts by the probability that they outperform a benchmark β which may itself be a random variable; that is, an act f is evaluated by means of the functional V ( f ) = P ( f ⩾ β ) . Expected utility is a special case of benchmarking procedure, where the acts and the benchmark are stochastically independent. This paper provides axiomatic characterizations of preference relations that are representable as benchmarking procedures. The key axiom is the sure-thing principle. When the state space is infinite, different continuity assumptions translate into different properties of the probability P.

Benchmarking Systems of Seasonally Adjusted Time Series

When a system of time series is seasonally adjusted, generally the accounting constraints originally linking the series are not fulfilled. To overcome this problem, we discuss an extension to a system of series linked by an accounting constraint of the classical univariate benchmarking procedure due to Denton (1971), which is founded on a movement preservation principle that is very relevant in this case. The presence of linear dependence between the variables makes it necessary to deal with the whole set of contemporaneous and temporal aggregation relationships. The cases of one-way classified (e.g., by regions or by industries) and of two-way classified (e.g., by regions and by industries) systems of series are studied. An empirical application to the Canadian retail trade series by province (12 series) and trade groups (18 series) is considered to show the capability of the proposed procedures.

Matching Science with Coastal Management Needs: The Search for Appropriate Coastal State Indicators

Abstract Through an analysis of the interaction between end users and researchers participating in the European Union–funded CoastView project (EVK3-CT-2001-0054), this article illustrates some of the difficulties associated with end user–oriented research. A way of structuring and focusing discussion between end users and researchers, which was applied and further developed during the course of the project, is suggested as a method to deal with these difficulties. The analysis in this article indicates that successful specialist support of decision making is related to the use of a systematic “frame of reference.” This involves explicit definitions of both strategic and operational objectives applied in a four-step decision recipe consisting of (1) a quantitative state concept, (2) a benchmarking procedure, (3) a design procedure for measures or interventions, and (4) an evaluation procedure.

Determination of base-stock levels in a serial supply chain: a simulation-based simulated annealing heuristic

This paper presents simulation-based heuristic methodologies to compute installation base-stock levels in a serial supply chain so as to minimise the total supply chain cost. A simulated annealing heuristic is proposed to compute the installation base-stock levels for different installations in a serial supply chain. In addition, a simple heuristic to determine the installation base-stock levels is also presented. Simulation is used to evaluate the installation base-stock levels yielded by the proposed simulated annealing heuristic and the simple heuristic. A best-move local search is also used as a benchmark procedure. The solution constructed by the simple heuristic is initialised as the seed solution to both simulated annealing heuristic and the best-move local search, and their performances are studied. All the solution methodologies are extensively evaluated with different supply chain test problems. The solutions yielded by the proposed solution methodologies is compared against the optimal installation base-stock level which is obtained through a complete enumeration of the solution space for each of the supply chain test problems. A comparison of the performance of all the solution methodologies shows that the proposed simulated annealing heuristic initialised with the heuristic solution as the initial seed solution performs the best by consistently producing good-quality solutions with little computational effort.

IGRA. A tool for applying the benchmarking initiative to irrigated areas

en

Density functional response theory calculations of three-photon absorption

Three-photon absorption probabilities delta(3PA) have been calculated through application of a recently derived method for cubic response functions within density functional theory (DFT). Calculations are compared with Hartree-Fock (HF) and with a coupled cluster hierarchy of models in a benchmarking procedure. Except for cases having intermediate states near resonance, density functional theory is demonstrated to be in sufficient agreement with the highly correlated methods in order to qualify for predictions of delta(3PA). For the larger systems addressed, a set of acceptor A and donor D substituted pi-conjugated systems formed by trans-stilbene and dithienothiophene (DTT), we find noticeable differences in the magnitude of delta(3PA) between HF and DFT, although similar trends are followed. It is shown that the dipolar structures, TS-AD and DTT-AD, have substantially larger delta(3PA) than other types of modifications which is in accordance with observations for two-photon absorption. This is the first application of density functional theory to three-photon absorption beyond the use of few-state models.

Productivity in the tourism industry

This paper analyzes the efficiency of a Portuguese public-owned hotel chain, Enatur. It applies data envelopment analysis (DEA) to estimate total factor productivity (TFP) change, while breaking it down into technical efficiency and technological change. The benchmarking procedure used is an internal one, which compares hotels with each other. For the period 1999–2001, the hotels are ranked according to their total productivity change. It is concluded that some hotels experienced productivity growth, while others faced a decline. Some implications beneficial for managerial policies were drawn from this study.

Sensitivity analysis of transportation corridor location in wetland areas: A multiobjective programming and GIS approach

A multiobjective programming approach for locating transportation corridors through areas containing wetlands is presented. The approach uses a geographic information system in combination with multiple objectives for environment and construction costs, and the North Carolina Coastal Region Evaluation of Wetland Significance (NC-CREWS) program. The multiobjective function is a weighted combination of wetland values and construction cost. The objective minimizes total cost of construction and wetland impacts resulting from a specific transportation corridor. The transportation designer specifies starting and ending points of the desired corridor. The model then finds the best path given the weights placed on each individual objective (i.e. habitat quality, water quality, hydrology, and construction expense). By varying the weight set, several different paths can be found. The approach is tested on a watershed in Carteret County North Carolina using a benchmarking procedure. One benchmark minimizes a combined wetland significance loss (cost). This path avoids crossing wetlands. The other benchmark minimizes construction expense. Varying the weight placed on each of the objective function components generates additional paths. The approach is useful in locating alternative road corridor paths in attempts to satisfy conflicting societal interests.

Average Conditional Correlation and Tree Structures for Multivariate GARCH Models

We propose a simple class of multivariate GARCH models, allowing for time-varying conditional correlations. Estimates for time-varying conditional correlations are constructed by means of a convex combination of averaged correlations (across all series) and dynamic realized (historical) correlations. Our model is very parsimonious. Estimation is computationally feasible in very large dimensions without resorting to any variance reduction technique. We back-test the models on a six-dimensional exchange-rate time series using different goodness-of-fit criteria and statistical tests. We collect empirical evidence of their strong predictive power, also in comparison to alternative benchmark procedures.