Black-box Methods Research Articles

On the testing methods used by beginning software testers

This paper describes our experiences of the methods used by novice software testers to test their own programs, as well as their perception of the classification-tree method, which is a black box testing method first introduced by Grochtmann and Grimm. We conducted two case studies involving novice software testers. The subjects in the first study possessed one-year working experience while those in the second study had a wider range of working experiences. Both studies found that white box testing methods were initially far more popular than black box methods, but the majority of the subjects were convinced of the benefits of the classification-tree method after they had learned and used it. About two-third of them indicated their preference of the classification-tree method over the methods they originally used.

Genetic programming for the prediction of insolvency in non-life insurance companies

Prediction of non-life insurance companies insolvency has arised as an important problem in the field of financial research, due to the necessity of protecting the general public whilst minimizing the costs associated to this problem, such as the effects on state insurance guaranty funds or the responsibilities for management and auditors. Most methods applied in the past to predict business failure in non-life insurance companies are traditional statistical techniques, which use financial ratios as explicative variables. However, these variables do not usually satisfy statistical assumptions, what complicates the application of the mentioned methods. Emergent statistical learning methods like neural networks or SVMs provide a successful approach in terms of error rate, but their character of black-box methods make the obtained results difficult to be interpreted and discussed. In this paper, we propose an approach to predict insolvency of non-life insurance companies based on the application of genetic programming (GP). GP is a class of evolutionary algorithms, which operates by codifying the solution of the problem as a population of LISP trees. This type of algorithm provides a diagnosis output in the form of a decision tree with given functions and data. We can treat it like a computer program which returns an answer depending on the input, and, more importantly, the tree can potentially be inspected, interpreted and re-used for different data sets. We have compared the performance of GP with other classifiers approaches, a Support Vector Machine and a Rough Set algorithm. The final purpose is to create an automatic diagnostic system for analysing non-insurance firms using their financial ratios as explicative variables.

Automatic generation of fuzzy rule-based models from data by genetic algorithms

A methodology for the encoding of the chromosome of a genetic algorithm (GA) is described in the paper. The encoding procedure is applied to the problem of automatically generating fuzzy rule-based models from data. Models generated by this approach have much of the flexibility of black-box methods, such as neural networks. In addition, they implicitly express information about the process being modelled through the linguistic terms associated with the rules. They can be applied to problems that are too complex to model in a first principles sense and can reduce the computational overhead when compared to established first principles based models. The encoding mechanism allows the rule base structure and parameters of the fuzzy model to be estimated simultaneously from data. The principle advantage is the preservation of the linguistic concept without the need to consider the entire rule base. The GA searches for the optimum solution given a comparatively small number of rules compared to all possible. This minimises the computational demand of the model generation and allows problems with realistic dimensions to be considered. A further feature is that the rules are extracted from the data without the need to establish any information about the model structure a priori. The implementation of the algorithm is described and the approach is applied to the modelling of components of heating ventilating and air-conditioning systems.

Computer-aided Simulation for Optimizing Nanofiltration Processes

The modeling of membrane filtration processes is only possible by applying black-box models or short-cut methods, because of its complexity due to molecular interactions on and inside the membrane. The assumptions made for the short-cut methods are applicable for reverse osmosis, whereas the simulation of nanofiltration processes leads to unreliable results, which sometimes deviate highly from real conditions. A steady-state process simulation NF-PROJECT, which is based on input information from membrane characterization, was developed. The individual separation characteristics of every membrane element are calculated in iteration, which leads to a reduction in the permeability and retention of the elements inside the pressure vessel. The simulation provides information on the increasing feed concentration and osmotic pressure, the hydraulic pressure loss, the deterioration of the flow conditions in the vessel and the combined performance of the membrane elements to be analyzed. As an example, the simulation was applied in a two-stage nanofiltration pilot plant, the results of which are presented.

Efficient design and optimisation of two-stage NF processes by simplified process simulation

The modelling of membrane filtration processes is often conducted by applying black-box models or short-cut methods because of its complexity due to molecular interactions on and inside the membrane. The assumptions made for the short-cut-methods are applicable for reverse osmosis, whereas the simulation of nanofiltration processes can lead to unreliable results, which sometimes deviate greatly from real conditions. A steady-state process simulation NF-PROJECT, which is based on input information from membrane characterisation, was developed (isothermal operation). The individual separation characteristics of every membrane element are calculated in iteration, which leads to reduction of permeability and rejection for every further element arranged in series inside the pressure vessel. The simulation provides information on the increasing feed concentration and osmotic pressure, the hydraulic pressure loss, the deterioration of the flow conditions along the feed side in the vessel and the combined performance of the membrane elements to be analysed. Serving as an example from practical applications, a two-stage nanofiltration pilot plant was simulated, the results of which will be presented. Examples of design and optimisation potentials will be illustrated for the target criteria of economic efficiency (specific energy costs), permeate quality and flow.

Comparative study of black-box and hybrid estimation methods in fed-batch fermentation

A neural network based softsensor is proposed for a PHB fed-batch fermentation process. The softsensor is designed to estimate the biomass concentration on-line. The design is based on the following model structures: 1. a feedforward neural network, 2. a RBFN (radial basis function neural network) and 3. hybrid models composed of either feedforward or RBFN neural network and the a priori known dilution term of the mass balance equations. The different designs are experimentally implemented and compared using Alcaligenes eutrophus as a model fed-batch system. Additionally, the possibility of directly inferring the substrate (glucose) concentration from the estimated biomass was investigated by assessing the variability of the corresponding yield coefficient. The combination of the neural network model and mechanistic differential equation provided the best results. Because of the variability in the yield coefficient, substrate concentration could not be inferred directly.

Using the Right Tools to Answer the Right Questions: The Importance of Evaluative Research Techniques for Health Services Evaluation Research in the 21st Century

Abstract: The marked changes in health care expenditures in recent years have led to a call for greater accountability in the areas of health education, policy, services, and reform. In recent years, evaluative research has been conducted in health arenas under the rubric of health services research. The research methods employed have evolved not from evaluation research methodology, but from frameworks that often do not lend themselves to deriving appropriate causal inferences in multi-causal environments. Given the complexity of the interrelated political, social, psychological, and economic factors that can affect health services, more complex evaluative techniques are needed. This article describes the epistemology of evaluative research and, through a series of examples from the health services literature, demonstrates the strengths of theory-driven approaches and statistical multivariate techniques compared to traditional black-box methods, as ways to increase validity for causal inference.

Gray box identification of flexible structures: application to robust active vibration suppression control

AbstractThe paper deals with gray box identification of flexible structures and active vibration suppression from a robust control perspective. First, the linearized mathematical model of an N‐storey flexible structure is presented. Next, the generalized mathematical model is particularized for the investigated three‐storey flexible structure. The considered flexible structure is identified based on black box and gray box identification methods and the model's parametric uncertainties are deduced. Furthermore, control constraints are presented for the design problem, in case of velocity as well as acceleration feedback, from a robust control perspective. Finally, the effectiveness of the control system is tested through experiments, when the input disturbance is assumed to be a sinusoidal one as well as a historical earthquake record (1940 El Centro record). Copyright © 2001 John Wiley & Sons, Ltd.

A STUDY ON A PATH-BASED STRATEGY FOR SELECTING BLACK-BOX GENERATED TEST CASES

Various black-box methods for the generation of test cases have been proposed in the literature. Many of these methods, including the category-partition method and the classification-tree method, follow the approach of partition testing, in which the input domain is partitioned into subdomains according to important aspects of the specification, and test cases are then derived from the subdomains. Though comprehensive in terms of these important aspects, execution of all the test cases so generated may not be feasible under the constraint of tight testing resources. In such circumstances, there is a need to select a smaller subset of test cases from the original test suite for execution. In this paper, we propose the use of white-box information to guide the selection of test cases from the original test suite generated by a black-box testing method. Furthermore, we have developed some techniques and algorithms to facilitate the implementation of our approach, and demonstrated its viability and benefits by means of a case study.

Kriging as a surrogate fitness landscape in evolutionary optimization

The problem of finding optimal values in complex parameter optimization problems has often been solved with success by evolutionary algorithms (EAs). In many cases, these algorithms are employed as black-box methods over imprecisely known domains. Such problems arise frequently in engineering design. The principal barrier to the general use of EAs for those problems is the huge number of function evaluations that is often required. This makes EAs an impractical approach when the function evaluation depends on numerically heavy design analysis tools, for example, finite elements methods. This paper presents the use of kriging interpolation as a function approximation method for the construction of an internal model of the fitness landscape. This model is intended to guide the search process with a reduced number of fitness function evaluations.

Rainfall–runoff relations for karstic springs. Part I: convolution and spectral analyses

Karstic basins contain large reserves of subsurface water. In this paper, three karstic systems located in the Pyrénées Mountains (Ariège, France) are studied. Long records of rainfall and discharge rates for these karstic springs are available, sampled at different rates: daily, hourly and half-hourly. This study aims at illustrating and assessing the capabilities and limitations of linear black-box methods for analysing rainfall–runoff type relationships and reconstructing runoffs from rainfall rate data using such systems. In this study, precipitation and discharge rates are considered as two autocorrelated and cross-correlated stochastic processes. A linear and stationary rainfall–runoff model is adopted, which is used for identification and simulation purposes. Different versions are analysed, including a model based on a convolution integral between the precipitation rate P( τ) and a transfer function h( t− τ) which can be thought of as the unit impulse response of the system. It is shown that this linear stochastic model (i.e. the statistical version), although accurate in some respects, does not represent the hydraulic behaviour of the system very well during low flow episodes and floods. It is also shown that the use of Fourier analysis, alone, does not lead to a satisfactory reconstitution of observed runoff sequences. For these reasons, the use of non-linear random process input–output models based on Volterra integral series is proposed and discussed.

Algorithm 802

Different automatic (also called universal or black-box) methods have been suggested to sample from univariate log-concave distributions. Our new automatic algorithm for bivariate log-concave distributions is based on the method of transformed density rejection. In order to construct a hat function for a rejection algorithm the bivariate density is transformed by the logarithm into a concave function. Then it is possible to construct a dominating function by taking the minimum of several tangent planes, which are by exponentiation transformed back into the original scale. The choice of the points of contact is automated using adaptive rejection sampling. This means that points that are rejected by the rejection algorithm can be used as additional points of contact. The article describes the details how this main idea can be used to construct Algorithm ALC2D that can generate random pairs from all bivariate log-concave distributions with known domain, computable density, and computable partial derivatives.

Identification of the rainfall–runoff relationship in urban drainage networks

The calibration of conceptual models for the design of urban drainage networks is an important and well-known problem in hydraulic engineering. In this paper the problem is analysed and the use of black-box identification methods is proposed and applied to experimental data. Both linear (ARX and state space) and nonlinear (polynomial and neural NARX) models are considered and their performance in the simulation and prediction of the network flow from rainfall measurements is evaluated.

Time-Domain Quantification of Series of Biomedical Magnetic Resonance Spectroscopy Signals

Quantification of individual magnetic resonance spectroscopy (MRS) signals is possible in the time domain using interactive nonlinear least-squares fitting methods which provide maximum likelihood parameter estimates under certain assumptions or using fully automatic, but statistically suboptimal, black-box methods. In kinetic experiments time series of consecutive MRS spectra are measured in which information concerning the time evolution of some of the signal parameters is often present. The purpose of this paper is to show how AMARES, a representative example of the interactive methods, can be extended to the simultaneous processing of all spectra in the time series using the common information present in the spectra. We show that this approach yields statistically better results than processing the individual signals separately.

A rejection technique for sampling from log-concave multivariate distributions

Different universal methods (also called automatic or black-box methods) have been suggested for sampling form univariate log-concave distributions. The descriptioon of a suitable universal generator for multivariate distributions in arbitrary dimensions has not been published up to now. The new algorithm is based on the method of transformed density rejection. To construct a hat function for the rejection algorithm the multivariate density is transformed by a proper transformation T into a concave function (in the case of log-concave density T(x) = log( x ).) Then it is possible to construct a dominating function by taking the minimum of serveral tangent hyperplanes that are transformed back by T -1 into the original scale. The domains of different pieces of the hat function are polyhedra in the multivariate case. Although this method can be shown to work, it is too slow and complicated in higher dimensions. In this article we split the ℝ n into simple cones. The hat function is constructed piecewise on each of the cones by tangent hyperplanes. The resulting function is no longer continuous and the rejection constant is bounded from below but the setup and the generation remains quite fast in higher dimensions; for example, n = 8. The article describes the details of how this main idea can be used to construct algorithm TDRMV that generates random tuples from a multivariate log-concave distribution with a computable density. Although the developed algorithm is not a real black box method it is adjustable for a large class of log-concave densities.

Correlated, Static and Dynamic Polarizabilities of Small Molecules. Comparison of Four “Black Box” Methods

Molecular static and dynamic polarizabilities for thirteen small molecules have been calculated using four “black box” ab initio methods, the random phase approximation, RPA, the second-order polarization propagator approximation, SOPPA, the second-order polarization propagator approximation with coupled cluster singles and doubles amplitudes, SOPPA(CCSD), and the coupled cluster singles and doubles linear response function method, CCSDLR. The frequency dependence of the polarizabilities is given in terms of the dipole oscillator strength sum rules or Cauchy moments S(-4) and S(-6). Two basis sets were employed, Sadlej’s medium size polarized basis set and Dunning’s correlation consistent basis set of triple- œ quality augmented by two diffuse functions of each angular momentum (daug-cc-pVTZ). The results are compared to other theoretical results as well as to experimental values for the static polarizabilities, polarizability anisotropies, and Cauchy moments. Frequency-dependent polarizabilities and polarizability anisotropies, calculated at the CCSDLR level using the daug-cc-pVTZ basis set, are presented for five typical laser frequencies. The molecular dipole polarizability enters into the description of many physical and chemical processes, such as the scattering of light by molecules, and intermolecular interactions. Calculated polarizabilities are often used in the verification of experimental data and in the prediction of properties of new chemical species. An accuracy of a few percent in the calculated values is necessary for this purpose. Over the years several methods for the calculation of molecular properties have emerged. Among these are correlated methods, i.e. methods trying to improve on the Hartree-Fock approximation by perturbation theory or a multiconfigurational ansatz, as well as density functional theory (DFT) methods. Most of these methods, however, are only capable of calculating static properties like the static molecular polarizability, excitation energies, and transition moments. A direct comparison of calculated and experimental polarizabilities requires the ability to calculate frequency-dependent polarizabilities since experiments are mostly performed at nonzero frequencies. In an earlier study 1 the performance of some perturbation theory methods in the calculation of static polarizabilities was investigated. In this work, calculations of static and dynamic molecular polarizabilities are presented using four different “black box” methods, that is, methods where the only choices to be made are of the basis set and molecular geometry. These methods are in contrast to multiconfigurational methods where the selection of configurations to be included in the wave function requires considerable experience and might even become impossible for larger molecules. The black box methods, on the other hand, are relatively easy to use also by nonexperts, and their application is, apart from hardware limitations, not restricted to small molecules.

Kinematic Wave Modeling in Water Resources

Kinematic Wave Modeling in Water Resources fills an important gap in the literature on physically based hydrologic modeling. The author should be complimented on his approach, which is based on solving the governing partial differential equations. He displays remarkable ingenuity at finding solutions to a system of differential equations that is particularly intractable. This volume covers the fullest possible spectrum of solutions emerging from the method of characteristics. The author's physically based, deterministic approach is refreshing in a world overwhelmed with black box and numerical methods.

An integrated fault diagnostic mechanism based on statistical characteristics of products for heterogeneous manufacturing environments

A fault diagnosis mechanism for heterogeneous manufacturing environments is developed. Using the correlation coefficient among quality characteristic values (QCVs) of products enables the functional correlation between processes to be uniformly treated according to causal relations among processes, despite their functional heterogeneity. Complex causal relations between QCVs are graphically represented by the influence graph, which is developed to model the diagnostic knowledge of our method. This mechanism realizes more accurate diagnostic capabilities than black-box methods, and still, does not require exhaustive modelling of the entire process, unlike model-based reasoning methods. The viability of this diagnostic approach was verified via simulation.

Random variate transformations in the Gibbs sampler: issues of efficiency and convergence

In the non-conjugate Gibbs sampler, the required sampling from the full conditional densities needs the adoption of black-box sampling methods. Recent suggestions include rejection sampling, adaptive rejection sampling, generalized ratio of uniforms, and the Griddy-Gibbs sampler. This paper describes a general idea based on variate transformations which can be tailored in all the above methods and increase the Gibbs sampler efficiency. Moreover, a simple technique to assess convergence is suggested and illustrative examples are presented.

A comparison of optimization-based approaches for a model computational aerodynamics design problem

The objective of this paper is to compare three optimization-based methods for solving aerodynamic design problems. We use the Euler equations for one-dimensional duct flow as a model problem. The optimization methods are (i) the black-box method with finite difference gradients, (ii) a modification where gradients are found by an algorithm based on the implicit function theorem, and (iii) an all-atonce method where the flow and design variables are simultaneously altered. The three methods are applied to the model problem and compared for efficiency, robustness, and implementation difficulty. We also show that the black-box (implicit gradient) method is equivalent to applying the “variational” or “optimal control” approach to design optimization directly to the discretized analysis problem, rather than to the continuous problem as is usually done. The black-box method with implicit gradients seems to provide a good compromise of features, and can be retrofitted to most existing analysis codes to turn them into design codes. Although the all-at-once method was found to be less robust than the black-box methods, when it succeeded it was considerably more efficient.