Complex Testing Problems Research Articles

Permutation tests for multivariate order-constrained testing problems

This article discusses an effective permutation-based approach to solve order-constrained testing problems, which are very difficult or even impossible to solve within the framework of parametric likelihood. The nonparametric combination (NPC) methodology is a flexible tool which relies on dependent permutation tests and allows us to deal with a large variety of complex testing problems, including the stochastic dominance and monotonic stochastic ordering problems of interest. To deal with them in line with Roy’s Union-Intersection (UI) approach, the NPC procedure decomposes the original hypothesis into suitable sub-hypotheses. In this article we discuss and compare two possible types of decomposition, exploiting an extensive simulation study. A real data application is also proposed, in which multivariate ordinal data from the medical field are analyzed.

Coupling a Borehole Thermal Model and MT3DMS to Simulate Dynamic Ground Source Heat Pump Efficiency.

We present a novel integrated two-region model that couples simulation of local heat transfer processes in borehole heat exchangers (BHEs) with field-scale heat transport simulation using MT3DMS to fulfill the dynamic simulation of the borehole geothermal systems. This includes the prediction of subsurface thermal perturbation induced by BHEs, derivation of the U-pipe circulating fluid temperature profile within boreholes, and the evaluation of the ground source heat pump efficiency based on available time series of building heat load. In our approach, MT3DMS is the simulator for the two-dimensional field-scale heat transport, while new Python modules are developed to analytically solve the thermal transfer process within boreholes and interface iteratively with MT3DMS. A Python package for scripting MODFLOW-based code named Flopy is used to establish the MT3DMS numerical model. The proposed model is validated against analytical solutions and we demonstrate the application to more complex test problems with field-scale heterogeneity and pumping. Instructions are provided to access the source code and example problems which are available online.

Проверка численной эффективности методов глобальной оптимизации

In this paper, new difficult test problems are proposed to test the numerical efficiency of global optimization methods. These are problems of unconstrained optimization with unknown solutions. The proposed test problems are inseparable and have arbitrary dimensions. The author also proposes to include the test functions by J. Nie in the list of test functions for numerical verification of the effectiveness of methods. These functions are also inseparable functions of arbitrary dimensions with unknown solutions. The proposed test problems have many local extrema. Testing the effectiveness of global optimization methods for such functions is simplified. If the method allows improving the found solutions to test problems, then it will be more effective. The existing global optimization methods are compared with the exact quadratic regularization method developed by the author. This method is compared with known software packages that implement modern methods of global optimization. These packages include several methods. The best of them use convex relaxation of the problem to obtain estimates of solutions with subsequent use of local optimization programs. But even such powerful packages have difficulties in solving the considered test problems. Some test problems, for example, with the Rana or Egg Holder function, have been solved by different methods for over 20 years. During this time, no method has allowed obtaining results that are obtained by the method of exact quadratic regularization. For almost all complex test problems with unknown solutions, this method yielded better solutions. Sometimes the advantage of this method was significant, as is the case with the Rana test function. The essence of the exact quadratic regularization method is to transform any global optimization problem to maximize the square of the Euclidean norm of a vector on a convex set. This problem is computationally much simpler. Often, with such a transformation, the multimodal problem becomes unimodal, which is easy to solve. Keywords: test problems, global optimization, unimodal problems, multimodal problems, numerical methods.

Power analysis, sample size calculation for testing the largest binomial probability

A procedure is developed for power analysis and sample size calculation for a class of complex testing problems regarding the largest binomial probability under a combination of treatments. It is shown that the asymptotic null distribution of the likelihood-ratio statistic is not parameter-free, but is a conservative asymptotic null distribution. A nonlinear Gauss-Seidel algorithm is proposed to uniquely determine the alternative for the power and sample size calculation given the baseline binomial probability. An example from an animal clinical trial is discussed.

Multi-Scale Quantum Harmonic Oscillator Algorithm With Truncated Mean Stabilization Strategy for Global Numerical Optimization Problems

A multi-scale quantum harmonic oscillator algorithm (MQHOA) is a quantum population-based algorithm proposed recently. It utilizes the quantum wave function to locate the global optimum of a global numerical optimization problem. As the MQHOA employs the elitism to replace the worst particle in each iteration cycle, it reduces one of the particles in each run, which will cripple the diversity of the population and slow down the convergence speed. Therefore, the particles will be easily trapped into local optima. In this paper, we suggest a new MQHOA with truncated mean stabilization (TS-MQHOA) policy to alleviate the above-mentioned problems. The theoretical and experimental analyses indicate that the truncated mean stabilization strategy helps to diversify the populations and improve the convergence efficiency. The proposed TS-MQHOA is evaluated on a number of dimensionwise unimodal and multimodal CEC benchmark functions, and the computational results are compared with several popular population-based algorithms. The experimental results on complex test problems demonstrate that the proposed TS-MQHOA, in most function evaluations, is able to obtain better convergence toward the global optimum compared with several renowned heuristic algorithms based on swarm intelligence. Meanwhile, the comparative results reveal the competitiveness and superiority of the proposed algorithm, especially on high-dimensional function evaluations.

Competing optical instruments for the estimation of Full Field FRFs

Thanks to the progress of optical measurement systems, known to use non-contact transducers (with their own peculiarities, application domains and drawbacks), nowadays addressing complex testing problems, especially direct dynamic measurements and modal analyses on lightweight structural components, is becoming even more common than in the recent past. Furthermore, obtaining vibration patterns with optical gears, in broad frequency bands with high spatial resolution, is easier than before, for the advantage of many advanced applications in vibration engineering, testing and product development.The characteristics and uniqueness of three different measurement technologies, based on distinguished physics and processing grounds, are here compared on a shared test rig with multiple excitations. SLDV, Hi-Speed DIC and Dynamic ESPI are here challenged in a complex and unique matching test for the estimation of FRFs with high spatial resolution, exploiting, as benchmark of the methodologies, a vibrating lightweight rectangular plate. The plate was selected for its exhibiting a broad band dynamics with high modal density and potentially repeated and closely spaced roots in the common frequency domain, where the techniques can find an operative intersection. The comparison of estimated FRFs and related Coherences, here as quality features of the measurements, is detailed with specific notes on acquisition and processing techniques, in order to draw extended conclusions on the full field measurement approaches, specific gears and the multiple excitation sources. The peculiar pointwise procedure, here proposed in details as the minimal paradigm to make a meaningful comparison, is addressed by means of discrete geometry transforms to put all the three technologies on an effective and consistent trial at each physical location of the real surface.The obtained results are a significant step forward in understanding the cross comparison of what these optical technologies, which belong to different domains and developing paths, can offer nowadays to further extend the experimental characterisation of vibrating components.

Analysis of the LIFT Variance-Reduction Method Applied to Monte Carlo Radiation Transport Simulations of a Realistic Nonproliferation Test Problem

The Local Importance Function Transform (LIFT) method is a sophisticated automated variance-reduction technique for Monte Carlo simulation of radiation transport problems. In previous publications, the LIFT method was tested on geometrically simple problems with a coarse representation of radiation energy dependence, and the performance of the method was found to be promising when compared to traditional weight windows–based variance-reduction techniques. In this work, the LIFT method is tested on a spatially complex benchmark test problem with a more realistic representation of energy dependence (50 energy groups) and heterogeneous materials. The performance of the method in comparison with a CADIS (Consistent Adjoint Driven Importance Sampling)–based weight windows method and an analog Monte Carlo simulation is studied. A multigroup Monte Carlo code that utilizes portions of the framework of the deterministic tool Attila has been developed such that the overhead time in implementing the variance-reduction techniques is minimal. The Monte Carlo simulations are performed on an arbitrary tetrahedral mesh created by the mesh generator in Attila. A method to transfer the deterministic solution generated on a finer mesh to a coarser mesh for implementing the hybrid simulations has been developed, and the results are quantified.

Constrained min–max optimization via the improved constraint-activated differential evolution with escape vectors

In system design, the best system designed under a simple experimental environment may not be suitable for application in real world if dramatic changes caused by uncertainties contained in the real world are considered. To deal with the problem caused by uncertainties, designers should try their best to get the most robust solution. The most robust solution can be obtained by constrained min–max optimization algorithms. In this paper, the scheme of generating escape vectors has been proposed to solve the problem of premature convergence of differential evolution. After applying the proposed scheme to the constrained min–max optimization algorithm, the performance of the algorithm could be greatly improved. To evaluate the performance of constrained min–max optimization algorithms, more complex test problems have also been proposed in this paper. Experimental results show that the improved constrained min–max optimization algorithm is able to achieve a quite satisfied success rate on all considered test problems under limited accuracy.

Statistical significance of variables driving systematic variation in high-dimensional data

Motivation: There are a number of well-established methods such as principal component analysis (PCA) for automatically capturing systematic variation due to latent variables in large-scale genomic data. PCA and related methods may directly provide a quantitative characterization of a complex biological variable that is otherwise difficult to precisely define or model. An unsolved problem in this context is how to systematically identify the genomic variables that are drivers of systematic variation captured by PCA. Principal components (PCs) (and other estimates of systematic variation) are directly constructed from the genomic variables themselves, making measures of statistical significance artificially inflated when using conventional methods due to over-fitting.Results: We introduce a new approach called the jackstraw that allows one to accurately identify genomic variables that are statistically significantly associated with any subset or linear combination of PCs. The proposed method can greatly simplify complex significance testing problems encountered in genomics and can be used to identify the genomic variables significantly associated with latent variables. Using simulation, we demonstrate that our method attains accurate measures of statistical significance over a range of relevant scenarios. We consider yeast cell-cycle gene expression data, and show that the proposed method can be used to straightforwardly identify genes that are cell-cycle regulated with an accurate measure of statistical significance. We also analyze gene expression data from post-trauma patients, allowing the gene expression data to provide a molecularly driven phenotype. Using our method, we find a greater enrichment for inflammatory-related gene sets compared to the original analysis that uses a clinically defined, although likely imprecise, phenotype. The proposed method provides a useful bridge between large-scale quantifications of systematic variation and gene-level significance analyses.Availability and implementation: An R software package, called jackstraw, is available in CRAN.Contact: jstorey@princeton.edu

Simulation of Water-Table Aquifers Using Specified Saturated Thickness

Simulating groundwater flow in a water-table (unconfined) aquifer can be difficult because the saturated thickness available for flow depends on model-calculated hydraulic heads. It is often possible to realize substantial time savings and still obtain accurate head and flow solutions by specifying an approximate saturated thickness a priori, thus linearizing this aspect of the model. This specified-thickness approximation often relies on the use of the "confined" option in numerical models, which has led to confusion and criticism of the method. This article reviews the theoretical basis for the specified-thickness approximation, derives an error analysis for relatively ideal problems, and illustrates the utility of the approximation with a complex test problem. In the transient version of our complex test problem, the specified-thickness approximation produced maximum errors in computed drawdown of about 4% of initial aquifer saturated thickness even when maximum drawdowns were nearly 20% of initial saturated thickness. In the final steady-state version, the approximation produced maximum errors in computed drawdown of about 20% of initial aquifer saturated thickness (mean errors of about 5%) when maximum drawdowns were about 35% of initial saturated thickness. In early phases of model development, such as during initial model calibration efforts, the specified-thickness approximation can be a very effective tool to facilitate convergence. The reduced execution time and increased stability obtained through the approximation can be especially useful when many model runs are required, such as during inverse model calibration, sensitivity and uncertainty analyses, multimodel analysis, and development of optimal resource management scenarios.

New insights on permutation approach for hypothesis testing on functional data

The permutation approach for testing the equality of distributions and thereby comparing two populations of functional data has recently received increasing attention thanks to the flexibility of permutation tests to handle complex testing problems. The purpose of this work is to present some new insights in the context of nonparametric inference on functional data using the permutation approach, more specifically we formally show the equivalence of some permutation procedures proposed in the literature and we suggest the use of the permutation and combination-based approach within the basis function approximation layout. Validation of theoretical results is shown by simulation studies.

Robust non-parametric tests for complex-repeated measures problems in ophthalmology

The NonParametric Combination methodology (NPC) of dependent permutation tests allows the experimenter to face many complex multivariate testing problems and represents a convincing and powerful alternative to standard parametric methods. The main advantage of this approach lies in its flexibility in handling any type of variable (categorical and quantitative, with or without missing values) while at the same time taking dependencies among those variables into account without the need of modelling them. NPC methodology enables to deal with repeated measures, paired data, restricted alternative hypotheses, missing data (completely at random or not), high-dimensional and small sample size data. Hence, NPC methodology can offer a significant contribution to successful research in biomedical studies with several endpoints, since it provides reasonably efficient solutions and clear interpretations of inferential results. Pesarin F. Multivariate permutation tests: with application in biostatistics. Chichester-New York: John Wiley &Sons, 2001; Pesarin F, Salmaso L. Permutation tests for complex data: theory, applications and software. Chichester, UK: John Wiley &Sons, 2010. We focus on non-parametric permutation solutions to two real-case studies in ophthalmology, concerning complex-repeated measures problems. For each data set, different analyses are presented, thus highlighting characteristic aspects of the data structure itself. Our goal is to present different solutions to multivariate complex case studies, guiding researchers/readers to choose, from various possible interpretations of a problem, the one that has the highest flexibility and statistical power under a set of less stringent assumptions. MATLAB code has been implemented to carry out the analyses.

An evolutionary approach for multi-objective dynamic optimization applied to middle vessel batch distillation

In this paper, a new approach is presented to perform multi-objective dynamic optimizations of novel batch distillation utilizing an evolutionary algorithm. The contribution is divided into two major parts. First, the development of an efficient hybrid evolutionary algorithm covering multi-objective mixed integer dynamic optimization problems is presented. The efficiency of the optimization solver is proven by several complex test problems. Second, the application of the algorithm is shown by the optimization of a middle vessel batch distillation. The challenging non-linear dynamic model, which takes the start-up phase into account, is solved in Aspen Custom Modeler. It could be proven that the proposed evolutionary algorithm can be applied to complex mathematical problems. Likewise the algorithm was found to successfully handle the optimization of middle a vessel batch distillation. The results show the feasibility of the proposed methodology and a significant increase in profitability of the process.

Some new results on univariate and multivariate permutation tests for ordinal categorical variables under restricted alternatives

In several sciences, especially when dealing with performance evaluation, complex testing problems may arise due in particular to the presence of multidimensional categorical data. In such cases the application of nonparametric methods can represent a reasonable approach. In this paper, we consider the problem of testing whether a “treatment” is stochastically larger than a “control” when univariate and multivariate ordinal categorical data are present. We propose a solution based on the nonparametric combination of dependent permutation tests (Pesarin in Multivariate permutation test with application to biostatistics. Wiley, Chichester, 2001), on variable transformation, and on tests on moments. The solution requires the transformation of categorical response variables into numeric variables and the breaking up of the original problem’s hypotheses into partial sub-hypotheses regarding the moments of the transformed variables. This type of problem is considered to be almost impossible to analyze within likelihood ratio tests, especially in the multivariate case (Wang in J Am Stat Assoc 91:1676–1683, 1996). A comparative simulation study is also presented along with an application example.

Comparison of Algorithms for Nonlinear Integer Optimization: Application to Monitoring Network Design

This paper presents an empirical comparison of eight heuristic algorithms for solving nonlinear integer optimization problems. The algorithms are first applied to a simple network location problem to assess their relative performance. The best performing algorithms are then applied to a more complex test problem involving an irregular correlation surface and partitioned spatial domain, typical of a ground-water monitoring network design application. Simulated annealing and TABU search exhibit superior performance among the six heuristic methods and two supplemental nonlinear integer optimization solvers that were tested.

A 1.9-GHz GaAs chip set for the personal handyphone system

The Japanese Personal Handyphone System (PHS) is representative of the latest generation of digital portable communications systems currently being deployed. Enabling technologies for these systems include high performance Radio Frequency Integrated Circuit (RFIC) chip sets. These chip sets allow all the RF transceiver functions to be included in low cost surface mount plastic packages. With the addition of filters and bypassing capacitors, the RF portion of the phone shares the same printed circuit board (PCB) as the DSP, CODEC, and logic ICs. The availability of such highly integrated 1.9-GHz RFICs requires the solution of many complex design, manufacturing, and test problems. This paper explains the critical issues relating to the air interface of the PHS system and how it affects the RFIC design. The chip partition, design, and performance of each subfunction is discussed relative to the requirements imposed by the air interface. The result is a highly integrated, cost effective solution that occupies the minimum board area.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

An Extension of the Csörgő-Horváth Functional Limit Theorem and Its Applications to Changepoint Problems

Consider a triangular array X n 1, ..., X n n , n ∈ N , of rowwise independent random elements with values in a measurable space. Suppose there exists θ ∈ [0, 1)such that X n 1, ..., X n [ nθ ] have distribution ν 1 and X n [ nθ] + 1 (n), ..., X n n have distribution ν 2. Csörgő and Horváth derived an invariance principle for a one-time parameter process, which is the foundation of a test for H 0 : θ = 0 versus H 1: θ ∈(0, 1). We are interested in the more complex test problem H̃ 0: θ ∈ Θ 0 versus H̃ 1,: θ ∈ Θ 0, where Θ 0⊆(0, 1). To treat this new situation, we extend the Csörgő-Horváth result in proving a functional limit theorem for a suitable two-time parameter process. We briefly sketch several applications of our result. Especially, the power of the Csörgő-Horváth test is investigated in detail.

Automated synthesis of pseudo-exhaustive test generator in VLSI BIST design

Built-In Self Test (BIST) has been proposed as a powerful technique for addressing the highly complex testing problems of VLSI circuits. In the BIST methodology, two major problems which must be addressed are test generation and response analysis. In this paper, we present an efficient unified procedure, named three-phase cluster partitioning, to automatically synthesize a pseudo-exhaustive test generator for VLSI BIST design. Previous approaches to the problem of test generation have optimized computational efficiency at the expense of the required hardware overhead, or vice versa. Our design procedure is computationally efficient and produces test generation circuitry with low hardware overhead. The procedure minimizes the number of test patterns that are required for pseudo-exhaustive test. Based on three-phase cluster partitioning, a design generator named BISTSYN has been developed and implemented to facilitate the BIST design. The input to the design generator is a circuit description at the gate level which is viewed as a netlist. BISTSYN provides the BIST mechanisms as the output. For those conventional circuits which are extremely unsuitable for pseudo-exhaustive test, BISTSYN employs a circuit partitioning tool, named Autonomous, to partition the combinational portion of the circuit into different structural subcircuits so that each subcircuit can be pseudo-exhaustively tested. We demonstrate the effectiveness of BISTSYN by applying the method to different examples and practical VLSI designs. The detailed comparisons of our benchmark simulation results against those that would be obtained by existing techniques are also presented. >

On a Nonparametric Combination Method for Dependent Permutation Tests with Applications

This paper deals with a combination method for dependent permutation tests, which is nonparametric with respect to the underlying unknown dependence structure. The method is based on a simulation or resampling procedure, conditional on the data, which provides a simulated estimate of the permutation distribution of any statistic. Applications to some unusual and quite complex testing problems are shown.

Using the Bootstrap to Assess Statistical Significance in the Cluster Analysis of Species Abundance Data

Clustering techniques are frequently used to analyze species abundance data, despite a lack of objective criteria for assessing the results of such analyses. We show that a nonparametric statistical technique known as the "bootstrap" can, in certain circumstances, be used to overcome this deficiency. The bootstrap uses the distributional properties of a "bootstrap sample", i.e. a sample obtained by an appropriate resampling of the data, to make statistical inferences about the underlying population. The method is versatile and can be readily applied to complex hypothesis testing problems, since the statistical properties of a bootstrap sample can always be determined by simulation. To illustrate the application of the bootstrap to the cluster analysis of ecological data, we derive a test for the "statistical significance" of clusters of communities and show that two dendrograms can be compared by bootstrapping the Fowlkes–Mallow statistic.