Nonlinear Decision Rules Research Articles

Models and methods for RZ-signals distinction in non-Gaussian noise for information-measurement systems

Abstract Polynomial Decision Rules (DR) for the problems of discrete RZ (Return-to-Zero) signal distinction in asymmetric-excess non-Gaussian noise are proposed. A new approach is proposed, which is based on the use of a modified moment quality criterion of statistical hypothesis testing and the application of higher-order statistics to describe the characteristics of non-Gaussian noise. Simulation of the DR with different parameters of the signal and noise was carried out. It is shown that taking into account the coefficients of skewness and kurtosis of the non-Gaussian noise the efficiency of signal distinction increases with non-linear processing DR compared to known results, which are optimal for the Gaussian noise model. The conducted studies demonstrate a reduction in false decisions in the processing of RZ signals when considering the coefficients of skewness and kurtosis of non-Gaussian noise. Such an increase in efficiency can exceed twofold, depending on the noise parameters. It is shown that the efficiency of the proposed approach is much higher for small SNR (Signal-to-Noise Ratio) values, for example, less than 1.

Nonlinear Decision Rules Made Scalable by Nonparametric Liftings

Sequential decision making often requires dynamic policies, which are computationally not tractable in general. Decision rules provide approximate solutions by restricting decisions to simple functions of uncertainties. In this paper, we consider a nonparametric lifting framework where the uncertainty space is lifted to higher dimensions to obtain nonlinear decision rules. Current lifting-based approaches require predetermined functions and are parametric. We propose two nonparametric liftings, which derive the nonlinear functions by leveraging the uncertainty set structure and problem coefficients. Both methods integrate the benefits from lifting and nonparametric approaches, and hence provide scalable decision rules with performance bounds. More specifically, the set-driven lifting is constructed by finding polyhedrons within uncertainty sets, inducing piecewise-linear decision rules with performance bounds. The dynamics-driven lifting, on the other hand, is constructed by extracting geometric information and accounting for problem coefficients. This is achieved by using linear decision rules of the original problem, also enabling one to quantify lower bounds of objective improvements over linear decision rules. Numerical comparisons with competing methods demonstrate superior computational scalability and comparable performance in objectives. These observations are magnified in multistage problems with extended time horizons, suggesting practical applicability of the proposed nonparametric liftings in large-scale dynamic robust optimization. This paper was accepted by David Simchi-Levi, optimization. Supplemental Material: The electronic companion and data files are available at https://doi.org/10.1287/mnsc.2024.4988 .

Adapting reservoir operation rules to hydrological drought state and environmental flow requirements

Negligence of environmental flow (e-flow) has severe adverse impacts on wetlands. Additionally, hydrological droughts, compounded by mismanagement of available water resources and anthropogenic activities, lead to water shortage and degrade wetland ecosystem. However, in dealing with reservoir operation/re-operation, a few studies have addressed inflow uncertainty and hydrological drought state to improve the monthly operation performance of an upstream reservoir in coordination with downstream wetland needs. Hence, the study objective is to adopt a linear/nonlinear reservoir operation rule to hydrological drought states and e-flow requirements. So, first, the soil and water assessment (SWAT) model is applied to simulate the natural daily flow regime by eliminating human impacts. Then, using the Tennant e-flow scheme, the intra/inter-annual variability of e-flow and the degree of alteration of the flow are measured. Afterward, using the Standardized Hydrological Drought Index (SHDI), the drought/wet severity and duration are determined. A linear/nonlinear simulation model adopted for the presentation of reservoir policies and the non-dominated sorting genetic algorithm-II (NSGA-II) was employed to determine the optimal monthly operation of the upstream reservoir. The simulated operation policies correspond to each hydrological drought state were evaluated with reliability, resilience, vulnerability, and water-deficit indices. Results indicated that incorporation of hydrological drought state could improve the overall system reliability by 23% over simply enforcing the monthly variable-parameter nonlinear decision rules. Also, the proposed methodology simultaneously maximized the downstream allocated water and e-flow of the terminal wetland. It also implicitly minimized the change in extreme flow metrics that are known to benefit ecosystem diversity. The proposed framework supports reservoir re-operation in regions subject to challenges in supplying water to different sectors while maintaining e-flow during drought periods.

Multistage robust mixed-integer optimization under endogenous uncertainty

Endogenous, i.e. decision-dependent, uncertainty has received increased interest in the stochastic programming community. In the robust optimization context, however, it has rarely been considered. This work addresses multistage robust mixed-integer optimization with decision-dependent uncertainty sets. The proposed framework allows us to consider both continuous and integer recourse, including recourse decisions that affect the uncertainty set. We derive a tractable reformulation of the problem by leveraging recent advances in the construction of nonlinear decision rules, and introduce discontinuous piecewise linear decision rules for continuous recourse. Computational experiments are performed to gain insights on the impact of endogenous uncertainty, the benefit of discrete recourse, and computational performance. Our results indicate that the level of conservatism in the solution can be significantly reduced if endogenous uncertainty and mixed-integer recourse are properly modeled.

Hybrid strategies using linear and piecewise-linear decision rules for multistage adaptive linear optimization

Decision rules offer a rich and tractable framework for solving certain classes of multistage adaptive optimization problems. Recent literature has shown the promise of using linear and nonlinear decision rules in which wait-and-see decisions are represented as functions, whose parameters are decision variables to be optimized, of the underlying uncertain parameters. Despite this growing success, solving real-world stochastic optimization problems can become computationally prohibitive when using nonlinear decision rules, and in some cases, linear ones. Consequently, decision rules that offer a competitive trade-off between solution quality and computational time become more attractive. Whereas the extant research has always used homogeneous (i.e., either linear or piecewise-linear) decision rules, the major contribution of this paper is a computational exploration of hybrid decision rules combining the benefits of the two classes of decision rules. We first verify empirically that having higher uncertainty resolution or more linear pieces in early stages is more significant than having it in late stages in terms of solution quality. Then, we compare non-increasing and non-decreasing (i.e., higher uncertainty resolution in early and late stages, respectively) hybrid decision rules in a computational study to illustrate the trade-off between solution quality and computational cost. We also demonstrate a case where, unexpectedly, a linear decision rule is superior to a more complex piecewise-linear decision rule within a simulator. This observation bolsters the need to assess the quality of decision rules obtained from a look-ahead model within a simulator rather than just using the optimal look-ahead objective function value.

The Brain–Computer Interface: Experience of Construction, Use, and Potential Routes to Improving Performance

Neurocomputer interfaces or, as they have come to be known in the Russian literature, brain–computer interfaces (BCI), are used in several areas and have the potential for uses in solving both research and applied tasks. Pilot studies in the clinical application of BCI to poststroke neurorehabilitation are currently under way [Frolov et al., 2013; Ang et al., 2010], and there are prospects for the use of BCI for direct restoration of movement/communication capabilities by creating an alternative information exchange channel with intelligent prostheses and the surroundings. Studies using electrophysiological data generate the need to process multidimensional, nonstationary signals, refl ecting complex physiological processes. Interfaces based on noninvasive technologies for recording brain activity do not as yet provide reliable information links with the user’s brain. The results of our studies show that improvements in the working characteristics of these systems can be obtained by constructing new machine learning algorithms considering the physiological and psychoemotional characteristics of BCI use. These algorithms can be developed either in the classical Bayesian paradigm or using state-of-the-art deep learning techniques. In addition, the creation of methods for the physiological interpretation of nonlinear decision rules found by multilayered structures opens up new potentials for the automatic and objective extraction of knowledge from experimental neurophysiological data. Despite the attractiveness of noninvasive technologies, radical increases in the throughput of BCI communication channels and the use of this technology to control prostheses can only be obtained using invasive methods of recording brain activity. Electrocorticograms (ECoG) are the least invasive of these technologies, and in the concluding part of this work we will demonstrate that ECoG can be used for decoding of the kinematic characteristics of finger movements.

Judgment Analysis in a Dynamic Multitask Environment: Capturing Nonlinear Policies Using Decision Trees

Policy capturing is a judgment analysis method that typically uses linear statistical modeling to estimate expert judgments. A variant to this technique is to capture decision policies using data-mining algorithms designed to handle nonlinear decision rules, missing attributes, and noisy data. In the current study, we tested the effectiveness of a decision-tree induction algorithm and an instance-based classification method for policy capturing in comparison to the standard linear approach. Decision trees are relevant in naturalistic decision-making contexts since they can be used to represent “fast-and-frugal” judgment heuristics, which are well suited to describe human cognition under time pressure. We examined human classification behavior using a simulated naval air defense task in order to empirically compare the C4.5 decision-tree algorithm, the k-nearest neighbors algorithm, and linear regression on their ability to capture individual decision policies. Results show that C4.5 outperformed the other methods in terms of goodness of fit and cross-validation accuracy. Decision-tree models of individuals’ judgment policies actually classified contacts more accurately than their human counterparts, resulting in a threefold reduction in error rates. We conclude that a decision-tree induction algorithm can yield useful models for training and decision support applications, and we discuss the application of judgmental bootstrapping in real time in dynamic environments.

The impact of the existence of multiple adjustable robust solutions

In this note we show that multiple solutions exist for the production-inventory example in the seminal paper on adjustable robust optimization in Ben-Tal et al. (Math Program 99(2):351---376, 2004). All these optimal robust solutions have the same worst-case objective value, but the mean objective values differ up to 21.9 % and for individual realizations this difference can be up to 59.4 %. We show via additional experiments that these differences in performance become negligible when using a folding horizon approach. The aim of this paper is to convince users of adjustable robust optimization to check for existence of multiple solutions. Using the production-inventory example and an illustrative toy example we deduce three important implications of the existence of multiple optimal robust solutions. First, if one neglects this existence of multiple solutions, then one can wrongly conclude that the adjustable robust solution does not outperform the nonadjustable robust solution. Second, even when it is a priori known that the adjustable and nonadjustable robust solutions are equivalent on worst-case objective value, they might still differ on the mean objective value. Third, even if it is known that affine decision rules yield (near) optimal performance in the adjustable robust optimization setting, then still nonlinear decision rules can yield much better mean objective values.

Generalized decision rule approximations for stochastic programming via liftings

Stochastic programming provides a versatile framework for decision-making under uncertainty, but the resulting optimization problems can be computationally demanding. It has recently been shown that primal and dual linear decision rule approximations can yield tractable upper and lower bounds on the optimal value of a stochastic program. Unfortunately, linear decision rules often provide crude approximations that result in loose bounds. To address this problem, we propose a lifting technique that maps a given stochastic program to an equivalent problem on a higher-dimensional probability space. We prove that solving the lifted problem in primal and dual linear decision rules provides tighter bounds than those obtained from applying linear decision rules to the original problem. We also show that there is a one-to-one correspondence between linear decision rules in the lifted problem and families of nonlinear decision rules in the original problem. Finally, we identify structured liftings that give rise to highly flexible piecewise linear and nonlinear decision rules, and we assess their performance in the context of a dynamic production planning problem.

Learning Nonlinear Functions Using Regularized Greedy Forest.

We consider the problem of learning a forest of nonlinear decision rules with general loss functions. The standard methods employ boosted decision trees such as Adaboost for exponential loss and Friedman's gradient boosting for general loss. In contrast to these traditional boosting algorithms that treat a tree learner as a black box, the method we propose directly learns decision forests via fully-corrective regularized greedy search using the underlying forest structure. Our method achieves higher accuracy and smaller models than gradient boosting on many of the datasets we have tested on.

Developing reservoir operational decision rule by genetic programming

The reservoir operational decision rule is an equation that can balance reservoir system parameters in each period by considering previous experiences of the system. That equation includes variables such as inflow, volume storage and released water from the reservoir that are commonly related to each other by some constant coefficients in predefined linear and nonlinear patterns. Although optimization tools have been extensively applied to develop an optimal operational decision rule, only optimal constant coefficients have been derived and the operational patterns are assumed to be fixed in that operational rule curve. Genetic programming (GP) is an evolutionary algorithm (EA), based on genetic algorithm (GA), which is capable of calculating an operational rule curve by considering optimal operational undefined patterns. In this paper, GP is used to extract optimal operational decision rules in two case studies by meeting downstream water demands and hydropower energy generation. The extracted rules are compared with common linear and nonlinear decision rules, LDR and NLDR, determined by a software package for interactive general optimization (LINGO) and GA. The GP rule improves the objective functions in the training and testing data sets by 2.48 and 8.53%, respectively, compared to the best rule by LINGO and GA in supplying downstream demand. Similarly, the hydropower energy generation improves by 48.03 and 44.21% in the training and testing data sets, respectively. Results show that the obtained objective function value is enhanced significantly for both the training and testing data using GP. They also indicate that the proposed rule, based on GP, is effective in determining optimal rule curves for reservoirs.

Non-linear signal detection algorithms on background additive-multiplicative non-gaussian noise

The problem of signal detection on the background noise is one of the most important and actual in many application s . The most complete model of the interaction of signals and noise is additive-multiplicative model, which is typical for many practical cases. The task signal processing is more difficult when considering Non-Gaussian noise, both additive and multiplicative character. Therefore, the actual task is the development of methods and algorithms for statistical signal processing on the background additive-multiplicative Non-Gaussian noise in radar, sonar, geophysics, systems of mobile communications. This paper illustrates the use of an alternative approach to the description of random variables - moment– cumulant description. In this work the additional question of efficient algorithms for signal detection on a background additive-multiplicative Non-Gaussian noise are consider. Purpose of work is improve the efficiency of systems Signal Detection on a background additive-multiplicative Non-Gaussian noise on the basis of polynomial decision rules, that are optimal for the moment criterion. Presented synthesis of decision rules radiosignal detection on the background additive-multiplicative Non-Gaussian noise to the degree stochastic polynomial . It is shown that with increasing degree stochastic polynomial and considering Non-Gaussian noise, value of the criterion of the upper limit of probability of errors of the first and second kind decreases, indicating a decrease in the probability of errors synthesized nonlinear decision rules and increase their effectiveness in comparison with linear decision rules

Scenario Trees and Policy Selection for Multistage Stochastic Programming Using Machine Learning

In the context of multistage stochastic optimization problems, we propose a hybrid strategy for generalizing to nonlinear decision rules, using machine learning, a finite data set of constrained vector-valued recourse decisions optimized using scenario-tree techniques from multistage stochastic programming. The decision rules are based on a statistical model inferred from a given scenario-tree solution and are selected by out-of-sample simulation given the true problem. Because the learned rules depend on the given scenario tree, we repeat the procedure for a large number of randomly generated scenario trees and then select the best solution (policy) found for the true problem. The scheme leads to an ex post selection of the scenario tree itself. Numerical tests evaluate the dependence of the approach on the machine learning aspects and show cases where one can obtain near-optimal solutions, starting with a “weak” scenario-tree generator that randomizes the branching structure of the trees.

Hidden Convexity in Partially Separable Optimization

The paper identifies classes of nonconvex optimization problems whose convex relaxations have optimal solutions which at the same time are global optimal solutions of the original nonconvex problems. Such a hidden convexity property was so far limited to quadratically constrained quadratic problems with one or two constraints. We extend it here to problems with some partial separable structure. Among other things, the new hidden convexity results open up the possibility to solve multi-stage robust optimization problems using certain nonlinear decision rules.

Stochastic Capital Depreciation and the Co-movement of Hours and Productivity

An unresolved question concerning stochastic depreciation shocks is whether they have to be unrealistically large to have any useful role in a dynamic general equilibrium model economy, as Ambler and Paquet (1994) first suggested. We first consider implied depreciation rates from sectoral data from the Bureau of Economic Analysis. These depreciation rates vary across time solely due to compositional changes within each sector. Hence, they tend to understate the range of fluctuation that would hold if the economic shelf life of capital varied endogenously as in Cooley et al. (1997). We find, however, that if depreciation rates follow a Markov switching process, a low variance of the depreciation rate is sufficient to allow a model economy to match the low correlation between hours worked and productivity observed in the data. White noise and autoregressive depreciation shocks, in contrast, require a counterfactually large variance in the depreciation rate to reduce the hours-productivity correlation. We also illustrate the level effects implied by nonlinear decision rules in simulations of dynamic general equilibrium models that include Markov switching parameters. Linear decision rules, in contrast, imply certainty equivalence and ignore the aversion that agents have to the skewed shock distributions that characterize Markov switching.

The effect of nonlinear human visual system components on performance of a channelized Hotelling observer in structured backgrounds

Linear model observers based on statistical decision theory have been used successfully to predict human visual detection of aperiodic signals in a variety of noisy backgrounds. However, some models have included nonlinearities such as a transducer or nonlinear decision rules to handle intrinsic uncertainty. In addition, masking models used to predict human visual detection of signals superimposed on one of two identical backgrounds (masks) usually include a number of nonlinear components in the channels that reflect properties of the firing of cells in the primary visual cortex (V1). The effect of these nonlinearities on the ability of linear model observers to predict human signal detection in real patient structured backgrounds is unknown. We evaluate the effect of including different nonlinear human visual system components into a linear channelized Hotelling observer (CHO) using a signal known exactly but variable (SKEV) task. In particular, we evaluate whether the rank order of two compression algorithms (JPEG versus JPEG 2000) and two compression encoder settings (JPEG 2000 default versus JPEG 2000 optimized) based on model observer signal detection performance in X-ray coronary angiograms is altered by inclusion of nonlinear components. The results show: 1) the simpler linear CHO model observer outperforms CHO model with the nonlinear components; 2) the rank order of model observer performance for the compression algorithms/parameters does not change when the nonlinear components are included. For the present task and images, the results suggest that the addition of the nonlinearities to a channelized Hotelling model may add complexity to the model observers without great impact on rank order evaluation of image processing and/or acquisition algorithms.

Complex decision rules in categorization: Contrasting novice and experienced performance.

not b e easily verbalized. Therefore, this article focuses on nonlinear decision rules in perceptual categorization that have no simple verbal description. Some studies have focused on such categorization problems (e.g., Medin & Schwanenflugel, 1981; Nosofsky, 1986, 1987, 1989; Shepard & Chang, 1963; Wattenmaker, Dewey, Murphy, & Medin, 1986). In these experiments, however, many different nonlinear decision rules predicted the same categorization accuracy. Therefore, although i t was shown that subjects did use a nonlinear rule, the design of the experiments made it impossible to determine exactly which of the many nonlinear rules were used. The present studies extend this line of research in several respects. We use an experimental design in which unique decision rules lead t o unique patterns of responding. This makes i t possible t o identify uniquely a subject's decision rule.'