Optimal Decision Rules Research Articles

Discounted fully probabilistic design of decision rules

Axiomatic fully probabilistic design (FPD) of optimal decision rules strictly extends the decision making (DM) theory represented by Markov decision processes (MDP). This means that any MDP task can be approximated by an explicitly found FPD task whereas many FPD tasks have no MDP equivalent. MDP and FPD model the closed loop — the coupling of an agent and its environment — via a joint probability density (pd) relating the involved random variables, referred to as behaviour. Unlike MDP, FPD quantifies agent's aims and constraints by an ideal pd. The ideal pd is high on the desired behaviours, small on undesired behaviours and zero on forbidden ones. FPD selects the optimal decision rules as the minimiser of Kullback-Leibler's divergence of the closed-loop-modelling pd to its ideal twin. The proximity measure choice follows from the FPD axiomatics.MDP minimises the expected total loss, which is usually the sum of discounted partial losses. The discounting reflects the decreasing importance of future losses. It also diminishes the influence of errors caused by:▪ the imperfection of the employed environment model;▪ roughly-expressed aims;▪ the approximate learning and decision-rules design.The established FPD cannot currently account for these important features. The paper elaborates the missing discounted version of FPD. This non-trivial filling of the gap in FPD also employs an extension of dynamic programming, which is of an independent interest.

Bivariate BMM-based hybrid domain image watermark detector

Robustness, imperceptibility, and watermark capacity are three indispensable and contradictory properties for any image watermarking systems. It is a challenging work to achieve the balance among the three important properties. In this paper, by using the bivariate Beta mixture model (Bivariate BMM), we present a statistical image watermark scheme in nonsubsampled Contourlet transform (NSCT)-fractional order Jacobi Fourier moments (FJFMs) amplitude hybrid domain. The whole watermarking algorithm includes two parts: watermark embedding and detection. NSCT is firstly performed on host image to obtain the frequency subbands, and the NSCT subbands are divided into non overlapping blocks. Then, the significant NSCT domain blocks are selected using local binary patterns (LBP). Meanwhile, for each selected NSCT coefficient block, FJFMs are calculated to obtain the NSCT-FJFMs amplitude. Finally, watermark signals are inserted into the amplitude hybrid domain of NSCT-FJFMs. In order to detect accurately watermark signal, the statistical characteristics of NSCT-FJFMs magnitudes are analyzed in detail. Then, NSCT-FJFMs magnitudes are described statistically by Bivariate BMM, which can simultaneously capture the marginal distribution and strong dependencies of NSCT-FJFMs magnitudes. Also, Bivariate BMM parameters are estimated accurately by the rough-enhanced-Bayes mixture estimation & expectation–maximization (REBMIX&EM) approach. Finally, a statistical watermark detector based on the locally optimum (LO) decision rule and Bivariate BMM is developed in NSCT-FJFMs magnitude hybrid domain. Also, the closed-form expressions on the LO statistic is derived and the receiver operating characteristic (ROC) about our detector is analyzed. Extensive experimental results show the superiority of the proposed image watermark detector over some state-of-the-art statistical watermarking methods.

Dispersal of optimal foragers in a patchy environment: Simulations with a mathematical model and tests of predictions in field experiments

To predict the effect of density on the dispersal of foraging parasitoids, we developed a spatially explicit individual-based model in which parasitoids move among host patches at random but use an optimal decision-rule about when to leave these patches. We used a simple decision rule where an individual forager exploits a patch as long as the instantaneous attack rate is higher than the average attack rate it has experienced in the environment. Such a rule implies that foragers compute and remember attack rates as they forage. Simulations with different combinations of patch distribution, host density, mutual interference, and parasitoid density predicted that dispersal distance should increase with parasitoid density. To test this prediction, we used data from two field experiments in which we released either few or many adults of the aphid parasitoid Aphelinus asychis in replicated sites, and subsequently assessed dispersal from the spatial distribution of parasitoid offspring. In the first experiment, we did not to detect a relation between the number released and dispersal distance, but in the second experiment, dispersal distance increased with initial density, as predicted by our model. We propose two hypotheses to explain the discrepancy between the experiments. Different levels of environmental variability among experiments, resulting from differences in experimental designs, could cause differences in statistical conclusions. However, there could be a threshold density below which dispersal is not density-dependent, and this threshold may have been exceeded in the second experiment where large releases involved many more individuals than in the first experiment. In any case, our approach linked individual behavior and spatial distribution, and our results show that the insect distributions in the field can be predicted, qualitatively, from theory about individual behavior.

Estimating optimal decision trees for treatment assignment: The case of K > 2 treatment alternatives.

For many problems in clinical practice, multiple treatment alternatives are available. Given data from a randomized controlled trial or an observational study, an important challenge is to estimate an optimal decision rule that specifies for each client the most effective treatment alternative, given his or her pattern of pretreatment characteristics. In the present paper we will look for such a rule within the insightful family of classification trees. Unfortunately, however, there is dearth of readily accessible software tools for optimal decision tree estimation in the case of more than two treatment alternatives. Moreover, this primary tree estimation problem is also cursed with two secondary problems: a structural missingness in typical studies on treatment evaluation (because every individual is assigned to a single treatment alternative only), and a major issue of replicability. In this paper we propose solutions for both the primary and the secondary problems at stake. We evaluate the proposed solution in a simulation study, and illustrate with an application on the search for an optimal tree-based treatment regime in a randomized controlled trial on K = 3 different types of aftercare for younger women with early-stage breast cancer. We conclude by arguing that the proposed solutions may have relevance for several other classification problems inside and outside the domain of optimal treatment assignment.

Multi-modal incomplete label information three-way bidirectional decision-making: Applications of disease assessment

Disease assessment involves two stages of decision-making process. The first process is to infer the development trend of the disease through the existing judgment conditions, which is to make a positive estimation (prediction) decision. The second process is to update the judgment conditions for different disease states and optimize the clinical assessment evidence by fusing disease progression data, that is, to make a reverse evaluation (optimization) decision by reviewing historical diagnostic and treatment data to summarize the best judgment conditions for the disease characteristics. Therefore, we construct a three-way bidirectional decision method for multi-modal incomplete labeling information, considering the mixed data features with no labels and labels, and the bidirectional decision features of positive prediction and reverse optimization. Specifically, the method first introduces semi-supervised learning (SSL) into the rough sets, constructs a rough sets model based on cosine similarity relation, generates pseudo-labels for incomplete information, and explains the positive prediction decision process principle. Second, the gradient space is constructed on the predictive information system by the three-way decision-making principle, the loss function formed by the combination of KL divergence and cross entropy loss function is used to find the direction of the fastest decline in attribute information value, and the optimal decision rule and the automatic updating strategy of the threshold value are obtained. At the same time, the principle of the reverse optimization decision process is explained. Finally, multimodal information from pathological tissue images is used to evaluate clinical progress problem. A pathological tissue images dataset includes 2000 colorectal cancer and 300 kidney disease cases used for simulation experiments. The proposed three-way bidirectional decision-making model can be widely applied to actual decision-making scenarios for evaluating clinical progress of different diseases. Compared with previous studies, this method can realize objective calculation and automatic update of threshold and loss values and effectively solve complex prediction and optimization decision-making problems.

Optimal decision rules for marked point process models

We study a Markov decision problem in which the state space is the set of finite marked point patterns in the plane, the actions represent thinnings, the reward is proportional to the mark sum which is discounted over time, and the transitions are governed by a birth-death-growth process. We show that thinning points with large marks maximises the discounted total expected reward when births follow a Poisson process and marks grow logistically. Explicit values for the thinning threshold and the discounted total expected reward over finite and infinite horizons are also provided. When the points are required to respect a hard core distance, upper and lower bounds on the discounted total expected reward are derived.

Adjustability in robust linear optimization

AbstractWe investigate the concept of adjustability—the difference in objective values between two types of dynamic robust optimization formulations: one where (static) decisions are made before uncertainty realization, and one where uncertainty is resolved before (adjustable) decisions. This difference reflects the value of information and decision timing in optimization under uncertainty, and is related to several other concepts such as the optimality of decision rules in robust optimization. We develop a theoretical framework to quantify adjustability based on the input data of a robust optimization problem with a linear objective, linear constraints, and fixed recourse. We make very few additional assumptions. In particular, we do not assume constraint-wise separability or parameter nonnegativity that are commonly imposed in the literature for the study of adjustability. This allows us to study important but previously under-investigated problems, such as formulations with equality constraints and problems with both upper and lower bound constraints. Based on the discovery of an interesting connection between the reformulations of the static and fully adjustable problems, our analysis gives a necessary and sufficient condition—in the form of a theorem-of-the-alternatives—for adjustability to be zero when the uncertainty set is polyhedral. Based on this sharp characterization, we provide two efficient mixed-integer optimization formulations to verify zero adjustability. Then, we develop a constructive approach to quantify adjustability when the uncertainty set is general, which results in an efficient and tight poly-time algorithm to bound adjustability. We demonstrate the efficiency and tightness via both theoretical and numerical analyses.

Distributed Sequential Detection for Cooperative Spectrum Sensing in Cognitive Internet of Things.

The rapid development of wireless communication technology has led to an increasing number of internet of thing (IoT) devices, and the demand for spectrum for these devices and their related applications is also increasing. However, spectrum scarcity has become an increasingly serious problem. Therefore, we introduce a collaborative spectrum sensing (CSS) framework in this paper to identify available spectrum resources so that IoT devices can access them and, meanwhile, avoid causing harmful interference to the normal communication of the primary user (PU). However, in the process of sensing the PUs signal in IoT devices, the issue of sensing time and decision cost (the cost of determining whether the signal state of the PU is correct or incorrect) arises. To this end, we propose a distributed cognitive IoT model, which includes two IoT devices independently using sequential decision rules to detect the PU. On this basis, we define the sensing time and cost functions for IoT devices and formulate an average cost optimization problem in CSS. To solve this problem, we further regard the optimal sensing time problem as a finite horizon problem and solve the threshold of the optimal decision rule by person-by-person optimization (PBPO) methodology and dynamic programming. At last, numerical simulation results demonstrate the correctness of our proposal in terms of the global false alarm and miss detection probability, and it always achieves minimal average cost under various costs of each observation taken and thresholds.

Optimal strategy for Bayesian two-armed bandit problem with an arched reward function

In the classical two-armed bandit (TAB) problems, the goal of maximizing the total return specifies the optimal decision rule and fosters the development of abundant practicable methods, such as $ \epsilon $-greedy and upper confidence bound for TAB procedures. However, the existence of an arched reward function for the total return breaks the previously updated algorithms, like traditional myopic strategy, and prompts the generation of novel computation and theory for maximizing its expectation. Here we develop a special class of Bayesian two-armed bandit problems by imposing a prior probability on which arm has a greater expectation of the returns and propose a myopic strategy to specify the decision rule. We prove that the proposed myopic strategy is optimal under the structure of the arched reward function by exploring the dynamic programming principle. This article also demonstrates that arched functions are extensions of the monotone functions including the classical linear functions. Meanwhile, we establish a corresponding law of large numbers for our myopic strategy. Simulation studies pose supportive evidence that the newly proposed strategy performs well.

Optimal sample sizes and statistical decision rules

A statistical decision rule is a mapping from data to actions induced by statistical inference on the data. We characterize these rules for data that are chosen strategically in persuasion environments. A designer wishes to persuade a decision maker (DM) to take a particular action and decides how many Bernoulli experiments about a parameter of interest the DM can obtain. After obtaining these data and estimating the parameter value, the DM chooses to take the action if the estimated value exceeds some threshold. We establish that as the threshold changes, the resulting statistical decision rules in many environments are either simple majority or reverse unanimity.

Accommodating misclassification effects on optimizing dynamic treatment regimes with Q-learning.

Research on dynamic treatment regimes has enticed extensive interest. Many methods have been proposed in the literature, which, however, are vulnerable to the presence of misclassification in covariates. In particular, although Q-learning has received considerable attention, its applicability to data with misclassified covariates is unclear. In this article, we investigate how ignoring misclassification in binary covariates can impact the determination of optimal decision rules in randomized treatment settings, and demonstrate its deleterious effects on Q-learning through empirical studies. We present two correction methods to address misclassification effects on Q-learning. Numerical studies reveal that misclassification in covariates induces non-negligible estimation bias and that the correction methods successfully ameliorate bias in parameter estimation.

Poster Session I: How important is it to know your own limitations when making perceptual decisions?

Many models of perceptual performance consist of two components: (1) a representation of the available sensory and memory information, and (2) a Bayes-optimal decision computation. If the model accurately predicts performance, then that finding is often interpreted as evidence for both components of the model. However, it is important and informative to also test heuristic decision rules. Here, we illustrate such analyses for the task of covert visual search, where a target can appear at one of n locations (or be absent) with some prior probability distribution (the "prior map"). The available sensory information can be represented by the signal-to-noise ratio at each retinal location (the "d' map") together with the prior map. It can be shown that the Bayes-optimal decision rule is to multiply the response at each location by the d' at that location, add the log prior for that location, and then pick the location that has the maximum. For a wide range of d' and prior maps we simulated performance for the optimal and various heuristic decision rules. As expected, varying the d' and prior maps has a large effect on performance due to the variation in the available sensory information. However, a wide range of simple decision rules perform almost as well as the Bayes-optimal rule. In other words, in the covert search task, our decision processes can be idiosyncratic and almost completely ignorant of the d' and prior maps and still perform optimally.

Reflections on the concept of optimality of single decision point treatment regimes.

In many areas, applied researchers as well as practitioners have to choose between different solutions for a problem at hand; this calls for optimal decision rules to settle the choices involved. As a key example, one may think of the search for optimal treatment regimes (OTRs) in clinical research, that specify which treatment alternative should be administered to each patient under study. Motivated by the fact that the concept of optimality of decision rules in general and treatment regimes in particular has received so far relatively little attention and discussion, we will present a number of reflections on it, starting from the basics of any optimization problem. Specifically, we will analyze the search space and the to be optimized criterion function underlying the search of single decision point OTRs, along with the many choice aspects that show up in their specification. Special attention is paid to formal characteristics and properties as well as to substantive concerns and hypotheses that may guide these choices. We illustrate with a few empirical examples taken from the literature. Finally, we discuss how the presented reflections may help sharpen statistical thinking about optimality of decision rules for treatment assignment and to facilitate the dialogue between the statistical consultant and the applied researcher in search of anOTR.

Bayesian decision making using partial data for fractured poroelastic media

Partial data is a common challenge in many applied problems, where we can measure only the part of the solution. This paper considers the poroelasticity problem in fractured media, where only partial measurements of fracture pressures are available. The poroelasticity models have applications in many engineering and scientific fields, such as geomechanics, hydrology, and reservoir simulation. This paper aims to develop a Bayesian decision making approach using partial data and consider its application to the poroelasticity problem in fractured media. The methodology of the proposed approach combines Proper Orthogonal Decomposition, Discrete Empirical Interpolation Method, Bayesian Inversion Approach, and Bayesian Decision Theory. It allows us to compute posterior probabilities and decide according to the optimal decision rule based on the partial data. Moreover, we apply partially explicit temporal discretization to solve forward problems efficiently. For testing the proposed approach, we consider a two-dimensional model problem. We simulate the cases when we need to make a single decision about the reservoir or decisions about the elements of the set of reservoirs. The results show that the proposed approach can efficiently compute the posterior probabilities and make a decision in the presence of only partial data for both cases.

Optimal adaptive decision rules in geotechnical construction considering uncertainty

To design a geotechnical engineering structure optimally, an iterative decision-making process is required due to the prevailing uncertainty of the ground conditions. At present, these decisions are taken based on simple deterministic rules and models. This paper proposes a risk-based decision-theoretic framework to the optimal planning for geotechnical construction. This framework combines geotechnical probabilistic models, cost analysis using Monte Carlo simulation and the observational method. The framework is illustrated on the design of the surcharge for an embankment on soft soil, whereby the optimal preloading sequence of added surcharge is adapted to the observed settlement. The approach balances the cost of surcharge material against financial penalties related to project delays and insufficient overconsolidation, which causes damage due to residual settlement. The result is a preloading strategy that optimally accounts for information obtained from settlement measurements under uncertain ground conditions. The findings highlight the potential of using risk-based decision planning in geotechnical engineering, in particular when combined with the observational method. For the investigated case study, a reduction in the expected cost in the order of 25% is observed.

Screening for experiments

I study a problem in which the principal is a decision maker and the agent is an “experimenter.” Neither the agent nor the principal observes the true state, but the agent can conduct an experiment that reveals information about the true state. The agent has private information about which experiments are feasible, his type. Before the agent conducts an experiment, the principal commits to a decision rule which is contingent on the experiments and their results. When the first-best outcome is unachievable, the principal faces a trade-off between the quality of the experiment and the ex post optimal decisions given experimental results. I characterize two kinds of optimal decision rules: one that sacrifices the ex post optimal decisions for the quality of the experiment, and the other that resolves the trade-off the other way around; which one is optimal depends on the properties of each type's set of feasible experiments.

Constructing time-invariant dynamic surveillance rules for optimal monitoring schedules.

Dynamic surveillance rules (DSRs) are sequential surveillance decision rules informing monitoring schedules in clinical practice, which can adapt over time according to a patient's evolving characteristics. In many clinical applications, it is desirable to identify and implement optimal time-invariant DSRs, where the parameters indexing the decision rules are shared across different decision points. We propose a new criterion for DSRs that accounts for benefit-cost tradeoff during the course of disease surveillance. We develop two methods to estimate the time-invariant DSRs optimizing the proposed criterion, and establish asymptotic properties for the estimated parameters of biomarkers indexing the DSRs. The first approach estimates the optimal decision rules for each individual at every stage via regression modeling, and then estimates the time-invariant DSRs via a classification procedure with the estimated time-varying decision rules as the response. The second approach proceeds by optimizing a relaxation of the empirical objective, where a surrogate function is utilized to facilitate computation. Extensive simulation studies are conducted to demonstrate the superior performances of the proposed methods. The methods are further applied to the Canary Prostate Active Surveillance Study (PASS).

Investigation on jamming detection in WSN using optimal decision rule

A jamming attack is a special case of a Denial of Service (DoS) attack that completely blocks the data transmission in Wireless Sensor Networks (WSNs). When sensor nodes are distributed in the field, numerous attacks, such as collision, black hole, selective forwarding, jamming, etc., caused by the presence of malicious nodes have the potential to cause network damage. Jamming is a highly risky attack that completely blocks data transmission within the wireless network. The existing technique for detecting jamming attacks are based on predetermined hopping-sequence, cryptographic, or random frequency hopping techniques. However, these mechanisms are more complex and frequently have energy constraints and high overhead. A novel jamming detection method based on a statistical approach that provides high network performance measures is proposed. It is a technique that uses energy-based clustering with a Received Signal Strength Indicator (RSSI). The selection of thresholds used for the detection of jamming is analyzed. The proposed approach employs three detection performance metrics for investigating the jamming attack, namely, Packet to Delivery Ratio (PDR), ENERGY, and RSSI. The jamming node is identified using the Optimal Decision Rule (ODR), which is determined by the hypothesis rule. If the hypothesis is not satisfied, then jamming exists; otherwise, there is no jamming. The novel technique is implemented using a Network Simulator, and various performance metrics such as PDR, Energy consumption, Network throughput, Routing overhead, network, and node lifetime are evaluated to conclude that the statistical approach outperforms the timestamp and IEWMA approaches.

Identifying mangroves through knowledge extracted from trained random forest models: An interpretable mangrove mapping approach (IMMA)

Black-box algorithms are among the dominant mangrove mapping approaches with complex decision-making procedures. Model internals and tacit knowledge were neglected, such as a large number of decision rules provided by random forest (RF) analyses. Explainable artificial intelligence (XAI) has emerged to emphasize the interpretability of an approach. However, current knowledge-based mangrove mapping approaches rely on extensive experiments. Thus, they cannot be easily updated to accommodate new issues, such as prevalent false positives resulting from insufficient consideration of the spectral mixture of vegetation and water in existing studies. To combine the advantages of black-box-based approaches with high update rates and knowledge-based approaches with high interpretability, this study developed a knowledge extraction method by parsing trained RF models, reconstructing decision rules to incorporate the ensemble procedure, and selecting the optimal decision rule as the target. Using this method, an interpretable mangrove mapping approach (IMMA) consisting of five features was constructed, which derived from Sentinel-2 image bands and a digital elevation model: B12 < 0.06 & B8/B2 > 3.50 & elevation < 4.70 & mangrove vegetation index (MVI) > 2.92 & normalized difference index4 (NDI) < 0.07. The study achieved an overall accuracy (OA) of 82.3% along the entire coast of China using test samples. Comparatively, it achieved an OA of 78.8% in south Florida, with no training samples for the RF models. The IMMA approach had a limited number of false positives compared with the black-box-based and knowledge-based approaches. By analyzing, we found B12 < 0.06 & B8/B2 > 3.50 & elevation < 4.70 dominated the IMMA to achieve comparable classification results to the existing studies, and B8/B2 > 3.50 was the key to suppressing the false positives resulting from the spectral mixture. The IMMA provided a bridge between training samples and interpretable decision rules, a tool to discover new knowledge, a key to improving fundamental scientific understanding of mangrove mapping, and an alternative to black-box algorithms in the XAI era expandable to various fields.

The Effect of Wages on Human Capital and its Virtuous Dynamic Cycle

This paper provides economic theory with a dynamic structural model that mathematically proves both a positive effect of wages on the workers’ human capital intertemporal investment decisions and the existence of a virtuous dynamic cycle between wages and human capital.Among the desired characteristics of the proposed model is that it has an analytical solution, permitting the achievement of an optimal decision rule for each choice variable and the calibration of its parameters using observed data; this favors an easy implementation by policymakers and researchers. The constructed statistics and the results of the empirical application of the model, to the Mexican developing economy, support the mathematical conclusions. The model predicts two benchmark human capital gross returns and their corresponding wages, by levels of education. Below the second one, human capital coming from formal education vanishes over time, due to investment not being enough to offset its depreciation; below the first one, workers stop investing in education. Unfortunately, around 40% and 20% of Mexican workers are located below them, respectively.