Standard Sensitivity Analysis Research Articles

Research outcomes and recommendations for the assessment of progression in cancer clinical trials from a PhRMA working group

PurposeProgression free survival (PFS) is increasingly used as a primary end-point in oncology clinical trials. This paper provides recommendations for optimal trial design, conduct and analysis in situations where PFS has the potential to be an acceptable end-point for regulatory approval. Patients and methodsThese recommendations are based on research performed by the Pharmaceutical Research and Manufacturers Association (PhRMA) sponsored PFS Working Group, including the re-analysis of 28 randomised Phase III trials from 12 companies/institutions. Results(1) In the assessment of PFS, there is a critical distinction between measurement error that results from random variation, which by itself tends to attenuate treatment effect, versus bias which increases the probability of a false negative or false positive finding. Investigator bias can be detected by auditing a random sample of patients by blinded, independent, central review (BICR). (2) ITT analyses generally resulted in smaller treatment effects (HRs closer to 1) than analyses that censor patients for potentially informative events (such as starting other anti-cancer therapy). (3) Interval censored analyses (ICA) are more robust to time-evaluation bias than the log-rank test. ConclusionA sample based BICR audit may be employed in open or partially blinded trials and should not be required in true double-blind trials. Patients should be followed until progression even if they have discontinued treatment to be consistent with the ITT principle. ICAs should be a standard sensitivity analysis to assess time-evaluation bias. Implementation of these recommendations would standardize and in many cases simplify phase III oncology clinical trials that use a PFS primary end-point.

Local Perturbation Analysis of Linear Programming with Functional Relation Among Parameters

In this paper, the authors study the sensitivity analysis for a class of linear programming (LP) problems with a functional relation among the objective function parameters or those of the right-hand side (RHS). The classical methods and standard sensitivity analysis software packages fail to function when a functional relation among the LP parameters prevail. In order to overcome this deficiency, the authors derive a series of sensitivity analysis formulae and devise corresponding algorithms for different groups of homogenous LP parameters. The validity of the derived formulae and devised algorithms is corroborated by open literature examples having linear as well as nonlinear functional relations between their vector b or vector c components.

Supervised Manifold Distance Segmentation

We present a simple and robust method for image and volume data segmentation based on manifold distance metrics. This is done by treating the image as a function that maps the 2D (image) or 3D (volume) to a 2D or 3D manifold in a higher dimensional feature space. We explore a range of possible feature spaces, including value, gradient, and probabilistic measures, and examine the consequences of including these measures in the feature space. The time and space computational complexity of our segmentation algorithm is O(N), which allows interactive, user-centric segmentation even for large data sets. We show that this method, given appropriate choice of feature vector, produces results both qualitatively and quantitatively similar to Level Sets, Random Walkers, and others. We validate the robustness of this segmentation scheme with comparisons to standard ground-truth models and sensitivity analysis of the algorithm.

Practical issues in handling data input and uncertainty in a budget impact analysis

The objective of this paper was to address the importance of dealing systematically and comprehensively with uncertainty in a budget impact analysis (BIA) in more detail. The handling of uncertainty in health economics was used as a point of reference for addressing the uncertainty in a BIA. This overview shows that standard methods of sensitivity analysis, which are used for standard data set in a health economic model (clinical probabilities, treatment patterns, resource utilisation and prices/tariffs), cannot always be used for the input data for the BIA model beyond the health economic data set for various reasons. Whereas in a health economic model, only limited data may come from a Delphi panel, a BIA model often relies on a majority of data taken from a Delphi panel. In addition, the dataset in a BIA model also includes forecasts (e.g. annual growth, uptakes curves, substitution effects, changes in prescription restrictions and guidelines, future distribution of the available treatment modalities, off-label use). As a consequence, the use of standard sensitivity analyses for BIA data set might be limited because of the lack of appropriate distributions as data sources are limited, or because of the need for forecasting. Therefore, scenario analyses might be more appropriate to capture the uncertainty in the BIA data set in the overall BIA model.

Standard sensitivity analysis and additive tolerance approach in MOLP

We consider sensitivity analysis of the objective function coefficients in multiple objective linear programming (MOLP). We focus on the properties of the parameters set for which a given extreme solution is efficient. Moreover, we compare two approaches: the standard sensitivity analysis (changing only one coefficient) and the additive tolerance approach (changing all coefficients). We find the connections between these two approaches by giving a theorem describing the upper bound on the maximal additive tolerance.

Sensitivity analysis of the combined travel demand model with applications

The conventional sequential four-step procedure of travel demand forecasting has been widely adopted by practitioners. However, it suffers from inconsistent consideration of travel times and congestion effects in various steps of the procedure. A combined travel demand model overcomes the problems associated with the sequential four-step procedure by integrating travel-destination-mode-route choice together. In this paper, a standard sensitivity analysis for non-linear programming is employed for conducting the sensitivity analysis of the combined travel demand model. Explicit expressions of the derivatives of model variables with respect to perturbations of input variables and parameters of the combined travel demand model are developed. These derivatives could be used to assess changes in solution variables and various system performance measures when the network characteristics are changed slightly. To gain insight into the usefulness of the sensitivity expressions, five applications, such as identification of critical parameters, paradox analysis, access control, destination choice, and error and uncertainty analysis, are presented with numerical results.

Application of artificial neural networks for prediction of natural convection from a heated horizontal cylinder

A generalized neural network analysis for natural convection heat transfer from a horizontal cylinder is developed in this paper. Cylinder diameter, cylinder surface temperature and ambient temperature are selected as the input parameters, while the Nusselt number as the output. A three-layer network is used for predicting the Nusselt number. The number of the neurons in the hidden layer was determined by a trial and error process together with cross-validation of the experimental data evaluating the performance of the network and standard sensitivity analysis. The trained network gives the best values over the correlations with less than 2.5% mean relative error. The experimental data of the average Nusselt number over the horizontal cylinders having different diameters of 4.8 mm–9.45 mm are from Atayılmaz and Teke [1]. The results from the trained network were compared with the proposed correlation for the average Nusselt number over the cylinder and it is shown that the results are in satisfactory agreement. The Nusselt numbers obtained from the experimental study were seen to be consistent by ± 20% with the well known correlations for natural convection heat transfer from horizontal cylinder developed by Morgan [2], Fand and Brucker [3], and Churchill and Chu [4]. Moreover it is seen that that results from the trained network show absolute agreement with the experimental data in ± 5% deviation band better than the correlations given by Morgan [2], Fand and Brucker [3], and Churchill and Chu [4].

Generalized neural network model of alternative refrigerant (R600a) inside a smooth tube

The correct prediction of refrigerant condensation heat transfer performance is important for design of condensers. A generalized neural network correlation for condensation heat transfer coefficient of alternative refrigerant R600a inside horizontal tube has been developed in this paper. Mass flow rate, vapor qualities, saturation temperature, difference value temperature are selected as the input parameters, while the Nusselt number and heat transfer coefficient as the output. Three-layer network is used for predicting the Nusselt number and the heat transfer coefficient. The number of the neurons in the hidden layer was determined by a trial and error process together with cross-validation of the experimental data evaluating the performance of the network and standard sensitivity analysis. The trained network gives the best values over the correlations with less than 4% mean relative error. The experimental data of the heat transfer coefficients of R600a, a hydrocarbon refrigerant, in a horizontal smooth copper tube with an inner diameter of 4 mm and outer diameter of 6 mm are from Agra et al. [O. Agra, “Condensation of refrigerants in a horizontal tube in annular flow regime”, PhD thesis Yildiz Technical University, 2007]. The condensing heat transfer coefficients obtained from the experimental study were seen to be consistent by ± 20% with the correlations developed by Shah [M.M. Shah, A general correlation for heat transfer during film condensation inside pipes, Int. J. Heat Mass Transfer 22 (1979) 547–556], Travis [D.P. Traviss, W.M. Rohsenow, A.B. Baron, Forced convection condensation inside tubes: a heat transfer equation for condenser design, ASHRAE Trans. 79 (1972) 157–165] and Cavallini-Zecchin [A. Cavallini, R. Zecchin, A dimensionless correlation for heat transfer in forced convection condensation, Proceedings of the Fifth International Heat Transfer Conference, vol. 3, 1974, pp. 309–313]. And it is seen that results from the trained network shows good agreement with the experimental data and better results than the correlations given by Shah, Cavallini and Travis.

Methodological Issues in Analyzing Time Trends in Biologic Fertility: Protection Bias

One method of assessing biologic fertility is to measure time to pregnancy (TTP). Accidental pregnancies do not generate a valid TTP value and lead to nonrandom missing data if couples experiencing accidental pregnancies are more fertile than the general population. If factors affecting the rate of accidental pregnancies, such as availability of effective contraception and induced abortion, vary over time, then the result may be protection bias in the estimates of fertility time trends. Six European data sets were analyzed to investigate whether evidence of protection bias exists in TTP studies of fertility trends in Europe over the past 50 years. Couples experiencing accidental pregnancies tended to be more fertile than the general population. However, trends in accidental pregnancy rates were inconsistent across countries and were insufficient to produce substantial bias in fertility trends in simulated data. Where protection bias is suspected, the authors demonstrate use of 2 multiple imputation methods to generate realizations for the missing TTP values for accidental pregnancies. Simulation studies show that both methods successfully reduce or eliminate protection bias. The authors also demonstrate that standard sensitivity analyses for dealing with accidental pregnancies provide an upper bound on the extent of any bias.

Applying a One-Dimensional Mass Transport Model Using Groundwater Concentration Data

One-dimensional analytical mass transport models are familiar to environmental professionals because they are typically used as learning devices in undergraduate groundwater courses. The application of the models requires relatively certain knowledge of contaminant release to the saturated zone. However, release data are typically not reliably known at sites with uncontrolled contaminant releases. A mass balance approach has been developed to calculate contaminant release parameters based on site-specific groundwater concentration data. Standard numerical calibration and sensitivity analysis techniques were modified for use with the one-dimensional spreadsheet model. A groundwater concentration dataset from a Superfund site was used to evaluate three schemes for calculating the model initial concentration. The site application demonstrates how the spreadsheet model could be used for preliminary remediation system comparisons including restoration time estimating. The use of the spreadsheet model may reduce the effort associated with subsequent numerical modeling typically required for remedial design. The spreadsheet application highlights the importance of collecting physical data with groundwater concentration data.

Validity Issues Relating to Time-to-Pregnancy Studies of Fertility

Circumstantial evidence regarding the deterioration of male fecundity, together with the fact that the majority of infertility cases have unknown causes, make it important to identify risk factors that can be eliminated and thus pave the way for prevention of infertility. Rapidly increasing costs of artificial fertilization and cumbersome lengthy treatments of infertile couples with limited success rates add to the importance of such studies and contribute to a wider acceptance of epidemiologic studies of fertility. Worldwide fertility is now close to replacement level, and the previous fear of uncon trolled population growth should no longer stand in our way for studying environmental causes that impair fecundity. More than 20 years ago, a seminal paper1 on subfecundity and infertility examined occupational risk factors in the Danish workplace. During the late 1980s, the use of time-to-pregnancy (TTP) was introduced and elaborated. TTP is the number of months or menstrual cycles it takes a couple to conceive and covers the entire distribution of waiting times from 0 (high-level fecundity) to several years (low-level fecundity).2'3 Valid information about TTP is easy and inexpensive to obtain by questionnaires and interview. With increasing interest in reproductive issues in public health, TTP soon became a widely used measure in epidemiologic studies. As time went on, it became clear that TTP studies needed to take several population characteristics into consideration, especially if the compared populations differed in such factors as desired family size, use of contraceptive methods, and sexual activity. Many specific pitfalls inherent in the TTP methodology were identified,4'5 and new types of biases have been added to the list in recent years.6 Lately, we have witnessed discussions on costly prospective designs versus inexpensive retro spective designs.7 Also, we have seen differing opinions as to whether it is possible to obtain valid retrospective information on secular trends in fecundity by the TTP meth odology8'9 or on fecundity among different cultures.' 0 " Is the TTP design option still to be considered a valid method for obtaining knowledge about fecundity in humans? To obtain a direct measure of human fecundity, couples must attempt to use their reproductive capability. This is where the troubles start. First, not all persons get to try, even if they want children (for example, if they never find a partner). Second, some couples get pregnant while using contraception or by accident. Such pregnancies have no TTP value but provide important information. These couples may be more fecund than others, but how much more we do not know. As a result, those who provide a measurable TTP may underrepresent highly fecund couples. Third, not all couples succeed in conceiving, and TTP distributions that include only those who succeed will underrepresent less fertile couples, thus indicating the need for collecting data on unprotected intercourse

Consistency of the Results of Different MCA Methods: A Critical Review

Multiple criteria decision analysis (MCA) has become an indispensable tool for dealing with complex and unstructured decision problems in environmental and natural resource management which involve a number of conflicting objectives and a variety of stakeholders. Despite their popularity, choosing which of the many multicriteria methods to use is tricky. Different methods may yield different results and therefore the decision may depend on the method selected. In this paper we review a number of experiments conducted to compare the results of different MCA methods when applied to the same decision problem. I compare and critically examine the hypotheses postulated and the results obtained from these experiments. Despite the equivocal results yielded in some experiments, many authors recommend applying two or more MCA methods, especially in the case of unstructured decision problems. This multimethod MCA application may be regarded as a type of validation which is more extensive than standard sensitivity analysis and which enables the decisionmaker to review the preferences and judgments previously elicited by a single method.

Generalized neural network correlation for flow boiling heat transfer of R22 and its alternative refrigerants inside horizontal smooth tubes

The correct prediction of refrigerant boiling heat transfer performance is important for the design of evaporators. A generalized neural network correlation for boiling heat transfer coefficient of R22 and its alternative refrigerants R134a, R407C and R410A inside horizontal smooth tubes has been developed in this paper. Four kinds of dimensionless parameter groups from existing generalized correlations are selected as the input of neural network, while the Nusselt number is used as the output. Three-layer perceptron is employed as the universal approximator to build the relationship between the input and output parameters. The neuron number of hidden layer is determined by the performance of model accuracy and the standard sensitivity analysis. The experimental data of the four refrigerants in open literatures are used for correlation. The results show that the input parameter group based on the Gungor–Winterton correlation is better than the other three groups. Compared with the experimental data, the average, mean and root-mean-square deviations of the trained neural network are 2.5%, 13.0% and 20.3%, respectively, and approximately 74% of the deviations are within ±20%, which is much better than that of the existing generalized correlations.

Single-period location models for subsidized housing: Project-based subsidies

Large urban public housing authorities (PHAs) in the US have abandoned the traditional model of high-rise public housing developments and increasingly use lower-density project-based subsidized housing as well as tenant-based housing subsidies to provide affordable shelter. PHAs generally apply practitioner expertise to locate subsidized housing and do not identify housing alternatives using prescriptive planning models that optimize policy criteria and incorporate well-defined stakeholder impacts. Using policy characteristics and estimated economic impacts of subsidized housing, a new single-period multi-objective integer programming model for location of project-based subsidized housing is developed. Data requirements, relationships with standard planning models, solution characteristics and sensitivity analyses are examined. It is shown that this model gives planners considerable flexibility in choosing alternative housing configurations that can satisfy the needs of various interest groups.

Bounding Causal Effects Under Uncontrolled Confounding Using Counterfactuals

Common sensitivity analysis methods for unmeasured confounders provide a corrected point estimate of causal effect for each specified set of unknown parameter values. This article reviews alternative methods for generating deterministic nonparametric bounds on the magnitude of the causal effect using linear programming methods and potential outcomes models. The bounds are generated using only the observed table. We then demonstrate how these bound widths may be reduced through assumptions regarding the potential outcomes under various exposure regimens. We illustrate this linear programming approach using data from the Cooperative Cardiovascular Project. These bounds on causal effect under uncontrolled confounding complement standard sensitivity analyses by providing a range within which the causal effect must lie given the validity of the assumptions.

Generalisability in economic evaluation studies in healthcare: a review and case studies

To review, and to develop further, the methods used to assess and to increase the generalisability of economic evaluation studies. Electronic databases. Methodological studies relating to economic evaluation in healthcare were searched. This included electronic searches of a range of databases, including PREMEDLINE, MEDLINE, EMBASE and EconLit, and manual searches of key journals. The case studies of a decision analytic model involved highlighting specific features of previously published economic studies related to generalisability and location-related variability. The case-study involving the secondary analysis of cost-effectiveness analyses was based on the secondary analysis of three economic studies using data from randomised trials. The factor most frequently cited as generating variability in economic results between locations was the unit costs associated with particular resources. In the context of studies based on the analysis of patient-level data, regression analysis has been advocated as a means of looking at variability in economic results across locations. These methods have generally accepted that some components of resource use and outcomes are exchangeable across locations. Recent studies have also explored, in cost-effectiveness analysis, the use of tests of heterogeneity similar to those used in clinical evaluation in trials. The decision analytic model has been the main means by which cost-effectiveness has been adapted from trial to non-trial locations. Most models have focused on changes to the cost side of the analysis, but it is clear that the effectiveness side may also need to be adapted between locations. There have been weaknesses in some aspects of the reporting in applied cost-effectiveness studies. These may limit decision-makers' ability to judge the relevance of a study to their specific situations. The case study demonstrated the potential value of multilevel modelling (MLM). Where clustering exists by location (e.g. centre or country), MLM can facilitate correct estimates of the uncertainty in cost-effectiveness results, and also a means of estimating location-specific cost-effectiveness. The review of applied economic studies based on decision analytic models showed that few studies were explicit about their target decision-maker(s)/jurisdictions. The studies in the review generally made more effort to ensure that their cost inputs were specific to their target jurisdiction than their effectiveness parameters. Standard sensitivity analysis was the main way of dealing with uncertainty in the models, although few studies looked explicitly at variability between locations. The modelling case study illustrated how effectiveness and cost data can be made location-specific. In particular, on the effectiveness side, the example showed the separation of location-specific baseline events and pooled estimates of relative treatment effect, where the latter are assumed exchangeable across locations. A large number of factors are mentioned in the literature that might be expected to generate variation in the cost-effectiveness of healthcare interventions across locations. Several papers have demonstrated differences in the volume and cost of resource use between locations, but few studies have looked at variability in outcomes. In applied trial-based cost-effectiveness studies, few studies provide sufficient evidence for decision-makers to establish the relevance or to adjust the results of the study to their location of interest. Very few studies utilised statistical methods formally to assess the variability in results between locations. In applied economic studies based on decision models, most studies either stated their target decision-maker/jurisdiction or provided sufficient information from which this could be inferred. There was a greater tendency to ensure that cost inputs were specific to the target jurisdiction than clinical parameters. Methods to assess generalisability and variability in economic evaluation studies have been discussed extensively in the literature relating to both trial-based and modelling studies. Regression-based methods are likely to offer a systematic approach to quantifying variability in patient-level data. In particular, MLM has the potential to facilitate estimates of cost-effectiveness, which both reflect the variation in costs and outcomes between locations and also enable the consistency of cost-effectiveness estimates between locations to be assessed directly. Decision analytic models will retain an important role in adapting the results of cost-effectiveness studies between locations. Recommendations for further research include: the development of methods of evidence synthesis which model the exchangeability of data across locations and allow for the additional uncertainty in this process; assessment of alternative approaches to specifying multilevel models to the analysis of cost-effectiveness data alongside multilocation randomised trials; identification of a range of appropriate covariates relating to locations (e.g. hospitals) in multilevel models; and further assessment of the role of econometric methods (e.g. selection models) for cost-effectiveness analysis alongside observational datasets, and to increase the generalisability of randomised trials.

Modeling the cost-effectiveness of treatment options for patients with clinical stage I non seminomatous germ cell tumors (NGSCT)

4525 Background: Survival rates for good risk clinical stage I NGSCT exceed 98%. Although long term outcomes for surveillance (SURV) and primary retroperitoneal lymph node dissection (RPLND) may be equivalent, their relative cost-effectiveness is unknown. Approximately 30% on SURV will relapse and will be treated with chemotherapy (CHEMO) +/- RPLND. We present a pilot study to model the cost effectiveness of SURV vs RPLND. Methods: Patients in 4 health states (>1 year post treatment) were evaluated: a) SURV b) RPLND c) CHEMO or d) RPLND and CHEMO. Utilities were collected using the EuroQol, Time Trade Off (TTO), Standard Gamble (SG) and Visual Analog Scale (VAS). Direct medical costs (DMC) were determined using the 2002 Medicare Fee Schedule. Costas and benefits were discounted 3% annually. Health care resource use was determined using National Comprehensive Cancer Network guidelines. Paients were assumed to have only one relapse and late relapses (more than 3 years) were considered equal in both groups. Quality Adjusted Life Years (QALY) =utility*life years. Cost effective ratio (C/E)=DMC/QALY. Decision analysis was modeled using a Markov process for a 25 year old man and tested using standard and probabilistic sensitivity analysis. Results: Mean utilities are listed in the table below. Total costs for RPLND:$48,372, SURV:$62,598(primarily due to increased diagnostic tests during SURV period). QALY were 23.33 and 24.77 respectively. The incremental C/E for surveillance was $9,870 per QALY, suggesting that SURV is cost effective. Conclusion: Though limited by the small sample size and the disproportionately few patients on SURV alone, this pilot study shows that this is a feasible model for determining cost-effectiveness. This will be used as a model for a future, larger study to further clarify this issue and guide clinicians in making treatment recommendations. No significant financial relationships to disclose.

Modeling the cost-effectiveness of treatment options for patients with clinical stage I non seminomatous germ cell tumors (NGSCT)

4525 Background: Survival rates for good risk clinical stage I NGSCT exceed 98%. Although long term outcomes for surveillance (SURV) and primary retroperitoneal lymph node dissection (RPLND) may be equivalent, their relative cost-effectiveness is unknown. Approximately 30% on SURV will relapse and will be treated with chemotherapy (CHEMO) +/- RPLND. We present a pilot study to model the cost effectiveness of SURV vs RPLND. Methods: Patients in 4 health states (>1 year post treatment) were evaluated: a) SURV b) RPLND c) CHEMO or d) RPLND and CHEMO. Utilities were collected using the EuroQol, Time Trade Off (TTO), Standard Gamble (SG) and Visual Analog Scale (VAS). Direct medical costs (DMC) were determined using the 2002 Medicare Fee Schedule. Costas and benefits were discounted 3% annually. Health care resource use was determined using National Comprehensive Cancer Network guidelines. Paients were assumed to have only one relapse and late relapses (more than 3 years) were considered equal in both groups. Quality Adjusted Life Years (QALY) =utility*life years. Cost effective ratio (C/E)=DMC/QALY. Decision analysis was modeled using a Markov process for a 25 year old man and tested using standard and probabilistic sensitivity analysis. Results: Mean utilities are listed in the table below. Total costs for RPLND:$48,372, SURV:$62,598(primarily due to increased diagnostic tests during SURV period). QALY were 23.33 and 24.77 respectively. The incremental C/E for surveillance was $9,870 per QALY, suggesting that SURV is cost effective. Conclusion: Though limited by the small sample size and the disproportionately few patients on SURV alone, this pilot study shows that this is a feasible model for determining cost-effectiveness. This will be used as a model for a future, larger study to further clarify this issue and guide clinicians in making treatment recommendations. No significant financial relationships to disclose.

Statistical sensitivity analysis of packed column reactors for contaminated wastewater

AbstractIn this article we consider the statistical sensitivity analysis of heavy metal biosorption in contaminated wastewater packed column reactors. In particular, the model describes the biosorption phenomenon using the Advection Dispersion Reaction equation under rapid local equilibrium. This allows computer simulation with random input parameters chosen from appropriate probability distributions. In order to have a statistical framework for analyzing the simulated data and assessing input importance, we introduce heteroskedastic and multivariate sensitivity analysis, which extends standard sensitivity analysis. Copyright © 2003 John Wiley & Sons, Ltd.

USING PVA FOR MANAGEMENT DESPITE UNCERTAINTY: EFFECTS OF HABITAT, HATCHERIES, AND HARVEST ON SALMON

Population viability analysis (PVA) is frequently used to quantify the level of risk associated with various populations and also to examine the relative benefits of alternative management actions. Typically, model predictions are extremely sensitive to estimated parameters, resulting in risk estimates that are very imprecise. Therefore, careful consideration of uncertainty is warranted when reporting PVA predictions. Here, we survey some simple and general simulation-based methods for assessing the uncertainty in model-based forecasts. We then illustrate how these methods can be used for incorporating parameter uncertainty when reporting estimates of extinction risk, using a PVA model developed from data on Pacific Chinook salmon stocks. Several researchers have suggested that relative assessments between management options will be more stable than estimates of extinction risk and population growth rate. We illustrate how the uncertainty in relative assessments can be quantified in a rigorous way. Finally, we show how Sobol' indices can be used to ask: “Where is the uncertainty coming from?” These indices are an ANOVA-like decomposition of the effects of parameters on model output that take into account both direct effects (as in standard sensitivity analysis) and effects due to interactions with other parameters. Throughout, we illustrate these methods using models relevant to the life cycle of Pacific salmon, with the intent of assessing the relative influences of habitat, harvest, and hatcheries on the viability of salmon populations. However, the same approaches are applicable to most situations where a model with estimated parameters is being used for forecasting or comparing management options. Corresponding Editor (ad hoc): J. S. Clark.