Measures Of Clinical Significance Research Articles

The Health-Economic Impact of a Prospectively Validated Severity of Illness Predictor for Adult Critical Care

Background: The ViSIG system, a clinical decision support system for adult intensive care units, was previously shown to be associated with significant reductions in length of stay, readmissions, and duration of mechanical ventilation. However, the health-economic benefits of ViSIG have yet to be fully understood. Infections and acute kidney injury are two measures of clinical significance with key financial implications, and their reduction could further strengthen the published association of ViSIG use with improved outcomes. Further, it is important to demonstrate the economic benefit of ViSIG for its widespread adoption by ICUs. Objective: To investigate the potential health-economic benefits associated with using a previously validated clinical decision support system, ViSIG. Study Design and Methods: In this study, we analyzed data from a previous analysis of ViSIG. The study cohort consisted of six ICUs at two hospitals, with a total of 2,256 admissions in the 'Blinded' Phase and 1,890 admissions in the 'Visible' Phase. We compared the frequency of new infections and Acute Kidney Injury (AKI) between the two phases and used multivariable mixed models to adjust for each patient’s severity of illness. To estimate the economic benefit of ViSIG, we calculated the mean total charges per patient for each phase and the change in patient throughput to arrive at a net gain/loss annual total charges per bed. Results: In the Visible Phase, infections declined by 40.5% (p<0.001). When adjusted for patient severity of illness, the Visible Phase had an odds ratio of 0.52 (95% confidence interval = 0.43, 0.62). There was a 24.5% relative decrease in the occurrence of AKI (p<0.001) and a 35% adjusted decrease from Blinded to Visible Phase (p<0.001). The mean total charges were $59,886 and $55,821 for the Blinded and Visible Phases, respectively. However, due to a reduced ICULOS in the Visible Phase, there was a 14.2% increase in admissions during the 90-day period. This resulted in an estimated mean increased revenue of $3,872 per patient at ICUs using ViSIG. Conclusion: ViSIG was associated with a reduction in two major clinical events: new infections and progression to AKI. This lessened clinical burden may be associated with reduced ICULOS, which resulted in increased patient throughput and, thus, increased revenues. Future studies need to measure specific clinician actions to characterize a cause-and-effect relationship.

One (effect) size does not fit at all: Interpreting clinical significance and effect sizes in depression treatment trials.

The efficacy of antidepressants in major depressive disorder has been continually questioned, mainly on the basis of studies using the sum-score of the Hamilton Depression Rating Scale as a primary outcome parameter. On this measure antidepressants show a standardised mean difference of around 0.3, which some authors suggested is below the cut-off for clinical significance. Prompted by a recent review that, using this argument, concluded antidepressants should not be used for adults with major depressive disorder, we (a) review the evidence in support of the cut-off for clinical significance espoused in that article (a Hamilton Depression Rating Scale standardised mean difference of 0.875); (b) discuss the limitations of average Hamilton Depression Rating Scale sum-score differences between groups as measure of clinical significance; (c) explore alternative measures of clinical importance; and (d) suggest future directions to help overcome disagreements on how to define clinical significance. We conclude that (a) the proposed Hamilton Depression Rating Scale cut-off of 0.875 has no scientific basis and is likely misleading; (b) there is no agreed upon way of delineating clinically significant from clinically insignificant; (c) evidence suggests the Hamilton Depression Rating Scale sum-score underestimates antidepressant efficacy; and (d) future clinical trials should consider including measures directly reflective of functioning and wellbeing, in addition to measures focused on depression psychopathology.

Functional significance of a novel 7-factor model of DSM-5 PTSD symptoms: Results from the National Health and Resilience in Veterans Study

BackgroundWhile posttraumatic stress disorder (PTSD) symptoms in the recently published Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) are clustered into four factors, emerging confirmatory factor analytic studies suggest that this disorder is best characterized by seven symptom clusters, including re-experiencing, avoidance, negative affect, anhedonia, externalizing behaviors, and anxious and dysphoric arousal symptoms. To date, however, data are lacking regarding the relation between this novel model of DSM-5 PTSD symptoms and measures of clinical significance in this population (e.g., functioning). MethodsUsing data from the National Health and Resilience in Veterans Study (NHRVS), a contemporary, nationally representative sample of 1484 U.S. veterans, we evaluated clinical and functional correlates of a novel 7-factor model of DSM-5 PTSD symptoms. ResultsDifferential patterns of associations were observed between DSM-5 PTSD symptom clusters, and psychiatric comorbidities, suicidal ideation, hostility, and functioning and quality of life. Anhedonia symptoms, in particular, were strongly related to current depression, as well as reduced mental functioning and quality of life. Externalizing behaviors were most strongly related to hostility, supporting the convergent validity of this construct. LimitationsCross-sectional design and employment of self-report measures. ConclusionsThese results suggest that a more refined 7-factor model of DSM-5 PTSD symptoms may provide greater specificity in understanding associations with comorbid psychopathology, suicidal ideation, and functioning and quality of life in U.S. veterans. They further suggest that prevention and treatment efforts that target distinct aspects of the PTSD phenotype may be more effective in mitigating key clinical and functional outcomes in this population.

Clinical Significance as It Relates to Evidence‐Based Practice

This paper analyzes the concept of clinical significance (CS) in relation to evidence-based practice (EBP). The purpose is to show that CS terminology is inconsistent in the nursing literature. It is argued that nursing outcomes and interventions that include findings of CS are difficult to interpret due to lack of an operational definition. It is further argued that the absence of a consistent operational definition is incompatible with EBPs which require standardization of terminology. The current literature and research studies, particularly from the electronic databases Cumulative Index to Nursing and Allied Health Literature, ISI Web of Knowledge, PubMed, and Cochrane Database of Systematic Reviews. The disparate uses of CS negatively impact standardizing and quantifying research outcomes to discern EBPs. The authors propose a definition of CS inclusive of the multifarious uses that were revealed in the literature, and conclude that there is a need for professional nursing consensus to define the term. A standard operational definition of CS would enable consistency as clinicians interpret research findings and facilitate translating research to practice. Given the centrality of CS to interpreting research findings and applying them to practice, there is a need to solidify the terminology of and measurements for CS in nursing. National nursing agencies, including The National Institute for Nursing Research and Sigma Theta Tau International, should make standardizing CS a high priority for targeted funding. One method of doing so would be to support a consensus convergence to review and select the optimal measures of CS for nursing research. Research to increase knowledge about what constitutes measurement and change, or CS from the patient perspective is needed. Editors and peer reviewers should encourage authors to include a discussion of CS. Discussions of CS should receive greater emphasis in research journals. It is hoped that the preliminary findings from the concept analysis presented in this article will facilitate the work of such a consensus forum. The most immediate and tangible advantages to a common conceptual definition and meaning of CS terminologies by nursing, regardless of which definition is selected, are less confusion and more clarity. Ultimately, the most enduring benefit of a common conceptual definition and measurement for CS is the bridge it provides between research and practice, and the facility with which it promotes the integration of research into EBP.

Ten Take Home Lessons from the First 10 Years of the CTN and 10 Recommendations for the Future

Background: The first 10 years of the National Institute on Drug Abuse’s Clinical Trials Network (CTN) yielded a wealth of data on the effectiveness of a number of behavioral, pharmacological, and combined approaches in community-based settings. Methods: We summarize some of the methodological contributions and lessons learned from the behavioral trials conducted during its first ten years, including the capacity and enormous potential of this national research infrastructure.Results: The CTN made contributions to the methodology of effectiveness research; new insights from secondary analyses; the extent to which approaches with strong evidence bases, such as contingency management, extend their effectiveness to real world clinical settings; new data on ‘standard treatment’ as actually practiced in community programs, the extent to which retention remains a major issue in the field; important data on the safety of specific behavioral therapies for addiction; and heightened the importance of continued sustained attention to bridging the gap between treatment and research. Conclusions: Areas of focus for the CTN’s future include defining common outcome measures to be used in treatment outcome studies for illicit drugs; incorporating performance indicators and measures of clinical significance; conducting comparative outcome studies; contributing to the understanding of effective treatments of comorbidity; reaching underserved populations; building implementation science; understanding long-term outcomes of current treatments and sustaining treatment effects; and conducting future trials more efficiently.

What does "work" mean? Reopening the debate about clinical significance.

[Clin Psychol Sci Prac 17: 48–51, 2010] Researchers and practitioners have long debated the meaning and measurement of clinical significance. Shearer-Underhill and Marker (2010) offer a valuable contribution to this discussion by drawing the psychotherapy research community’s attention to an additional statistical method for measuring clinical significance—the number needed to treat. After a decrease in publication rates in the last 5 years on methods for measuring clinical significance, the article by Shearer-Underhill and Marker represents a renewed interest in the construct of clinical significance of treatment outcome results. This commentary discusses the importance of the article by elaborating on the theoretical and methodological issues that cut across measures of clinical significance. Strengths and weaknesses of specific statistical methods are reviewed and a call is made for continued pursuit of conceptual clarity and methodological rigor for measures of clinical significance.

The Inventory of Interpersonal Problems (IIP): Sensitivity to change

Although the Inventory of Interpersonal Problems (IIP) is a standard instrument in both, psychotherapy research and quality control of psychotherapy, it's sensitivity to change has yet to be firmly established. The authors investigated the sensitivity to change of the IIP using a study design based on the assumption that psychoanalytic psychotherapy leads to significant improvement in interpersonal problems. To assess whether this type of change can be measured with the IIP, they did a pre/post comparison of IIP scores in 63 patients with depression. The authors evaluated therapeutic change by comparing means (t-test) and also by calculating standardized effect size (ES), the reliable change index and a measure of clinical significance (internal sensitivity to change). They also compared the IIP results with those of construct-near tests (external sensitivity to change). The IIP showed a satisfactory sensitivity to change, with highly significant differences on the t-test on all scales, an ES of 1.44 for the IIP total score and a large or medium ES (above 0.80 and 0.50, respectively) on most individual scales. Hence the IIP appears to be an appropriate instrument for assessing the effectiveness of psychotherapy.

Prognostic indices with brief and standard CBT for panic disorder: II. Moderators of outcome

Despite the growth of reduced therapist-contact cognitive behavioural therapy (CBT) programmes, there have been few systematic attempts to determine prescriptive indicators for such programmes vis-à-vis more standard forms of CBT delivery. The present study aimed to address this in relation to brief (6-week) and standard (12-week) therapist-directed CBT for panic disorder (PD) with and without agoraphobia. Higher baseline levels of severity and associated disability/co-morbidity were hypothesized to moderate treatment effects, in favour of the 12-week programme. Analyses were based on outcome data from two out of three treatment groups (n=72) from a recent trial of three forms of CBT delivery for PD. The dependent variables were a continuous composite panic/anxiety score and a measure of clinical significance. Treatment x predictor interactions were examined using multiple and logistic regression analyses. As hypothesized, higher baseline severity, disability or co-morbidity as indexed by strength of dysfunctional agoraphobic cognitions; duration of current episode of PD; self-ratings of panic severity; and the 36-item Short Form Health Survey (SF-36) (Mental component) score were all found to predict poorer outcome with brief CBT. A similar trend was apparent in relation to baseline level of depression. With high and low end-state functioning as the outcome measure, however, only the treatment x agoraphobic cognitions interaction was found to be significant. While there was no evidence that the above variables necessarily contraindicate the use of brief CBT, they were nevertheless associated with greater overall levels of post-treatment improvement with the 12-week approach.

Measurement of the effects of abstinence from tobacco: A qualitative review.

This article reviews terms, methodologies, and scales used to describe the effects of abstinence from tobacco in humans. The review suggests operational definitions for withdrawal, offset, novel, and indefinite abstinence effects. The review recommends the use of prospective studies that use smokers trying to quit, multiple data points pre-abstinence and very soon after abstinence, comparison groups, at least 16 hrs of abstinence, objective measures, measures of clinical significance, and data analytic techniques that allow use of all participants' data. The 8 most commonly used tobacco withdrawal measures differ in coverage, brevity, and use of single vs. multiple items to describe each symptom. Of the measures, 6 consistently increase with abstinence, but no single measure appears superior.

Assessing clinical significance: Does it matter which method we use?

Measures of clinical significance are frequently used to evaluate client change during therapy. Several alternatives to the original method devised by N. S. Jacobson, W. C. Follette, & D. Revenstorf (1984) have been proposed, each purporting to increase accuracy. However, researchers have had little systematic guidance in choosing among alternatives. In this simulation study, the authors systematically explored data parameters (e.g., reliability of measurement, pre-post effect size, and pre-post correlation) that might yield differing results among the most widely considered clinical significance methods. Results indicated that classification across methods was far more similar than different, especially at greater levels of reliability. As such, the existing methods of clinical significance appear highly comparable; future directions for clinical significance use and research are discussed.

Measures of Clinical Significance

Behavioral scientists are interested in answering three basic questions when examining the relationships between variables (Kirk, 2001). First, is an observed result real or should it be attributed to chance (i.e., statistical significance)? Second, if the result is real, how large is it (i.e., effect size)? Third, is the result large enough to be meaningful and useful (i.e., clinical or practical significance)? In this last column in the series, we treat clinical significance as equivalent to practical significance. Judgments by the researcher and the consumers (e.g., clinicians and patients) regarding clinical significance consider factors such as clinical benefit, cost, and side effects. Although there is no formal statistical test of clinical significance, researchers suggest using one of three types of effect size measures to assist in interpreting clinical significance. These include the strength of association between variables (r family effect size measures), the magnitude of the difference between treatment and comparison groups (d family effect size measures), and measures of risk potency. In this paper, we review the d and r effect size measures and five measures of risk potency: odds ratio, risk ratio, relative risk reduction, risk difference, and number needed to treat. Finally, we review a relatively new effect size, AUC (which for historical reasons irrelevant to the current discussion stands for area under the receiver operating characteristic [ROC] curve), that integrates many of the others and is directly related to clinical significance. Each of these measures, however, has limitations that require the clinician to be cautious about interpretation. Guidelines are offered to facilitate the interpretation and understanding of clinical significance. Problems With Statistical Significance

Evidence-based treatment of stuttering:: II. Clinical significance of behavioral stuttering treatments

An evidence-based framework can be described as an empirically-driven, measurement-based, client-sensitive approach for selecting treatments. It is believed that using such a framework is more likely to result in a clinically significant outcome. For this paper, a clinically significant outcome was defined as a meaningful treatment change. It was suggested that there are at least three groups for whom a treatment’s outcome is meaningful. These groups include clinicians/clinical researchers, the clients, and relevant others who have some interest in the outcome (e.g., parents of a child who stutters). The meaning and measurement of clinical significance was discussed for each of these three groups, based on research from the behavioral stuttering treatment literature. Educational objectives: The reader will learn about and be able to (1) broadly define a clinically significant outcome and identify some of the groups who are interested in such an outcome and (2) describe how clinical significance has been evaluated in stuttering treatment within an evidence-based framework.

Effects of Nefazodone on Body Weight

Evidence suggests that the newer antidepressant drugs may differ with respect to their effects on body weight, especially during long-term treatment. However, the published data about treatment-emergent weight change with the newer antidepressants are limited. Most reports of unexpected selective serotonin reuptake inhibitor (SSRI)-associated weight gain are anecdotal or from small controlled trials. To determine if differences exist among the newer antidepressants, the authors retrospectively analyzed data from clinical trials comparing nefazodone with SSRIs and with imipramine. Weight change data supplied by Bristol-Myers Squibb from 6 completed clinical trials comparing the antidepressant nefazodone (N = 523) with 3 SSRIs, fluoxetine, sertraline, and paroxetine (N = 513), as well as 3 trials comparing nefazodone (N = 225) with the tricyclic antidepressant imipramine (N = 224) were analyzed. In all studies, nefazodone was found to be equal in efficacy to the comparator antidepressants. Studies that included both acute and long-term treatment phases were included in the analysis. Acute phases of the trials lasted either 6 or 8 weeks, and long-term phases varied in duration from 16 to 46 weeks. The analysis included summarizing the number and percentage of patients in each group with a > or = 7% change in body weight from baseline at any point in the long-term and acute phases, at endpoint, and at week 16 of the long-term phases. Using 7% or greater weight change as the measure of clinical significance, 4.3% of SSRI-treated patients had lost weight at any point in the acute phase versus 1.7% of those treated with nefazodone (p = .017). However, at any point during the long-term phase, significantly more SSRI-treated patients than nefazodone-treated patients showed a significant increase in body weight (17.9% vs. 8.3%; p = .003). At any point in the acute phase, significantly more imipramine-treated patients than nefazodone-treated patients had a 7% or greater increase in body weight (4.9% vs. 0.9%; p = .027), and for the long-term phase the comparison yielded 24.5% versus 9.5%. The difference during the long-term phase was statistically significant in women (p = .017), but not in men (p = .078) due to the small numbers of men in each group. SSRIs caused more weight loss during short-term treatment but more weight gain during long-term treatment. These results lend support to the observation that some antidepressants have a greater expected risk of weight gain than others during long-term therapy.

The meanings and measurement of clinical significance.

The previous articles in this special section make the case for the importance of evaluating the clinical significance of therapeutic change, present key measures and innovative ways in which they are applied, and more generally provide important guidelines for evaluating therapeutic change. Fundamental issues raised by the concept of clinical significance and the methods discussed in the previous articles serve as the basis of the present comments. Salient among these issues are ambiguities regarding the meaning of current measures of clinical significance, the importance of relating assessment of clinical significance to the goals of therapy, and evaluation of the construct(s) that clinical significance reflects. Research directions that are discussed include developing a typology of therapy goals, evaluating cutoff scores and thresholds for clinical significance, and attending to social as well as clinical impact of treatment.

The meanings and measurement of clinical significance.

The meanings and measurement of clinical significance.

Quantifying the Clinical Significance of Drug—Drug Interactions: Scaling Pharmacists' Perceptions of a Common Interaction Classification Scheme

To develop a precise, interval level scale of the clinical significance of drug-drug interactions that reflects the professional judgments of practicing pharmacists. A convenience sample of 63 practicing pharmacists representing hospital (clinical and staff) and retail (chain and independent) practice settings. Pharmacists judged the similarity among 15 interaction categories that have been commonly used to classify drug-drug interactions. A multidimensional scaling technique produced a spatial representation (i.e., a psychological map) of the structure inherent in those similarity judgments. Pharmacists' ratings of clinical significance were projected onto that same spatial representation using a multiple regression procedure, and the resulting information was used to develop a scale of clinical significance. The clinical significance scale developed from pharmacists' judgments was substantially different from a comparison scale published in a popular reference. The new scale was more precise than the comparison scale, and it also approximated an interval level of measurement. The judgments used to produce the new clinical significance scale were not reliably influenced by pertinent demographic characteristics of the sample. Inconsistencies between published clinical significance scales and the professional judgments of practitioners could affect patient care to the degree that a summary measure of clinical significance affects a practitioner's response to a potential drug-drug interaction. The clinical significance scale developed in this study has good measurement characteristics and reflects the professional judgments of practicing pharmacists. Use of the new scale is recommended on these grounds, although further assessment of its generality is warranted.

AN ANALYSIS OF TOKEN REINFORCEMENT:

The effects of an adjusting, fixed-ratio schedule of token reinforcement on the mean number of legible letters written by special education students were examined in a pretest-posttest design with repeated measures. Data were also collected for the frequency of such behaviors by regular elementary special education students enrolled in the same school to determine if gains would be comparable to those of such a group. Teacher ratings as to legibility of the handwriting were also taken. The overall results indicate significant gains by the token-reinforcement group, with respect to both frequency and rating of legibility. No such improvement was evident in the control group. The importance of taking normative data for mainstreaming and measures of clinical significance is discussed.