Client Ratings Research Articles

Measurement Qualities of a Self-Report and Therapist-Scored Functional Capacity Instrument Based on the Dictionary of Occupational Titles

Studies provide convincing arguments to support the development of functional capacity instruments based on the Dictionary of Occupational Titles (DOT). The purpose of this study is to investigate the item-level measurement qualities of a newly developed DOT-based functional capacity instrument for clients undergoing rehabilitation treatment for back pain. Client and therapist ratings were collected on 124 clients from 27 rehabilitation sites using the newly developed Occupational Rehabilitation Data Base (ORDB) functional capacity instrument. Rasch analysis was used to investigate: (1) unidimensionality, (2) hierarchical item difficulty continuum, (3) rater severity, and 4) person-item match. Overall, the functional capacity scale of the ORDB showed good measurement qualities. All items, except the Handling item fit the Rasch measurement model. Because of high fit statistics and loading on factors independent from the remainder of the items, the "handling" item was removed, from further analyses. Separate client-rated and therapist-rated instruments retained good item-level psychometrics. While client and therapist items showed similar item-difficulty hierarchical structures, clients had a tendency to be more severe in their rating and the correlation between client and therapist ratings was relatively low, 0.32. These findings suggest that Handling items should not be included as a DOT measure for clients with back pain. While the above psychometric study supports using client or therapist ratings as independent instruments, the lack of concordance between these ratings requires further investigation.

Assessing Clients' Progress Through a Permanency Planning Program

Eighty-five HIV-positive mothers were tracked in a descriptive analysis of progress through a nonprofit organization's program of legal and mental health services designed to assist ill parents in making a permanency plan for their children after the parent's death. Data on service utilization was analyzed using a Hierarchical Classes (HICLAS) analysis, which uncovered an interpretable pattern of services that clearly distinguished basic (e.g., intake), legal, and clinical services. From the HICLAS output we created an index of progress: [no services] < [basic services only] < [basic + legal services] < [basic + legal + clinical] services. An index of mood was constructed from staff ratings of clients during home visits. Multiple regression analyses revealed that the number of different moods observed in a client was positively correlated with progress. Number of moods was also positively correlated with positive outcome, that is formulating a legal permanency plan and/or transferring care of the child to a new caregiver. Mediation analyses suggested that number of observed moods influences progress, which in turn increases the likelihood of a positive outcome. A second study replicated the prediction of mood on progress in a group of new caregivers who were not mothers.

Predictors of the Working Alliance in Assertive Community Treatment

Past research has found that a positive working alliance between clients and their case managers is modestly correlated with client outcomes. The current study tried to identify the predictors of the working alliance in a sample of 115 clients who were receiving services from Assertive Community Treatment (ACT) teams. All of the clients suffered from severe mental illness, had a substance use disorder and were homeless at baseline. Both the client's rating and the case manager's rating of the working alliance were assessed at 3 months and 15 months post baseline. Client characteristics, particularly motivation to change, explained more of the variance of the client's rating of the alliance than treatment variables or client change on the outcome variables. On the other hand, treatment variables (e.g., the amount of transportation services provided) and client change on the outcome variables explained more of the variance of the case manager's rating of the alliance.

Unilateral Treatment Exit: A Failure of Retention or a Failure of Treatment Fit?

This article highlights perceived weaknesses in the current understanding of unilateral client exit from alcohol and other drug psychosocial treatment. It is argued that unilateral client exit is typically interpreted as a “failure of retention” when it could equally be interpreted as a “failure of treatment fit”; that is, the failure to deliver treatment optimally suited to actual client attendance. The results from an exploratory study conducted within a failure of treatment fit framework are presented. This study explored the possibility that short-term outpatient alcohol and other drug (AOD) treatment attendance may be an intentional service use strategy and that short-term attendees may exit AOD treatment as satisfied, successful, consumers. Standardized questionnaires were administered to clients (n = 109) starting a new treatment episode between June-August 2003 at admission and two-months post admission. Questionnaires explored the accuracy of client estimates of future appointment attendance, the reasons for short-term service exit, the satisfaction and outcome ratings of short-term clients, and the extent to which these compared with their longer-term peers. Findings suggest short-term treatment attendance is not an intentional service use strategy. However, despite attending fewer appointments than intended, most short-term participants reported significant levels of service satisfaction and problem improvement, and at a level comparable with their longer term peers. Problem-improvement was endorsed as the number one reason for short-term service exit. Implications for treatment delivery are discussed in light of these findings.

Effects of motivational interviewing training on mental health therapist behavior

This study examined the effectiveness of training community mental health therapists in motivational interviewing (MI) adapted to treat clients with co-occurring disorders. Ten therapists with high caseloads of culturally diverse clients in two different community mental health settings fulfilled all study requirements. MI training consisted of a two-day didactic and experiential workshop followed by eight biweekly small group supervision (coaching) sessions. Using an interrupted time series design, 156 randomly selected therapy sessions involving 28 clients were coded for assessment of therapist fidelity to MI at multiple points in time, both pre- and post-training. Employing hierarchical linear modeling analysis, significant improvement in MI skill was observed after training on five of six key therapist ratings, and on the sole client rating (client change talk) that was examined. Importantly, the present study demonstrates training-related proficiency in motivational interviewing using: (a) a representative sample of mental health therapists from the community; (b) a protocol emphasizing adherence to a mental health treatment regimen as well as management of substance use behavior for clients with co-occurring disorders; (c) repeated random observations of therapy sessions; (d) measurement of training-related changes in clinician skills and self motivational statements by clients. Findings of this effectiveness study compared favorably with efficacy literature on MI training.

A Psychotherapy process study of therapist in session self-awareness

Therapist self-awareness has been hailed as a critical component of clinical practice. Yet initial evidence suggests that momentary states of in-session self-awareness in therapists may be distracting. To test the relationships among therapist in-session self-awareness, its affective and behavioral manifestations, therapist management strategies, and client perceptions of the therapy process, therapist and client dyads participated in process recall of a therapy session. Results suggest that therapist self-awareness was helpful and related to more positive client ratings of the therapy process. In addition, therapists were most likely to use basic techniques (e.g., asking a question, using paraphrase) as a way to manage distracting self-awareness, yet no one management strategy was seen as most effective. Implications of the findings are discussed.

Client and Counselor Trainee Attachment as Predictors of Session Evaluation and Countertransference Behavior in First Counseling Sessions.

This study investigated client and counselor trainee attachment as predictors of session evaluation and countertransference behavior in 93 first counseling sessions. Results indicated that client attachment predicted aspects of session evaluation, whereas counselor attachment and the interaction of client and counselor attachment predicted aspects of countertransference. Specifically, client fearful attachment was negatively associated with client ratings of session smoothness and depth and with counselor ratings of session smoothness. Counselor dismissing attachment was positively associated with supervisor ratings of hostile countertransference. Furthermore, interactions between client and counselor attachment predicted hostile and distancing countertransference reactions, such that countertransference was highest when the client had a preoccupied attachment pattern and the counselor trainee had a fearful or dismissing attachment pattern.

A Modelled Study of Content Delivery Networks

Delivery Networks (CDNs) appeared at late 90's as a means of reducing response time experienced by Internet users by distributedly placing multiple servers, called surrogates, in the client network neighbourhood. Many companies have deployed their own propietary CDN hiding implementational aspects, which guides to a lack of a general open model that describes CDN behaviour from an analytical perspective. This article proposes a general expression for a CDN considering certain design parameters, such as caching hit rate, delay and number of clients and surrogates.

Predicting change for individual psychotherapy clients on the basis of their nearest neighbors.

This study extended client-focused research by using the nearest neighbor (NN) approach, a client-specific sampling and prediction strategy derived from research on alpine avalanches. Psychotherapy clients (N=203) seen in routine practice settings in the United Kingdom completed a battery of intake measures and then completed symptom intensity ratings before each session. Forecasts of each client's rate of change and session-by-session variability were computed on the basis of that client's NNs (n=10-50 in different comparisons). Alternative forecasts used linear or log-linear slopes and were compared with an alternative prediction strategy. Results showed that the NN approach was superior to the alternative model in predicting rate of change, though the advantage was less clear for predicting variability.

Perspective divergence in the working alliance

The aim of this study was to investigate the divergence of client and trainee-therapist views of the different aspects of the alliance over the course of counseling and to relate those divergences to session-level therapeutic impacts. Forty-eight trainees and their clients completed therapies averaging 14 sessions. Results indicate that divergences in perspective did not influence the clients’ evaluations of the impact of their sessions. Only client ratings of the alliance were significantly correlated with positive session impacts. Measurements of the alliance taken at early, middle, and late phases indicated that, although both client- and therapist-rated alliance scores increased over time, convergence did not increase as therapy progressed. Implications for conceptualizing divergence as an expectable process that may not be a hindrance to positive therapy and the differences between experienced and trainee-therapists are discussed. Divergente Perspektiven zur Arbeitsbeziehung Das Ziel der Untersuchung...

The In-Session Self-Awareness of Therapist-Trainees: Hindering or Helpful?

Although therapist self-awareness has been hailed as a critical component of psychotherapy, recent evidence suggests that therapists' in-session self-awareness may hinder rather than help the therapeutic process. The authors examined the in-session self-awareness of therapists in training (trainees) in relation to their interpersonal involvement and the reactions of volunteer student clients (volunteer clients). Trainee in-session self-awareness variables predicted over 50% of the variance in their interpersonal involvement and the volunteer clients' perceptions of the therapeutic alliance. Contrary to previous research, trainees' level of in-session self-awareness was generally helpful rather than hindering from both the trainee and student-client perspectives. Potentially hindering aspects of in-session self-awareness did emerge, however, in that trainees' increasing efforts to manage their self-awareness were related to decreased trainee involvement and lower volunteer client ratings of the therapeutic alliance.

Criminal recidivism in three models of mandatory drug treatment

Although research has generally been supportive of compulsory treatment programs for drug abusers, findings remain mixed, and few studies have assessed the impacts of different coercive program elements. This study compared criminal recidivism outcomes of 350 clients mandated to the same long-term residential treatment facilities from three different legal sources. On several measures of recidivism, including long-term re-arrest rates that controlled for time at risk, clients mandated from two highly structured programs were found to recidivate at less than half the rate of comparison group clients. This group effect was upheld in multivariate models that controlled for pre-treatment differences and other factors related to recidivism. Combined with results of a previous retention study involving these clients, the findings provide support for the use of structured protocols for informing clients in mandatory programs about legal contingencies of participation and enforcing contingencies through frequent contact between legal agents and treatment staff.

Client Outcomes and the Working Alliance in Assertive Community Treatment Programs

This study examined the relationship between outcomes and the working alliance in clients who were receiving assertive community treatment only or integrated assertive community treatment (assertive community treatment plus substance abuse treatment). All 98 participants had a severe mental illness and a substance use disorder. The Working Alliance Inventory assessed the alliance from the perspective of both the client and the case manager at 3 and 15 months into treatment. The six outcome measures were stable housing, client rating of psychiatric distress, interviewer rating of psychiatric symptoms, self-report of days used alcohol or drugs, and interviewer rating of substance use. Only 4 of 24 correlations were significant, indicating little relationship between the strength of the working alliance and client outcome.

Neurocognitive correlates of therapeutic alliance in schizophrenia.

Although therapeutic alliance in schizophrenia has been linked with treatment adherence and outcome, less is known about its clinical correlates. This study explored neurocognition as a possible predictor of perceived therapeutic alliance among people with schizophrenia in cognitive behavior therapy. Twenty-four participants with schizophrenia spectrum disorders and their therapists were administered the Working Alliance Inventory, Short Form after 3 months of therapy. Totals for clients and therapists were correlated with measures of verbal memory, premorbid intelligence, visual spatial reasoning, executive function, and attention, all obtained before beginning therapy. Poorer performance on verbal memory was significantly related to client report of stronger alliance, whereas better performance on visual spatial reasoning was significantly related to therapist report of stronger alliance. Client and therapist ratings of therapeutic alliance were significantly and positively related. Clients' abilities may differentially affect therapist and client perception of therapeutic alliance in schizophrenia.

Measuring “addiction” in Europe: The diffusion of the Addiction Severity Index, and its purposes and functions

This paper discusses what an instrument like the Addiction Severity Index (ASI) offers to various actors in the diverse addiction treatment systems of Europe and what the use of the ASI means and implies at the treatment system level. It is concluded that the adoption of the ASI can be linked to a number of factors, including the demands for cost‐efficiency, scientific legitimacy of addiction treatment, professionalization, internationalism, and unification of a diverse system. Furthermore, it considers the specific choices embedded in the structure and content of the ASI—how the use of the ASI structures the user's view of the situation and life‐trajectory of the individual client. Features such as the length of the interview, the time frames of the question, the exclusion of measures of dependence and alcohol‐/drug‐attributable problems and the inclusion of client's rating, are discussed. It is concluded that the opinions about the usefulness of the ASI in Europe vary. For the future of measurement of alcohol and drug problems in Europe, for system monitoring and international comparisons, shorter instruments seem to be the likely path forward. The article draws to a large extent on the papers presented at a conference, “Measuring ‘addiction’ in Europe: clinical, sociological, cultural and policy aspects of Addiction Severity Index (ASI) and other international instruments”, at Skarpö, Sweden, in January 2003.

Clinical outcomes measurement: Interrater & interdevice reliability of physical therapist & client rating of low back pain with a visual analogue scale & a picture scale

The purposes of this study were to assess congruence between 1) physical therapist and client perception of low back pain and 2) Visual Analogue Scale (VAS) and Picture Scale (PS) pain ratings. Congruence of practitioner and client pain rating is essential in clinical decision making. Clinical use of these scales is widespread. However, there is limited research regarding their reliability in rehabilitation medicine. The sample of convenience (n = 28) consisted of volunteer subjects (median age = 40.5 years), from one outpatient orthopedic physical therapy practice, that were experiencing low back pain. A physical therapist and client simultaneously rated the client’s pain with the VAS and PS. Each was blinded to the other’s rating. The VAS was standard (10 cm. line) and the PS consisted of eight (8) hierarchically arranged realistic facial expressions. Descriptive statistics were used to summarize demographic and pain rating data. The Spearman (rho) Rank Correlation was used to determine the relationship between therapist and client pain ratings, and between VAS and PS scores. The Wilcoxon Signed Rank (α2=.05) was used to determine if there were corresponding significant differences. There was high correlation between physical therapist and client using the VAS (rranks=.80) and the PS (rranks=.75), and between VAS and PS values for physical therapists (rranks=.93) and clients (rranks=.83. There was no statistically significant difference (p>.05) between physical therapist and client pain ratings using either scale. However, there was a statistically significant difference between client VAS and PS ratings. Interrater reliability of the VAS and PS is high, suggesting that physical therapists can rely upon their judgement of client pain using either of these scales, in clinical decision making. However, interdevice reliability of these scales is questionable suggesting that the scales are interpreted differently and should not be used interchangeably.

Impact of Visual Cues in Computer-Mediated Vocational Test Interpretation

Online delivery of rehabilitation counseling can make services more widely available, but more information is needed about specific features before it can be considered a viable approach. Video conferencing with broadcast of counselor and client image is one recommended tool with some evidence that client rating of session depth may be equivalent to face-to-face sessions but at a cost of increased client discomfort. In this study, the counselor's image was broadcast to 24 participants in online test interpretation sessions Broadcast of participant image to the counselor was contingent on random assignment. The Session Evaluation Questionnaire completed by participants at the end of the session showed no significant impact on ratings of session of depth between camera-on and camera-off conditions. Session smoothness/comfort ratings, however, were significantly higher when the participant image was not broadcast to the counselor. Implications for rehabilitation counseling are discussed.

Clients' engagement with assertive outreach services: A comparison of client and staff perceptions of engagement and its impact on later engagement

Background: Studies have shown the effectiveness of assertive outreach services in engaging clients with severe mental health problems. However, there is little research examining this engagement process in more detail. Aims: This study developed a reliable self-report measure of engagement based on <citeref rid="b7">Hall, Meaden, Smith, & Jones' (2001)</citeref> observer-rated measure of engagement. The study investigated whether a self-report measure predicted clients' engagement with assertive outreach services 6 months later. The research also compared client and keyworker perceptions of engagement. Method: Twenty-five clients and their keyworkers, from three assertive outreach teams were interviewed. Results: The engagement measure (self-report version) was found to have good test-retest and internal reliability. Staff and client ratings were correlated at Time 1 but not at Time 2. Staff ratings at Time 1 were predictive of staff ratings at Time 2. Client ratings were not predictive of client ratings 6 months later. Items on the measures had different correlations with overall engagement for clients and staff, although there was some overlap. Conclusions: The engagement measure (self-report version) is quick, easy and reliable and can be used in conjunction with the observer version. The measure has significant clinical and research value. Declaration of interest: None

Empowerment Events in Process-Experiential Psychotherapy of Depression: An Exploratory Qualitative Analysis

The goal of this study was to analyze different types of empowering therapy events. Twelve significantly empowering therapy events, involving 9 depressed clients undergoing individual process-experiential psychotherapy, were selected from a larger collection of significant events (Elliott et al., 1990) using client ratings of feeling powerful/hopeful/active. The data consisted of transcripts of client-identified significant events along with parts of client and therapist Brief Structured Recall interviews. The events were first sorted into broad domains and then into types. Finally, general and variant themes across domains were tracked. We found 5 types of empowerment events: poignant, emergent, decisional, empowerment, and accomplishment. A description is provided for each event type across domains: client processes, therapist processes and interaction, and therapeutic impact. Different therapeutic processes were present in a variety of empowerment types. However, some therapist interventions contributed to several types of empowerment, among them communication of empathic understanding, empathic affirmation of client growth-promoting aspects, and 2-chair work.

Factors Affecting Auditors' Assessments of Planning Materiality

This study investigates the planning materiality values used by auditors in The Netherlands for a sample of engagements performed by Big 5 and non-Big 5 firms in 1998–99. We find that, consistent with archival evidence from KPMG in Elliott (1983), planning materiality is not a constant percentage of a base, but increases at a decreasing rate with client size. In addition, we find that planning materiality values increase with the quality of the client's control environment and the magnitude of the client's rate of return on assets, while decreasing with the complexity of the client. We also find that Big 5 firms use lower planning materiality values than non-Big 5 firms, ceteris paribus, which is consistent with the production of relatively higher audit quality levels by the Big 5. Finally, we find that auditors use lower materiality values in situations where earnings might be managed to show a small profit or a small loss.