Early psychologist to conclude there was no evidence to support the effectiveness of psychotherapy
Analysis of 475 studies showed that the effects of therapy are superior to no-treatment and placebo control conditions, and therapies appear to have equivalent effects when compared with each other across a variety of disorders. Who?
Smith, Glass, and Miller (1980)
Who says "Funders, providers, and consumers all like to pretend that efficacy is the same as effectiveness, and lists of empirically supported treatments feed this delusion."?
3 part agenda of the current Committee on Science and Practice?
1. reliability of review procedures through standardization and rules of evidence
2. improved research quality
3. increased relevance and dissemination to the professions and public.
(Weisz et al, 2000).
Despite being an "empirically supported" or efficacious treatment, what do we know success of treatment is largely dependent on?
the client and the therapist
average length of tx?
We tend to favor brief psychotherapy:
Average is about 5 sessions
Think about ethics of prolonged vs. shortened tx.
Think about third parties/insurance
Appropriateness of treatment
Need for booster sessions
(Hansen et al., 2002)
Dose effects and tx
Anderson and Lambert, 2001 and others found that about 50% of patients who enter treatment in clinical settings will show clinically meaningful change after 13-18 sessions, and an additional 25% meet the same standards after 50 sessions. Suggests that 50% of patients are not served by less than 20 sessions. That's a lot of people to not serve for the sake of electing brief psychotherapy.
have defined harm (ala Dimidjian and Hollon, 2010)--but we know that biased interpretation of data exists--that too is a form of harm. Such strict editors, little is to be gained by mentioning weaknesses in design of study. Ref?
Bergin & Garfield, 1994
Example of how the very strict nature of efficacy studies can be misleading?
One common rule is testing a treatment on a single disorder, thus the necessity of a homogenous treatment group. Those in the tx group are so carefully chosen for being free of comorbidity etc., that they no longer resemble the average patient. But researchers will convince readers that subjects look just like average patient. Bergin and Garfield (2004).
Empirically Supported Treatments are:
treatments shown to be efficacious in randomized clinical research trials with given populations (APA Task Force on Promotion and Dissemination of Psychological Procedures, 1995; Chambless & Hollon, 1998)
Must be accumulation of evidence, no one study does it.
Demonstrations of a treatment's efficacy typically involve:
An RCT in which an intervention is applied to diagnosed cases and analyzed against a comparison condition (e.g. wait list, alternative tx, treatment-as-usual), to determine the degree or relative degree of beneficial change associated with the treatment.
Hoagwood et al. (1995)
Criteria for EST
1. Comparison with control group in a randomized control trial, controlled single-case experiment...and in which the EST is significantly superior to no treatment, placebo or alternative treatments, or in which the EST is equivalent to a treatment already established in efficacy and power is sufficient to detect moderate diffs.
2. Studies conducted with manual, population, treated for specific problems, for whom inclusion criteria have been delineated in reliable, valid manner, where reliable, valied outcome assessment measures are used to target change, and use appropriate data analysis.
3. Superiority of EST must have been shown in at least 2 independent research settings.
4. For designation of efficacious and specific, EST must have been shown to be statistically significantly superior to pills or psychological placebo or to an alternative bona fide treatment in at leasr two independent research settings.
What are the advantages to no treatment control conditions?
When treatment condition does better, eliminates the potential of explanation of spontaneous remission, historical effects, maturation, regression to the mean.
Kendall et al. 2004
advantages of wait-list condition
Provides additional control
clients have taken the stop of initiating treatment and may anticipate change due to therapy
If assume the clients are the same in tx and wait-list control condition in terms of gender, age, ethnicity, severity of presenting problem, and motivation, can make inferences about changes made in tx condition over and aboive those manifested by wait list clients--like that they were due to intervention and not extraneous factors that were operative for both groups.
Potential problems with wait-list controls
wait list client might experience a life crisis that forces immediate professional attention
lower retention rate: differential attrition
ethical issue: withholding treatment (Bersoff and Bersoff, 1999)
Attention placebo conditions
(nonspecific control conditions)
Alternative to wait-list
rule out threats to internal validity but also control for the effects that might be due simoly to meeting with a therapist
Intention: to mobilize the EXPECTANCY of positive gains.
Allow to attribute changes produced by therapy over and above the effects of nonspecific factors.
Limitations of attention-placebo conditions
With long-term therapy, ethical question to offer contact that does not deal with the problem
Maybe be difficult for the therapist to accomplish a true placebo (and not address patient's problems)
Demand characteristics suggests that when a therapist predicts a favorable outcome, clients will tend to improve accordingly (Kazdin, 1998) and the demand characteristics may not be the same for therapists in the placebo condition.
Less attrition because both groups receving tx
Hopefully quality care is given to both groups, so less ethical concerns
Advantages of treatment manuals
1. enhancing internal validity of study
2. enhancing treatment integrity
3. facilitate training
4. enhance reliability
Kendall et al., 2004
What are important factors to make sure therapists in RCTS are comparable across?
1. length and type of of prof and clinical experience
2. training background
3. expertise in intervention
4. allegiance with treatment
5. expectation that intervention will be effective
In RCTs, interventions when comparing two txs should be comparable across what factors?
1. duration of tx
2. length, intensity, and frequency of therpaist contact w/ client
3. credibility of tx rationale given to client
4. setting where tx provided
5. degree of involvement of persons significant to client.
Best way to measure two alternative treatments in an RCT?
1. not be differentially sensitive
2. cover the range of psychosocial functioning that is a target for therapeutic change
3. include measures that tape costs and possible neg. side effects of interventions
4. be unbiased with respect to alternative kinds of intervention.
who found that use of manuals did not restrict flexibility?
Kendall & Chu, 1999
Who found that effective use of manual-based treatments must be preceded by adequate training (not enough to follow the manual)
What are some of the therapist variables known to affect treatment outcome that therapists can still attend to while using a manual?
Why are fidelity checks important?
Because if treatment adherence is not top notch during RCTs, then the very definition of the independent variable (the treatment) is violated, treatment integrity is impaired, and replication is impossible.
Dobson and Shaw, 1988
Example of an efficacy study that was also efficacious
Franklin, Abramowitz, Kozak, Levitt, & Foa, 2000.
Exposure and ritual prevention (EX/RP)
tx typically evaluated for efficacy, produced clinically significant change in outpatient clients, many of whom had comorbid problems and extensive tx histories.
Initial evidence that at least some types of treatments may be transported across contexts.
permits the evaluation of therapy-induced change on different levels: specifying level and impact level.
Specifying level: skills, cognitive, or emotional processes or behaviors modified during treatment (number of positive spouse statements generated during marital relationship task)
Impact level: general level of functioning of client (absence of diagnosis)
Kendall et al., 1981
evaluate outcomes for all participants at the point of randomization
so provides an estimate of effectiveness
When comparing conditions in an RCT, when does a group achieve statistical significance?
if magnitude of the mean differences is beyond what could have resulted by chance alone.
Statistical tests of signifiance in RCTs do not inform us about...
Reliance solely on tests of significance in RCTs can leads us to what conclusions?
Perceiving differences (like treatment gains) as potent when in fact they may be clinically insignificant.
EX: A treatment outcome study may demonstrate that mean BDI scores are significantly lower at post-treatment than prepreatment. An examination of the means, however, reveals a shift from 29 to 24. Given large sample size, this may well be significant statistically, but not practically since at pre and post test the scores are still in the range of depressive distress. Kendall et al., 1987.
Even Marked changes on one or more factors may have little effect on a person's perceived quality of life. Gladis et al, 1999.
the meaningfulness or convincingness of the magnitude of change
What questions does clinical significance answer?
"Did the remediation of the presenting problem produced by treatment return the patient to the point that the initial problem was no longer troublesome?"
Measures of clinical significance?
Normative sample comparisons
client's own perceptions of their quality of life
What is one established measure of clinical significance?
The Reliable Change Index (RCI)
Calculating the number of clients who move from a dysfunctional to normative range
difference score prepost divided by standard error of measurement
RCI is influenced by the magnitude of change and the reliability of the measure
Jacobson et al, 1984
Baron and Kenny, 1986
A variable that delineates conditions under which a given treatment is related to an outcome, so the nature o the treatment-to-outcome association is excpected to vary as a function of the moderator.
Influences either the direction or the strength of a relationship between an iv (predictor) and a dv (Criterion).
A variable that serves to explain the process by which a treatment impacts an outcome.
Are answering the question, "Why?"
Common moderators in research:
mode of delivery (indiv vs. group)
type and source of outcome measure
How do you test for a moderator?
It is an interaction
When using multiple regression, the predictor (e.g., treatment vs. no treatment) and the moderator (e.g., age of client) are main effects and are entered into the regression equation first, followed by interaction of the predictor and moderator. If only intersted in interaction, can throw them all in together first.
Then if significant, do post hoc probing to see where significance lies.
AIKEN & WEST, 1991
Advanatage of meta-analytic approach over box scores:
1. effectiveness is quantified on continuous scale
2. effect sizes can be computed
3. perhaps less bias, quantitative info
4. influence of treatment moderator variables and interactions can be assessed simultaneously, where the effect size is the dv
5. info is of use to policy makers
Durlak & Lipsey, 1991
A common measure of alliance in process research?
Working Alliance Inventory
Horvath & Greenberg, 1989
Perspective of client
problems with memory, and the importance of video or audiotaping sessions for studies, REF:
Who noted that therapists behave in different ways when recorded
ex: Rogerians behaved in less client-centered ways, and patients made more favorable self-reference statements
What is a good example of how murkey definitions of adequate psychotherapy outcome lead to descrepancies in the literature?
Eysenck (1952) vs. Bergin (1971) regarding the effectiveness of psychoanalysis using the same data set.
different criteria for success, Eysenck said psychoanalysis didn't work, Bergin said they were doing much better.
An example of a review of studies of that focused on one specific disorder but did not overlap at all in use of similar outcome measures?
1. Ogle, Lambert et al. 1990
2. All clients with agoraphobia (106 studies), 98 different outcome measures.
Three most common outcome measures found?
STAI, BDI, SCL-90
Froyd, Lambert, & Froyd, 1996
Example of Multitrait scale and monotrait scale
MMPI vs. BDI
Derogatis & Meliseratos, 1983
reflects psychological symptom patterns of clients
nine primary symptom dimensions
3 global indices of distress
Global Severity Index GSI = best indicator of distress overall
good for heterogeneous clienet samples
tells you about physical symptoms too
*sensitive to change
Reliability of change scores vs. reliability of measure
reliability of change scores is not equal to the reliability of the measure
Each time you test, have true score + error so error each time = more error in change score
Ability of a measure to estimate or describe the dimension, phenomenon, or construct it purports to measure.
What is a neglected validity issue in outcome research?
Sensitivity to change: Does BDI reflect same changes made after treatment that Hamilton does?
Attempts made to address the issue of clinical vs. statistical significance?
In earliest studies of therapy outcome, patients were categorized as improved, cured, etc. to imply meaningful change. But the lack of precision in these ratings resulted in waning use.
What's a better way to establish clinical significance?
Statistical method that says "what is a post test score that is likely to belong to a functional individual?"
patient must change enough so that you know score change exceeds measurement error (reliable change index)
many use this (Ogles et al., 2001)
What is the reliable change index not so effective at?
reliably estimating worsening in clients.
Example of a study where discrepancy between clinical and statistical significance:
TDCRP data: Treatment of Depression Collaborative Research Program
Over 75% of clients classified as making clinically sig change on all three measures used
25% made changes on one of the measures but not the other 2
This discrepancy is important!!! Should a client have to improve on more than one measure to be considered improved?
Ogles et al., 1995
Use of multiple normative groups to measure clinical significance in outcome studies?
Useful because allows people who are severely disturbed to make notable progress and perhaps begin functioning more like outpatients and have the noticed, even if not functioning like a "functional control."
Tingey et al., 1996
How has managed care influenced the delivery of mental health services?
1. number of sessions needed for improvement
2. differences in tx related to the practitioner's level of training.
3. management of the quality of care for the individual patient through outcome management or patient-focused research
4. search for empirically supported treatments (Chambless et al., 1996)