Having just read the two articles assigned for this week in direct succession - Chambless & Hollon (1998) first and Hunsley & Di Giulio (2002) second - I must say I was more struck by the dramatic difference in the tone of these articles than by the content of their claims and arguments (which, for the most part, I already agreed with and didn't find too controversial). The Chambless & Hollon article seemed almost too "wussy" in its tone - and then the Hunsley & Di Giulio paper, at least by contrast, seemed so flippant and sarcastically demeaning of its opposition that it actually lost some credibility in my eyes.
To tackle the "wussy"ness first: in outlining their criteria for evaluating whether a given treatment is actually "efficacious," "effective," etc., Chambless & Hollon repeatedly identified certain standards and potential methodological problems that seemed mighty compelling to me as evaluation criteria - and then they seemed to back off and soften their demands on the grounds that, basically, such criteria would be really hard to meet. For instance, in their explanation of the superiority of comparative treatment studies over RCTs with a no-treatment/control condition, they make a really powerful case (which I strongly agree with) that to make causal inferences about the specific treatment you're testing, you need to control for "common factors" and general assessment/expectation of gains/clinician attention effects. But then they state that it's perfectly fine for a treatment tested against a no-treatment condition to be called "efficacious" (supposing the other criteria, such as replication by independent researchers, etc., are met), because any treatment that has "a beneficial effect" is of value clinically and should be regarded as useful. Well, yeah - but don't we know by now, as I think both of these articles reference at one point or another, that most if not all forms of psychotherapy have some beneficial effect on some patients? Shouldn't the gold standard in treatment outcome research be at least a little more rigorous than this, supposing that our goal is to make progress and improve our methods rather than keep on confirming what we already know?
And speaking of a treatment's clinical value, I was equally frustrated by another instance of "backing off" and slackening of their standards in Chambless & Hollon's discussion of the distinction between "statistical significance" and "clinical significance." "It is desirable," they write, "that investigators go beyond assessment of symptoms and examine the effects of treatment on more general measures of functioning and quality of life. Not all life concerns can be reduced to specific signs and sympotms, and treatments may differ with respect to the breadth of their effect." Um, yes! But then they acknowledge that, "in practice, few studies as yet have included such measures," and that, although more studies should incorporate such data in the future, it cannot be included as a standard of efficacy. But why not? Because "authors have only recently begun to provide such data" and because "the issue of how much change to require for a given problem is quite complex." Well, ok - but does its "complexity" make it any less important a measure of outcome? What's the use of "statistical significance" if it doesn't actually capture a meaningful practical gain for the patients receiving treatment? Just because researchers haven't been looking at this to date does not mean we shouldn't hold them to a higher standard henceforth. On the contrary, it seems all the more crucial that we emphatically insist upon it.
One of the most memorable treatment research results I've learned about in the past year actually came from a recent finding by Chambless: namely, that when practicing clinicians were asked what they would find more compelling - a rigorously conducted randomized controlled trial statistically demonstrating the superiority of one treatment approach over another, on one hand, or a single detailed and descriptive clinical case report of a specific patient's treatment and gains on the other hand, they overwhelmingly chose the latter. That is, they were more likely to trust and implement the treatment approach illustrated via clinical case study than the one proven "efficacious" via fancy (but perhaps ultimately meaningless) statistical analyses. This finding certainly doesn't invalidate all treatment efficacy research in one fell swoop, but I think it does reveal a major gap (or maybe multiple gaps) somewhere between the research literature and the actual choices made by clinicians (even given research demonstrations of effectiveness and cost-effectiveness and so on). And Chambless is obviously aware of it and focusing much of her own work on attempting to address it. So, that all being the case, why be so soft-spoken in calling upon researchers to measure and report on the clinical as well as statistical significance of the treatments we're trying to put forth and market to clinicians?
That all said, my reading of the Chambless & Hollon article seems to have softened me up somewhat in spite of myself, because when I embarked on my reading of the "Dodo Bird" article, I was flabbergasted by the amount of hyperbole, sarcasm, and downright ridicule with which the authors establish their case (which I think is actually a pretty strong case - but now I can't help but wonder, is it really that strong?) against the proponents of "psychotherapeutic equivalence." Examples of this dismissive, sarcastic tone abound (e.g. "The rationale for the coding of tractability is hard to discern"; "Only by first (mis)classifying cognitive therapies with psychodynamic and humanistic therapies... and then statistically adjusting for supposed measurement problems (largely related directly to distinctions among therapies regarding what should be assessed in psychotherapy) did the results suggest equivalence across forms of psychotherapy"; and apparently presumptuous claims about what various authors would endorse, e.g. "presumably these authors would endorse the view that the same general treatment would be effective for all clients..."). In all honesty, these potshots against the psychotherapeutic equivalence advocates made the article more amusing to read, and I found myself agreeing with most of the article's claims (or at least feeling an affinity for them). But I also felt more reserved in the conclusions I drew because of the flippant, almost snide tone - and I suspect that someone who doesn't already agree with the authors would be even less likely to feel convinced, given the insults to their intelligence and the sweeping dismissal of their views at every turn.
I don't know what exactly to make of these opposite reactions I experienced to the tones of the two articles, except to note that tone is extremely important to the presentation of scientific theories and arguments - and that it can reflect on the authors' scientific credibility and earnestness in a way that inevitably colors the content itself. Seems worth thinking about, especially once we actually start writing our own scholarly papers and developing our own "scientific tone" (for better or worse... I hope for better!).
No comments:
Post a Comment