My head has been spinning at a faster and more disorienting rate since I embarked on this week's "Controversy"-themed readings than it has for any week prior. For all the furiously scribbled "Wow's," "What the's," and "Seriously??"s that pepper the margins of my copies of both the Lilienfeld and the Dawes et al papers, and the general sense of stunned disillusionment I felt with the vast majority of practitioners - and researchers! - within this field I'm venturing into (I mean, "rebirthing" as a "therapeutic treatment" eligible for continuing education credit from APA, at least as of 2002? Seriously??), the main themes that emerged from all my margin notes and scribblings were two-fold: 1) wow, more power to statistical analyses and rigorous empirical tests of existing theories for calling out the clinicians and self-assured "professional psychologists" on their dogmatic, obstinately fact-resistant, and often harmful B.S.! and 2) wow, so many exciting openings for good, fresh, explicitly articulated and empirically testable new theory to walk on the scene and finish off what the statistical expose started!
Before attempting to explain what I mean by either of these "themes," I should throw in the disclaimer that I'm still tangled up in massive conclusions regarding some of the claims made in each of these articles, and I don't claim to be at all certain that my interpretations and criticisms of what I took to be various conclusions that the authors seem to be arguing for aren't just the direct result of my own faulty understanding. I can't wait to address some of my confusions in class on Wednesday and get a better handle on what exactly was found in some of the clinical vs. actuarial judgment studies that Dawes cites, for example, and how it was that "accuracy" of prediction was measured in cases of psychiatric diagnoses and the like, and many other such questions. But meanwhile, I figure I might as well attempt to articulate my own current interpretation and the source of my at-least-partial dissatisfaction with the authors' conclusions in, which at least might shed some light on what exactly it is that I'm confused about.
So, with all that said, here goes: tackling first the Lilienfeld "Psychological Treatments That Cause Harm" article, with all its bone-chilling exposes of therapeutic "treatments" that continue to be done despite multiple research findings showing that they do more harm than good, I was completely on board and even cheering for Lilienfeld's explanation of what can be gained from studying potentially harmful treatments - such as a greater degree of caution and humility on clinicians' part, on account of realizing that not all therapies are equal and that there can be unknown and unintended consequences to even the most seemingly sound and uncontroversial treatments (such as relaxation therapy for Panic Disorder), and also a greater understanding of the specific mechanisms of change that underlie various therapeutic treatments and when, and with whom those mechanisms can most effectively be applied. But I parted ways with Lilienfeld when he made the claim that the identification of potentially harmful treatments should be accorded "higher priority" than the identification and testing of effective, evidence-based treatments, because I don't see that this solution would go deep enough to address the underlying problem. The article cites multiple examples of clinicians continuing to implement certain treatments, and the APA continuing to sanction them, despite the availability of ample data suggesting that those treatments do more harm than good. What reason have we to expect that gathering more such data - by focusing on the sorts of non-empirically-supported "fringe" treatments Lilienfeld describes and proving through further comparisons and statistical analyses that those treatments make at least some patients worse instead of better - would motivate those clinicians to change course and admit that they've been doing a fruitless or even harmful treatment all this time? Lilienfeld also observes in the article that most of the clinicians out there who are conducting these treatments are perfectly well-meaning and sincerely convinced, more and more so after years of selective information processing, self-fulfilling prophecies, and confirmation bias, that their treatment of choice is the most effective way to help reduce the suffering of the people who come through their door. What, then, would serve as a more compelling argument for them - a few more replications of treatment outcome trials whose results say that some patients got worse instead of better (which the clinicians who read this report can just dismiss on the grounds that "well, the therapists in the study were obviously poorly chosen and didn't do the treatment right" or "well, they didn't give the therapy enough time to work before testing the outcome - everyone knows 3 years isn't long enough to bring out all 13 of the repressed 'alters' in a DID patient's psyche and get them all to be friends again!")? Or a re-conceptualization of the basic causal mechanisms involved in the disorder and in the corresponding therapeutic change process, which in turn sheds direct light on the connection between what the clinician has been doing (e.g. Critical Incident Stress Debriefing) and the harm that results (e.g. recall of traumatic event get more firmly and elaborately encoded in long-term memory)? To my mind, at least, this is far more inescapable and compelling an argument - especially if paired with an alternative proposed treatment approach whose effectiveness is similarly explained in terms of known, basic causal relationships. As a case-in-point, I don't think anyone did extensive randomized controlled trials or meta-analyses to determine the possibly deleterious effects of bloodletting or phrenology or faith healing; rather, the advent of new empirical and theoretical breakthroughs in medicine and neurology led to radical re-conceptualizations of what causes certain illnesses, what sort of relationship exists between the structure and function of the human brain, etc. gradually replaced the once-universally-accepted treatments of old, rendering them implausible and obsolete.
And I had a strikingly similar reaction to Dawes et al's "Clinical vs. Actuarial Judgment" article, though it'll be even harder for me to articulate given my confusion about some of the main statistical results they cite; in a nut-shell, though, my fundamental question is: sure a mathematically derived formula does better at consistently extrapolating from a theory to a predicted outcome than any number of clinicians can do with their "bare hands," so to speak (actually their bare brains, in this case - which, as the article astutely observes, are really bad at doing multi-variable computations); but doesn't any actuarial measure ultimately hang or fall on the validity of the theory on which it was based (e.g. which variables and hypothesized predictors the creators of the measure saw fit to analyze in the first place)? And doesn't the theory itself depend crucially on "clinical judgment" - though hopefully of a more explicit, systematic, theoretically supported kind than the sort described in most of the studies the article reports on? When the authors cite the finding that an actuarial measure derived from a bunch of expert clinicians' ratings of various symptom categories outperformed the clinicians themselves in the "accuracy" or consistency of its predictions, that only tells me (importantly and non-trivially, I realize) that clinicians are very unreliable at applying their theoretical knowledge to the computation of diagnoses and predictions "by hand," whereas a calculator programmed with the variables and predictors postulated by their theory (which is essentially what an actuarial measure boils down to, if I understand correctly) does it better. Well, ok; but unless the actuarial measure is getting near-100% accuracy (which, in the studies reported here, it never seems to come anywhere close), we still don't know whether the theoretical assumptions behind the actuarial measure are sound. I can think of numerous examples of this problem, such as the various algorithms I've seen for computing diagnoses based on the DSM - or the problem of getting IQ scores with the WAIS and other cognitive tests that appear to reflect a lack of intelligence, based on the "actuarial" prediction, but are in fact much more likely to reflect an attentional or motivational or auditory processing deficit or the fact that the examinee didn't speak English as a native language. But my broader point is that, while I agree with these authors' admonitions that we strive to minimize the effects of human error and subjective bias in our science and as well as our practice, I'd also want to be quite wary of the complacent reliance on "actuarial tests" as substitutes for theoretically-based judgment and continued monitoring/revision - just as I'd be wary of the reliance on further replications of RCT findings without the corresponding conceptual/theoretical advances to bolster our claims.
No comments:
Post a Comment