New paper challenges evidence for some gold-standard psychotherapies

Psychologists rely on a “gold standard” of clinical treatments for a wide range of conditions from depression, insomnia and post-traumatic stress disorder to adult ADHD and borderline personality disorder. But a new paper led by University of Victoria social psychologist John Sakaluk and University of Kansas clinical psychologist Alexander Williams, published today, calls into question some of the statistics that support these psychotherapies. 

A master list of 78 empirically supported treatments (ESTs) is maintained by the American Psychological Association’s Division 12, the arm of the association that develops guidelines. All 78 ESTs have been clinically tested and are used by psychologists in Canada and the US based on empirical measures of scientific success which inform everything from clinical training to research funding. 

“But more than half of the ESTs fared poorly across most of our metrics,” says Sakaluk. “Increasing peer-reviewer attention to the metrics we evaluated would therefore complement the ongoing efforts of Division 12 to increase the quality of EST research and evaluation.” 

There is growing concern that many research findings in psychology cannot be replicated by independent research teams using the same or similar scientific procedures. In light of this “replication crisis,” Sakaluk and Williams point to the need to re-evaluate existing systems that rely on the empirical measures examined in their study. 

Both therapeutic clients and other researchers are counting on ESTs to score well across statistical metrics of evidence; otherwise, we need to rethink what ‘empirically supported’ is supposed to convey.
— UVic social psychologist John Sakaluk 

Over the last year, Sakaluk, Williams and their coauthors conducted a meta-scientific review of approximately 450 articles and re-analyzed more than 3,000 tests of the effectiveness of all 78 ESTs. 

In the first study of its kind to encompass such a large sample, the researchers found that many of the earlier studies claiming the efficacy of ESTs contained “statistical typos,” imprecise research designs and weak evidence that the therapies worked. 

The paper, “Evaluating the evidential value of empirically supported psychological treatments (ESTs),” was published today in the Journal of Abnormal Psychology.

A press kit with a high-resolution photo of Sakaluk and an infographic of the research is available on Dropbox.

-- 30 --