Investigating False Claims of Evidence-Based Psychotherapy

The term “evidence-based” therapy is currently being used by many in the field of psychotherapy , including practitioners and marketing personnel, to refer to something quite different from what someone might otherwise assume. It is therefore misleading and problematic.

The goal of this article is to create awareness of how false narratives of evidence-based claims are perpetuating the underlying turf-wars between different psychotherapy models and theoretical orientations. It will also weave a clear and compelling case that tackles the often overly restrictive rules of managed care on what practitioners may or may not do.

Introduction

In the evidence-based medical model, the gold standard for evidence of effectiveness is the randomized control trial (RCT). This was adopted and used to study the effectiveness of various psychotherapy models that were being practised. John Marzillier pointed out, in as early as 2004, that these RCT studies do not translate so well to measure outcome results in psychotherapy. To show why using the results of RCT studies alone to make claims of effectiveness is flawed and misleading, he lists the following problems:

• Patients’ problems are not solely defined by their “symptoms” or “diagnosis”, and so treatment cannot be determined without taking the person’s personalities, experiences and circumstances into consideration.

• There are limitations involved with outcome research using RCT studies.

• Therapy is essentially a personal relationship.

In fact, the better designed the research study using RCT (manualized scripts, frequent assessments, recordings of the process), the more distanced it can be from actual clinical work. This might seem obvious to practitioners of psychotherapy: an evidence-based medical model does not meet the demands and needs of evidence-based psychotherapy due to the complexities and interplay of personal and interpersonal variables in the actual practice of psychotherapy.

Marzillier (2004) states his position as follows:

"Evidence-based psychotherapy seems a reasonable aspiration. There are many weird and wonderful treatments in the field; surely it makes sense to know whether or not they work? [...] Yet want to argue that evidence-based psychotherapy is a myth.

I am not against scientific research in psychotherapy. On the contrary, I believe that scientific advances in psychology and related disciplines are important to the development of psychological therapies. But that is different from claiming that what psychotherapists do is, or should be, securely founded in evidence of effectiveness; that, for example, we can say with authority that depressed people are most likely to benefit from cognitive therapy or that research has shown that 90 per cent of people with panic attacks will recover with anxiety management. Such claims are in my view misleading and simplistic, and it is this ‘outcome research’ I have a problem with. It does justice neither to the complexity of people’s psychology nor to the intricacies of psychotherapy." (p.392)

The call and support for evidence-based psychotherapy practice (EBPP) made by the American Psychological Association (APA) in 2005 was to ensure that psychotherapy practices reflected clinical judgment, patients' values and preferences, and relevant scientific research ("APA Task Force on Evidence-Based Practice," 2006). By not limiting the evidence from relevant scientific research to the outcome studies using RCT studies alone, and by including the need to reflect clinical judgment (i.e., therapist expertise) and patient's values and preferences, EBPP seemed to take into consideration the concerns outlined by Marzillier (2004).

Research based on empirical studies shows that the quality of the therapeutic relationship is at least as important as techniques.

All of the above point to the need for EBPP to move away from the reliance on RCTs to scientific methods and studies more suited to psychotherapy (e.g., empirical research that includes quantitative and qualitative studies, practice-based evidence, and a focus on more person-centered outcome measures) that can balance the study and effect of interventions, as well as parameters of the therapeutic relationship, and the interplay between them. However, the field of psychotherapy continues to stay in the clutches of the medical model. The use of RCTs in which techniques dominate at the expense of personal and interpersonal variables prevails.

Why is that?

Evidence-Based Psychotherapy Practice (EBPP) vs. Empirically Supported Treatment (EST)

A treatment qualifies as an empirically supported treatment (EST) based on successfully replicated, randomized control trial (RCT) studies. Prior to the publication of the APA Task Force on Evidence-Based Practice (2006), the term evidence-based psychotherapy was used interchangeably with EST, as done by Marzillier (2004). The APA Task Force on Evidence-Based Practice (2006) has clearly delineated the differences between EST and EBPP:

“It is important to clarify the relation between EBPP and empirically supported treatments (ESTs). EBPP is a more comprehensive concept. ESTs start with a treatment and ask whether it works for a certain disorder or problem under specified circumstances. EBPP starts with the patient and asks what research evidence (including relevant results from RCTs) will assist the psychologist in achieving the best outcome. In addition, ESTs are specific psychological treatments that have been shown to be efficacious in controlled clinical trials, whereas EBPP encompasses a broader range of clinical activities (e.g., psychological assessment, case formulation, therapy relationships). As such, EBPP articulates a decision-making process for integrating multiple streams of research evidence—including but not limited to RCTs—into the intervention process.” (2006)

So what does this EBPP look like?

“How to improve psychotherapy outcome? Follow the evidence; follow what contributes to psychotherapy outcome. Begin by leveraging the patient's resources and self-healing capacities; emphasize the therapy relationship and so-called common factors; employ research-supported treatment methods; select interpersonally skilled and clinically motivated practitioners; and adapt all of them to the patient's characteristics, personality, and worldview. This, not simply matching a treatment method to a particular disorder, will maximize success." (Norcross and Lambert, 2011, p.13)

Proponents of mainly brief and manualized therapy models have appropriated the label "evidence-based" based solely on outcome research using RCTs. It is being widely asserted that these models are scientifically proven and superior to other forms of psychotherapy, which is a false narrative (Shedler, 2015). In other words, ESTs are claiming the term “evidence-based” therapy, when in reality, the term EBPP refers to a more comprehensive concept.

Although researchers and practitioners have questioned and challenged the use of RCTs in making such claims, and empirical research does not support these claims, these false narratives are being maintained and perpetuated by the proponents of these models. It is also being used to exclude other psychotherapy models that have a different value-base from being considered legitimate forms of therapy.

The False Narrative – An Example

I will use the policies of the healthcare company Lyra Health to illustrate the false narrative. Their selection criteria explicitly exclude psychotherapists who work from an attachment-focused orientation even though attachment theory models are based on decades of scientific research and study. An excerpt of their “evidence-based” claims is shown below:

"Unfortunately, accessing good care is shockingly difficult. Among therapists in health plan networks, just 10% are accepting new patients and exclusively using evidence-based techniques (treatments clinically proven to work). It’s hard enough finding someone to talk to and even harder finding a great therapist with the right expertise for your needs. It’s no surprise that six out of seven people do not receive effective care.

What makes this outcome particularly sad is that effective treatments exist. Short-term interventions like cognitive behavioural therapy (CBT) have been applied to a wide range of problems and are proven to reduce symptoms of depression, anxiety, stress, pain, and insomnia. With CBT, patients identify specific goals and then learn new skills, new ways of thinking, and new behaviours to improve their lives. CBT has been studied in hundreds of clinical trials and consistently delivers impressive results, usually after only 2-3 months. CBT, among other evidence-based therapies, including Acceptance and Commitment Therapy (ACT) and Dialectical Behavioral Therapy (DBT), and the Gottman Method, can be life-changing." (Ebersman, 2018, May 7)

The above marketing pitch is deconstructed as an example of simplistic and misleading “evidence-based” claims in Shedler (2015) and detailed below:

1. "Evidence-based techniques (treatments clinically proven to work)" and "CBT has been studied in hundreds of clinical trials and consistently delivers impressive results, usually after only 2-3 months." CBT interventions have been shown in RCT studies to be effective (albeit with very small gains) in the short-term. However, these results do not necessarily translate directly to practice in real-life. Using the results of RCT trials alone to state that a therapist who utilizes only CBT interventions (or interventions from other models that have been, or will be, added to this coveted "evidence-based" list) will be more effective is erroneous and problematic.

2. "Accessing good care is shockingly difficult" and "It’s no surprise that six out of seven people do not receive effective care" taken in its context implies "if it is not an "evidence-based” treatment, it is not good, i.e., therapies not in the "evidence-based" list are inadequate and ineffective.”

3. The statement "[it is] even harder finding a great therapist with the right expertise for your needs." stated in its context, equates clinical expertise with simply being skilled in certain “evidence-based” techniques. This conclusion marginalizes quality practitioners with clinical expertise, especially in specialized areas of focus, such as developmental trauma and dissociative disorders. The practice of psychotherapy is much more complex than simply following a list of techniques for a particular disorder or symptom list.

4. Making a claim of "exclusively using evidence-based techniques" is absurd in practice. Everything that is said or happens in therapy has to be considered an intervention or technique (e.g., sitting in silence with the client). That would imply "sitting in silence" has to be studied by an RCT study with different contexts/variables before an "evidence-based" practitioner can do so.

5. Few clinicians adhere to a puritan model or school of thought, and certainly do not adhere to a script. Most models employ common techniques (such as identifying false/negative beliefs) and all are based on the underlying principles of a healing therapeutic relationship. A study published in the Journal of Consulting and Psychology reports that only half of the clinicians claiming to use CBT use an approach that even approximates to CBT (Waller et al, 2011). This doesn't necessarily make such CBT therapists more or less effective than those who practised what was used in the RCT studies, but shows the problems of classifying an entire class of therapists as "evidence-based" or not, simply based on the model or label clinicians self-identify with.

6. A rich array of factors contributes to successful therapy (the patient, the therapist, the setting and the therapeutic relationship, to name a few) and not just the treatment method (Marzillier, 2014).

7. Diagnosis-specific interventions and ESTs are not equivalent to true EBPP. Critchfield and Knox (2010) discuss how therapists who are trained to simply implement ESTs have a relatively rigid skill set of limited applicability and lack the conceptual skills necessary for the practice of EBPP.

RCTs do have their place and value in psychotherapy research, especially in the study and fine-tuning of protocols. Manualized treatments and structured protocols are valuable when adapted to the specific patient and circumstances in practice. Techniques from the “evidence-based” list such as CBT and DBT interventions are good skill sets to add to a practitioner’s toolbox. The problems arise from the way these studies are being done and the sweeping conclusions that are being drawn from the research (Marzillier, 2014).

Claims made by outcome studies across the spectrum of different psychotherapy models (including person-centred) are suspect. Empirical research shows that most of these therapies are weak treatments. Their benefits are small, few patients get well, and even the small benefits do not last. Troubling research practices paint a misleading picture of the actual benefits of therapies, including systematic bias in research trials, sham control groups, cherry-picked patient samples, and suppression of negative findings (Shedler, 2015). Therefore, following rigid guidelines on specific effective treatments for different diagnoses based on these research studies as though the evidence is strong and conclusive is problematic (Marzillier, 2014).

Stuck in the World of Alternative Facts

After being repeated for nearly two decades, the narrative of "evidence-based" therapy has been ingrained in the minds of many influential non-practitioners of psychotherapy (especially medical practitioners, managed care and business leaders). Since the term "evidence-based" has been appropriated, it becomes difficult to point out the deficits of any of these treatments that have now been labelled "evidence-based" without being accused of being against science (Shedler, 2015). One of the main gains of having this "evidence-based" label is that it magically makes the therapy model "legitimate" and "scientifically proven", thereby providing easier access to funding for research and practice from managed care and healthcare companies.

Some practitioners, unable to get through to the proponents of these "evidence-based" practices to correct their false narratives, decided that the best way to survive was to join them. So every other model (e.g., DBT, ACT, EMDR) that wants the coveted "evidence-based" status may be forced to continue even meaningless RCT studies and therefore, knowingly, or unwittingly, keep the myth of the false narrative alive.

The result is that rather than trying to measure the effectiveness of psychotherapy actually being practised, we are being asked to practice psychotherapy that can be more easily replicated in RCT studies, to be conferred the valuable title of "evidence-based" therapy. This results in a net reduction in depth and quality of psychotherapy practice rather than an increase in fidelity to effective psychotherapy intervention (Edwards, 2018), and ultimately, is not in the best interests of the consumers of psychotherapy services.

Practitioners of these "evidence-based" psychotherapy models and others who have something to gain from the false narrative that has taken hold may decide to disregard all evidence to the contrary, including their own practice-based knowledge, for not wanting to forgo their short-term gains.

To make any systemic change, those that the broken system currently privileges and rewards (at least in the short term) need to wake up and join the movement for a call to change the current dysfunctional system. The first step would be to refrain from continuing the false narratives. The next step would be to call out discrepancies when someone makes misleading and false claims.

Polarization Into Two Camps

The artificial and unhealthy division of practitioners into two camps ("person-centred" aligned with the psychodynamic orientation vs. "techniques-centred" camps) has made it difficult to point out the deficiencies in our current “evidence-based” narratives. Criticizing the appropriation of the label "evidence-based" is often wrongly equated to:

• Opposing true evidence-based practice.

• Opposing the models that have been branded "evidence-based".

• Not wanting to be held to a specific standard when it comes to performance, outcomes and expectations, which then becomes part of the false narrative.

Those that identify with the “techniques-centred” approach are not necessarily being manipulative in supporting and propagating the myths of “evidence-based” therapies. They often simply have not questioned the truths of the claims, since it matches their own expectations of what makes therapy effective. We all have our prejudices and biases, and it is easy to adopt a stance that reinforces them.

The division into being techniques-centred or being person-centred is a false dichotomy. Technique and skill development is necessary, as well as developing the personal and interpersonal qualities of a clinician. Sometimes this truth is lost or forgotten in the heat of the either-or argument of tribalism (Godwin, 2018). However, a model of psychotherapy that sees techniques as just as one of many influences on outcome is better than one that privileges techniques, given that techniques have relatively less influence than the therapeutic relationship (Marzillier, 2014).

Disconnect Between Practitioners and Researchers

Policy makers and influencers who are making these decisions and conclusions that support ESTs exclusively are doing so without the input of practitioners and researchers in the field of psychotherapy. Some are being manipulated or misguided into making these misleading claims. Researchers who have published such claims are perhaps too removed from the actual practice of psychotherapy.

Most practising psychotherapists have not trained adequately or are not inclined to critically analyze or question the claims being made by researchers. So perhaps that is why such obvious flaws in the underlying claims of “evidence-based” models in psychotherapy have been allowed to continue for decades.

Conclusion

Let us say "no" to the current practice of marking an entire brand X of psychotherapy as "evidence-based" simply by relying on RCT outcome studies. ESTs are not equivalent to EBPP. Those who perpetuate unnecessary turf wars need to be called out. Practitioners and researchers across different psychotherapy models and theoretical orientations have to unify and work towards making progress with actual evidence-based practice in psychotherapy.

There is already a wealth of knowledge, experience, and innovative ideas on how to achieve this. With recent advances in technology, it is possible to analyze and leverage large amounts of data (quantitative as well as qualitative) using different research methods and utilizing feedback-informed treatments. This may allow us to move away from the traditional RCT based model and to investigate beyond our traditional symptom reduction outcomes to include person-c

entred outcomes.

Acknowledgements

I thank the authors referenced who responded graciously and timely to my queries. I am grateful to colleagues from around the world who contributed to discussions that shaped this article. Lastly, a special mention to Daryl Mahon for his comments highlighting the difference between ESTs and EBPP, that provided the necessary thread to weave a clear case.

Vinodha Joly, LMFT is a relational depth psychotherapist specializing in the treatment of complex trauma. She integrates structured protocols (e.g., EMDR reprocessing) into her work. Vinodha Joly worked as a computer engineer before transitioning to the field of psychology, and holds 14 patents. She received her Masters degree in Counseling Psychology from Santa Clara University, California and her Masters degree in Computer Engineering from the University of Michigan, Ann Arbor.

References

1. APA Task Force on Evidence-Based Practice. (2006). Evidence-based practice in psychology. American Psychologist, 61, 271–285. DOI: 10.1037/0003-066X.61.4.271.

2. Critchfield, K. L., & Knox, S. (2010). Conceptual skills needed for evidence-based practice of psychotherapy. Psychotherapy Bulletin. Retrieved from https :// societyforpsychotherapy . org / conceptual - skills - needed - for - evidence - based - practice - of - psychotherapy

3. Ebersman, D. (2018, May 7). New funding, same mission to transform mental health. Retrieved from https :// www . lyrahealth . com / blog / series - b - funding /

4. Edwards, B. G. (2018). The Empathor's New Clothes: When Person-Centered Practices and Evidence-Based Claims Collide in M. Bazzanno (Ed.) Re-visioning Person-Centred Therapy, 1st edition; Routledge.

5. Godwin, A. (2018). People Problems. 2018, September 7. Retrieved from https :// peopleproblems . org / september - 7 - 2018

6. Marzillier, J. (2004). The myth of evidence-based psychotherapy. The Psychologist, 17, [392-395]. Retrieved from https :// thepsychologist . bps . org . uk / volume - 17 / edition - 7 / myth - evidence - based - psychotherapy

7. Marzillier, J. (2014). Ch.4. The evaluation of trauma therapies in The Trauma Therapies, New York; Oxford University Press.

8. Norcross, J. C., & Lambert, M. J (2011). Ch.21. Evidence-Based Therapy Relationships, in J. C. Norcross (Ed.) Psychotherapy relationships that work, Evidence-Based Responsiveness, 2nd Edition, New York; Oxford University Press pp 3-21.

9. Shedler, J. (2015). Where is the evidence for “evidence-based” therapy? Journal of Psychological Therapies in Primary Care. 4:47–59. Retrieved from http :// jonathanshedler . com / wp - content / uploads / 2018 / 05 / Shedler - 2018 - Where - is - the - evidence - for - evidence - based - therapy . pdf

10. Waller, Glenn & Stringer, Hannah & Meyer, Caroline. (2011). What Cognitive Behavioral Techniques Do Therapists Report Using When Delivering Cognitive Behavioral Therapy for the Eating Disorders? Journal of consulting and clinical psychology. 80. 171-5. 10.1037/a0026559