Is ABA Evidence-Based?
Is ABA Evidence-Based?
Applied Behavior Analytic (ABA)treatments, such as Early Intensive Behavioral Intervention (EIBI) may be among the best examples of evidence-based behavioral health care. In contrast to some reports in the media, independent reviews consistently agree that ABA and EIBI treatments for autism are effective, and that the extensive body of research meets high standards of evidence. For further reading:
Are Applied Behavior Analysis (ABA) and Early Intensive Behavioral Intervention (EIBI) Effective, Medically Necessary Treatments for Autism? A Cumulative History of Impartial Independent Reviews by Eric V. Larsson, PhD, LP, BCBA-D, of the The Lovaas Institute for Early Intervention Midwest Headquarters
FAQs
Is there scientific evidence that ABA works?
Haven’t some reviews concluded that ABA doesn’t work?
What factors are important to consider in evaluating complex ABA interventions?
Are there studies on PECS?
Do studies with single subject experimental designs “count” as evidence?
Is there scientific evidence that ABA works?
We list 45 independent and systematic reviews and meta-analyses of research on ABA interventions in a bibliography. Every review cites the obvious positive results of ABA and EIBI and notes that such results have been replicated in many studies.
Haven’t some reviews concluded that ABA doesn’t work?
No. In none of the 45 reviews in our bibliography [link] do the authors systematically refute the published evidence for ABA treatments of autism. The reviews are critical evaluations – in many cases, other non-ABA treatments are assigned to categories such as “insufficient evidence,” “unproven,” or even “potentially harmful.” Yet the most “negative” conclusions that are offered are:
- ABA does not cure all children of autism
- ABA has not been compared to other treatments
- Research has not yet identified who benefits most from ABA intervention.
However, it should be noted that the above conclusions can be drawn about any treatment.
More Information
However, the lay impression persists that there are “negative” reviews in the literature. But let’s look at what the “negative” reviews do say. The following is one of the most skeptical recent publications in the scientific literature. But see one of their concluding statements.
“There is little question now that early intensive behavioral intervention is highly effective for some children. However, gains are not universal, and some children make only modest progress while others show little or no change, sometimes after extremely lengthy periods in treatment.” (page 36).
Howlin, P., Magiati, I., & Charman, T. (2009). Systematic review of early intensive behavioral interventions for children with autism. American Journal on Intellectual and Developmental Disabilities. 114. 23-41. (The authors are professors at the Institute of Psychiatry, King’s College (London, UK) and University College, London, Institute of Child Health).
Other “negative” reviews may exclude the majority of ABA research, by applying highly restrictive criteria for what qualifies as evidence.
For example, there is the Comparative Effectiveness Review published by the AHRQ in 2011. But, while this report has also been cited as “negative,” see their main conclusions regarding ABA and EIBI interventions.
“Evidence supports early intensive behavioral and developmental intervention, including the University of California, Los Angeles (UCLA)/Lovaas model and Early Start Denver Model (ESDM) for improving cognitive performance, language skills, and adaptive behavior in some groups of children.” (page vi).
“Evidence suggests that interventions focusing on providing parent training and cognitive behavioral therapy (CBT) for bolstering social skills and managing challenging behaviors may be useful for children with ASDs to improve social communication, language use, and potentially, symptom severity.” (page vi).
The “negative” qualifiers of these conclusions are stated as:
“All of these studies need to be replicated, and specific focus is needed to characterize which children are most likely to benefit.” (page vi).
“Information is lacking on modifiers of effectiveness, generalization of effects outside the treatment context, components of multicomponent therapies that drive effectiveness, and predictors of treatment success.” (page vi).
In comparison to the above comments, these are the clearly negative conclusions about traditional biomedical treatments that are currently widely covered by insurance policies:
“No current medical interventions demonstrate clear benefit for social or communication symptoms in ASDs.” (page vi).
“Little evidence is available to assess other behavioral interventions, allied health therapies, or complementary and alternative medicine.” (page vi).
Warren, Z., Veenstra-VanderWeele, J., Stone, W., Bruzek, J.L., Nahmias, A.S., Foss-Feig, J.H., Jerome, R.N., Krishnaswami, S., Sathe, N.A., Glasser, A.M., Surawicz, T., & McPheeters, M.L. (April, 2011). Therapies for Children With Autism Spectrum Disorders. Comparative Effectiveness Review No. 26. (Prepared by the Vanderbilt Evidence-based Practice Center under Contract No.290-2007-10065-I.) AHRQ Publication No. 11-EHC029-EF. Rockville, MD:Agency for Healthcare Research and Quality. Available at: www.effectivehealthcare.ahrq.gov/reports/final.cfm.
The AHRQ report reached these positive conclusions about ABA and EIBI despite excluding a large number of studies, including all studies published prior to 2000. Yet the AHRQ report still found 78 studies of behavioral interventions, which included 34 studies of EIBI that met their criteria for inclusion.
Other “negative” reviews cited are typically proprietary reports published privately. For example, the Kaiser Blue Cross report did not offer positive statements (Rothenberg & Samson, 2009). However in their methodology, they limited their analysis to only 16 studies, out of the hundreds available, and concluded that more research needs to be done. Interestingly, unlike the AHRQ review, this report did not comment on the comparable lack of data for psychotropic medications, yet insurance companies readily cover such treatment.
What factors are important to consider in evaluating complex ABA interventions?
One review that offers a useful framework is by Odom, Boyd, Hall and Hume (2010). These authors examined 30 comprehensive treatment models (CTMs). Four of the five CTMs judged to have the strongest scientific evidence were based on applied behavior analysis. Twenty of the 30 models reviewed were identified as behavioral.
Is ABA Evidence-Based Note 2
Odom et al’s evaluation of comprehensive treatment models offers readers a summary of a number of essential components of 30 documented comprehensive treatment models. A number of important factors were considered and rated. These factors included (1) operationalization of the model, (2) replication of the model, and (3) type of empirical evidence supporting the model.
Odom et al.’s (2010) review also illustrated some of the challenges in evaluating comprehensive models. While efforts were made to secure as much information as possible about each model, a good deal of information was obtained via a phone interview and by reviewing information requested of and supplied by the treatment model. As the authors note, it is possible information was not properly conveyed, and as a result some models may have been rated differently depending on the information that was provided to the interviewer. It is also important to note that on-site visits of the implementation of the models were not conducted by the authors. Thus the authors did not empirically evaluate actual implementation of the model.
Although not perfect, the review is a prompt to examine data and an attempt to organize current information. The call to develop measures that can document the appropriate implementation of CTM procedures and to research the efficacy of models is valuable.
Are there studies on PECS?
There has been a steady growth of PECS-related research over the years, a trend that has markedly increased of late. Over 100 publications, including book chapters and literature reviews concerning PECS have been published, of which over 60 involve case studies or other data-based work. There are now at least six reviews of the literature, each leading to somewhat different emphases, depending in part upon which publications have been included. However, they all agree that studies provide evidence that PECS is effective.
Is ABA Evidence-Based Note 3
The publications on PECS have an increasing international flavor as publications reflect the involvement of authors from 15 countries (see Sulzer-Azaroff, Hoffman, Horton, Bondy & Frost, 2009 for examples in a review of single-subject designed publications, Hart & Banda (2010) noted, “In summary, PECS may increase manding, social communicative behavior, and speech and decrease problem behaviors (p. 486).” Another review by Tien (2008) concluded, “Taken as a whole, therefore, results of the studies reviewed provide evidence for the effectiveness of PECS; specifically, PECS is effective in enhancing functional communication skills of individuals with ASD. Therefore, PECS is recommended as an evidence-based intervention for this purpose (p. 74).” Tincani & Devis (2011) wrote, “The findings of this meta-analysis support the PECS as an effective intervention to promote functional communication for individuals with ASD and other disabilities, (p. 9).” Other reviews have been more conservative in their overall assessment: “With one group design of strong quality and seven single subject experiments of at least adequate quality documenting gains in communication following PECS training, the body of evidence for the PECS approach demonstrates that PECS is a promising, although not yet established, evidence-based practice for promoting communication in children with autism (Flippin, Reszka, & Watson, 2010, p. 189).” Finally, Tincani & Devis (2011) also stressed the importance of research regarding the efficacy of the more advanced phases of the PECS protocol. [Recently, the National Professional Development Center on Autism Spectrum Disorders noted PECS as one of its ‘evidence-based practices’; see http://autismpdc.fpg.unc.edu/content/briefs retrieved Feb. 2012.]
Although studies generally support the use of PECS, there are some major problems with interpreting the studies. ). Most of the current review papers call for more detailed research regarding how PECS relates to overall functional communication, speech development, behavior management and social skill modifications.
Some publications have claimed to represent PECS, but have either clearly not followed the protocol or provided no evidence that the protocol was followed in a consistent manner1. Also, some have only examined a portion of the protocol (see Sulzer-Azaroff, et al., 2009 for a complete review of this issue). For example, some studies have looked at effects only through discrimination (i.e., Phase III), not because there were documented problems in getting users to achieve Phase IV and beyond or because their use of the early Phases of PECS lead to the rapid acquisition of alternative forms of functional communication (including speech) but for other, non-specified reasons (including, perhaps, pressure to publish). These incomplete attempts to teach the full protocol when attempting to compare “PECS” to some other system or modality make the comparisons difficult to interpret (see Sulzer-Azaroff et al., 2009 for a full review of these and other quality-assurance issues). When only a portion of the PECS protocol is used, it is difficult to understand the full meaning of a questions comparing “PECS versus X” This issue is especially relevant when production of speech is of concern because several studies which suggest that changes during Phase IV and its associated delayed-prompt strategy for speech production are associated with a significant boost in vocalization across the PECS phases (see Tincani, Crozier & Alazetta, 2006; Ganz & Simpson, 2004). Of interest is some preliminary evidence offered at a recent ABAI paper presentation that suggested shifting the use of the delayed-prompt strategy from Phase IV to Phase II may result in a concomitant shift in increased vocalizations (Rapoza-Houle & Muehlberger, 2010). When attempting to compare two different ‘systems’ it may be impossible to separate issues uniquely associated with the modality from the teaching strategies associated with use of that modality. That is, a teaching protocol distinct from that noted within PECS could be developed to teach the use of pictures within a manding function that results in vastly different outcomes than when using the current PECS protocol. That is, poor teaching strategies could be associated with the failure of any modality. However, evidence indicates that, when implemented properly.
While seemingly simple, the question, “Does PECS work?” may involve many facets2. For example, this question may focus on the acquisition of the use of pictures within the protocol described. On the other hand, it could be interpreted as meaning, “Did the user acquire speech?” or “How did the use of PECS impact on a broad array of other behaviors, from social orientation to behavior management?” In terms of acquiring the use of pictures alone, Lancioni et al. (2007) noted only three ‘failures’ of the 173 reported PECS users. With regard to the impact upon speech, one factor may be developmentally sensitive, involving the range of ages during which PECS and speech production are assessed. For example, in a well controlled random group assignment study by Howlin, Gordon, Pasco, Wade and Charman (2007) the mean age of the target group was 6.8 years and two ‘standardized assessments’ were used to evaluate broad outcome changes. One limitation of this type of assessment relates to whether there are any interventions which have demonstrated substantial changes in speech for children of this age (especially for children who display no vocal production); therefore, the fact that six months of teacher training on how to implement PECS did not change such measures should not be surprising nor suggest significant limitations of this modality. Furthermore, of the two standardized tests, the one involving expressive communication targeted tacting (naming) a series of pictures while the other test involved having children respond to spoken words by pointing to various pictures. Those students who achieved completion of Phase VI could have produced a small tacting repertoire relating to the stimuli used but it would be remarkable for any intervention to result in substantial generalization to receptive skills within 6 months of introduction. Of direct concern is the assessment of skills that did change as a function of PECS training with this population over this period of time. Skills taught directly by the PECS protocol (i.e., initiation and use of pictures) did in fact improve (although the study did not measure the number of pictures used per student.
Do studies with single subject experimental designs “count” as evidence?
Studies with single subject experimental designs (SSEDs) involve conducting repeated observations to compare an individual’s behavior during a baseline period when the individual receives no intervention to the behavior in one or more intervention phases. Some reviewers exclude SSED studies from their evaluation of the evidence for ABA interventions. They argue that, although SSED studies useful only for initial demonstrations that an intervention procedure might be useful, studies with larger numbers of participants are needed to identify evidence-based practices. However, SSED studies are important because they provide a powerful test of whether introducing an intervention reliably leads to a change in behavior.
Is ABA Evidence-Based Note 4
1. Professionals who are not trained in the scientific method as it applies to individuals are trained instead in studies that examine the effects of treatments on large groups of individuals, typically following the logic i shown in Figure 1. In this figure the goal is to make a decision about which of two drugs should be regarded as most effective. The drug is given to individuals in each group, the results averaged, and then the group average that shows the best improvement is the one that is regarded as the treatment of choice. To ensure that there is no bias in terms of the makeup of the groups, individual allocation to each group is randomized. 
Figure 1: Panel a — In an idealized group design, a treatment drug (Left-hand side) is given to a large number of participants in which variability between them is minimal. The control drug is given to a similar group of participants (Right-hand side). Panel b — Because an idealized group does not exist, participants for each group are randomly assigned to minimize the effects of extraneous variables that unexpectedly contribute to any differences between group results (Randomized Control Trial). Panel c — Variability in individual responses that arise because of the effects of the drug, and/or variability in individual histories are represented by the spectrum.
2. However, ABA emphasizes the careful study of each individual who receives intervention. Figure 3 (based on Reese, 1978) gives an overview of how the scientific method is used in the study of individuals. As you can see, the overall goal is to adapt known principles of behavior (laws of learning) to the needs of the individual and to continuously monitor changes in the behavior of the individual to maximize the benefit to that individual. This approach is fundamental to getting it right for the individual, whatever length of time it takes. And because adjustments are tailor-made for each individual, the opportunities for learning are maximized for each individual. How far each individual can progress depends on many factors, but focusing on the individual in such a fine-grained way, using known principles of behavior, is to be preferred to developing an intervention that is based on a group average and that is insensitive to the needs of the individual. 
Figure 2: The scientific method used in ABA provides a clear alternative to scientific method associated with RCTs when it comes to developing tailor-made procedures for producing and monitoring changes in an individual.
In conclusion, parents need to be aware of how conflict in scientific traditions affects policy decisions. When the evidence for using single-case designs is properly understood, a science that uses it can be seen to offer parents the best hope for their children (Green, 2008).
Green, G. (2008). Single-case research methods for evaluating treatments for autism spectrum disorders. Opportunities… Hope… Potential… The future of Pennsylvania, Autism in Pennsylvania: What lies ahead, 8, 69-81.
Keenan, M. & Dillenburger, K. (2011). When all you have is a hammer…: RCTs and hegemony in science. Research in Autism Spectrum Disorders, 5, 1-13.
Reese, E. (1978). Human operant behavior: Analysis and application. Brown & Benchmark.