Treatment Integrity

All Wing Resources

Publications

TITLE
SYNOPSIS
CITATION
Approaches to Increasing Treatment Integrity

Strategies designed to increase treatment integrity fall into two categories: antecedent-based strategies and consequence-based strategies.

Detrich, R., States, J. & Keyworth, R. (2017). Approaches to Increasing Treatment Integrity. Oakland, Ca. The Wing Institute

 

Dimensions of Treatment Integrity Overview

Historically, treatment integrity has been defined as implementation of an intervention as planned (Gresham, 1989). More recently, treatment integrity has been reimagined as multidimensional (Dane & Schneider, 1998). In this conceptualization of treatment integrity are four dimensions relevant to practice: (a) exposure (dosage), (b) adherence, (c) quality of delivery, and (d) student responsiveness.  It is important to understand that these dimensions do not stand alone but rather interact to impact the ultimate effectiveness of an intervention. It is important for educators to assess all dimensions of treatment integrity to assure that it is being implemented as intended.

Detrich, R., States, J. & Keyworth, R. (2017). Dimensions of Treatment Integrity Overview. Oakland, Ca. The Wing Institute

 

Innovation, Implementation Science, and Data-Based Decision Making: Components of Successful Reform

Schools are often expected to implement innovative instructional programs.  Most often these initiatives fail because what we know from implementation science is not considered as part of implementing the initiative.  This chapter reviews the contributions implementation science can make for improving outcomes for students.

Detrich, R. Innovation, Implementation Science, and Data-Based Decision Making: Components of Successful Reform. Handbook on Innovations in Learning, 31.

Overview of Treatment Integrity

For the best chance of a positive impact on educational outcomes, two conditions must be met: (a) Effective interventions must be adopted, and (b) those interventions must be implemented with sufficient quality (treatment integrity) to ensure benefit.  To date, emphasis in education has been on identifying effective interventions and less concern with implementing the interventions. The research on the implementation of interventions is not encouraging. Often, treatment integrity scores are very low and, in practice, implementation is rarely assessed. If an intervention with a strong research base is not implemented with a high level of treatment integrity, then the students do not actually experience the intervention and there is no reason to assume they will benefit from it. Under these circumstances, it is not possible to know if poor outcomes are the result of an ineffective intervention or poor implementation of that intervention. Historically, treatment integrity has been defined as implementing an intervention as prescribed. More recently, it has been conceptualized as having multiple dimensions, among them dosage and adherence which must be measured to ensure that it is occurring at adequate levels.

Detrich, R., States, J., & Keyworth, R. (2107). Overview of Treatment Integrity. Oakland, Ca. The Wing Institute.

Roles and responsibilities of researchers and practitioners for translating research to practice

This paper outlines the best practices for researchers and practitioners translating research to practice as well as recommendations for improving the process.

Shriver, M. D. (2007). Roles and responsibilities of researchers and practitioners for translating research to practice. Journal of Evidence-Based Practices for Schools, 8(1), 1-30.

Sustainability of evidence-based programs in education

This paper discusses common elements of successfully sustaining effective practices across a variety of disciplines.

Fixsen, D. L., Blase, K. A., Duda, M., Naoom, S. F., & Van Dyke, M. (2010). Sustainability of evidence-based programs in education. Journal of Evidence-Based Practices for Schools, 11(1), 30-46.

Treatment Integrity in the Problem Solving Process

The usual approach to determining if an intervention is effective for a student is to review student outcome data; however, this is only part of the task. Student data can only be understood if we know something about how well the intervention was implemented. Student data without treatment integrity data are largely meaningless because without knowing how well an intervention has been implemented, no judgments can be made about the effectiveness of the intervention. Poor outcomes can be a function of an ineffective intervention or poor implementation of the intervention. Without treatment integrity data, the is a risk that an intervention will be judged as ineffective when, in fact, the quality of implementation was so inadequate that it would be unreasonable to expect positive outcomes.

Detrich, R., States, J. & Keyworth, R. (2017). Treatment Integrity in the Problem Solving Process. Oakland, Ca. The Wing Institute.

 

Treatment Integrity in the Problem-Solving Process Overview

Treatment integrity is a core component of data-based decision making (Detrich, 2013). The usual approach is to consider student data when making decisions about an intervention; however, if there are no data about how well the intervention was implemented, then meaningful judgments cannot be made about effectiveness.

Treatment Integrity Strategies Overview

Student achievement scores in the United States remain stagnant despite repeated attempts to reform the education system. New initiatives promising hope arise, only to disappoint after being adopted, implemented, and quickly found wanting. The cycle of reform followed by failure has had a demoralizing effect on schools, making new reform efforts problematic. These efforts frequently fail because implementing new practices is far more challenging than expected and require that greater attention be paid to how initiatives are implemented. Treatment integrity is increasingly recognized as an essential component of effective implementation in an evidence-based education model that produces results, and inattention to treatment integrity is seen as a primary reason new initiatives fail. The question remains, what strategies can educators employ to increase the likelihood that practices are implemented as designed? The Wing Institute overview on the topic of Treatment Integrity Strategies examines the essential practice elements indispensable for maximizing treatment integrity.

 

States, J., Detrich, R. & Keyworth, R. (2017). Overview of Treatment Integrity Strategies. Oakland, CA: The Wing Institute. http://www.winginstitute.org/effective-instruction-treatment-integrity-strategies.

Treatment Integrity: A Fundamental Unit of Sustainable Educational Programs.

Reform efforts tend to come and go very quickly in education. This paper makes the argument that the sustainability of programs is closely related to how well those programs are implemented.

Detrich, R., Keyworth, R. & States, J. (2010). Treatment Integrity: A Fundamental Unit of Sustainable Educational Programs. Journal of Evidence-Based Practices for Schools, 11(1), 4-29.

Treatment Integrity: Fundamental to Education Reform

To produce better outcomes for students two things are necessary: (1) effective, scientifically supported interventions (2) those interventions implemented with high integrity.  Typically, much greater attention has been given to identifying effective practices.  This review focuses on features of high quality implementation.

Detrich, R. (2014). Treatment integrity: Fundamental to education reform. Journal of Cognitive Education and Psychology, 13(2), 258-271.

Why Education Practices Fail?

This paper examines a range of education failures: common mistakes in how new practices are selected, implemented, and monitored. The goal is not a comprehensive listing of all education failures but rather to provide education stakeholders with an understanding of the importance of vigilance when implementing new practices.

States, J., & Keyworth, R. (2020). Why Practices Fail. Oakland, CA: The Wing Institute. https://www.winginstitute.org/roadmap-overview

 

Data Mining

TITLE
SYNOPSIS
CITATION
How does performance feedback affect the way teachers carry out interventions?
This analysis examined the impact of performance feedback on the quality of implementation of interventions.
Detrich, R. (2015). How does performance feedback affect the way teachers carry out interventions? Retrieved from how-does-performance-feedback.
How long do reform initiatives last in public schools?
This analysis examined sustainability of education reforms in education.
Detrich, R. (2015). How long do reform initiatives last in public schools? Retrieved from how-long-do-reform.
How often are treatment integrity measures reported in published research?
This analysis examined the frequency that treatment integrity is reported in studies of research-based interventions.
Detrich, R. (2015). How often are treatment integrity measures reported in published research? Retrieved from how-often-are-treatment.

 

Presentations

TITLE
SYNOPSIS
CITATION
Evolution of the Revolution: How Can Evidence-based Practice Work in the Real World?
This paper provides an overview of the considerations when introducing evidence-based services into established mental health systems.
Chorpita, B. (2008). Evolution of the Revolution: How Can Evidence-based Practice Work in the Real World? [Powerpoint Slides]. Retrieved from 2008-wing-presentation-bruce-chorpita.
Roles and Responsibilities of Researchers and Practitioners Translating Research to Practice

This paper outlines the best practices for researchers and practitioners translating research to practice as well as recommendations for improving the process.

Shriver, M. (2006). Roles and Responsibilities of Researchers and Practitioners Translating Research to Practice [Powerpoint Slides]. Retrieved from 2006-wing-presentation-mark-shriver.

The Four Assumptions of the Apocalypse
This paper examines the four basic assumptions for effective data-based decision making in education and offers strategies for addressing problem areas.
Detrich, R. (2009). The Four Assumptions of the Apocalypse [Powerpoint Slides]. Retrieved from 2009-wing-presentation-ronnie-detrich.
The Role of Data-based Decision Making

This paper identifies the critical features required for developing and maintaining a systemic data-based decision making model aligned at all levels, beginning with an individual student in the classroom and culminating with policy makers.

States, J. (2009). The Role of Data-based Decision Making [Powerpoint Slides]. Retrieved from 2009-campbell-presentation-jack-states.

The Role of Data-based Decision Making

This presentation examines the impact of data-based decision making in schools. It looks at the critical role data plays in building an evidence-based model for education that relies on monitoring for the reliable implementation of practices.

States, J. (2010). The Role of Data-based Decision Making [Powerpoint Slides]. Retrieved from 2010-hice-presentation-jack-states.

Toward a Technology of Treatment Integrity
If research supported interventions are to be effective it is necessary that they are implemented with integrity. This paper describes approahes to assuring high levels of treatment integrtiy.
Detrich, R. (2011). Toward a Technology of Treatment Integrity [Powerpoint Slides]. Retrieved from 2011-apbs-presentation-ronnie-detrich.
Treatment Integrity and Program Fidelity: Necessary but Not Sufficient to Sustain Programs
If programs are to sustain they must be implemented with integrity. If there is drift over time, it raises questions about whether the program is sustaining or has been substantially changed.
Detrich, R. (2008). Treatment Integrity and Program Fidelity: Necessary but Not Sufficient to Sustain Programs [Powerpoint Slides]. Retrieved from 2008-aba-presentation-ronnie-detrich.
Treatment Integrity: Necessary by Not Sufficient for Improving Outcomes
Treatment integrity is necessary to improve outcomes but it is not sufficient. It is also necessary to implement scientifically supported interventions.
Detrich, R. (2015). Treatment Integrity: Necessary by Not Sufficient for Improving Outcomes [Powerpoint Slides]. Retrieved from 2015-ebpindisabilities-txint-presentation-ronnie-detrich.
What the Data Tell Us

This paper offers an overview of issues practitioners must consider in selecting practices. Types of evidence, sources of evidence, and the role of professional judgment are discussed as cornerstones of effective evidenced-based decision-making.

States, J. (2010). What the Data Tell Us [Powerpoint Slides]. Retrieved from 2010-capses-presentation-jack-states.

 

Student Research

TITLE
SYNOPSIS
CITATION
Effects of a problem solving team intervention on the problem-solving process: Improving concept knowledge, implementation integrity, and student outcomes.

This study evaluated the effects of a problem solving intervention package that included problem-solving information, performance feedback, and coaching in a student intervention planning protocol.

Vaccarello, C. A. (2011). Effects of a problem solving team intervention on the problem-solving process: Improving concept knowledge, implementation integrity, and student outcomes. Retrieved from student-research-2011.

Transporting an evidence-based school engagement intervention to practice: Outcomes and barriers to implementation.
Check and Connect is an intervention designed to increase student engagement in school. This study was a transportability study that evaluated the impact of Check and Connect when implemented by school personnel rather than researchers.
Pankow, C. (2009). Transporting an evidence-based school engagement intervention to practice: Outcomes and barriers to implementation. Retrieved from student-research-2009-a.
Back to Top