Categories for Decision Making

Approaches to Increasing Treatment Integrity (Wing Institute Paper)

October 4, 2017

Student achievement scores in the United States remain stagnant despite constant reform. New initiatives arise promising hope, only to disappoint after being adopted, implemented, and quickly found wanting. The cycle of reform followed by failure has had a demoralizing effect on schools, making new reform efforts more problematic. These efforts frequently fail because implementing new practices is far more challenging than expected and require that greater attention be paid to implementation. A fundamental factor leading to failure is inattention to treatment integrity. When innovations are not implemented as designed, it should not be a surprise that anticipated benefits are not forthcoming. The question is, what strategies can educators employ to increase the likelihood that practices will be implemented as designed?

Strategies designed to increase treatment integrity fall into two categories: antecedent-based strategies and consequence-based strategies. Antecedent-based strategies involve any setting event or environmental factor that happens prior to implementing the new practice and that increases the likelihood of success as well as eliminates setting events or environmental considerations that decrease the likelihood of success. Consequence-based strategies are designed to impact actions that happen after implementation of the new practice and that are likely to increase or decrease treatment integrity.

Citation: Detrich, R., States, J. & Keyworth, R. (2017). Approaches to Increasing Treatment Integrity. Oakland, Ca. The Wing Institute




Dimensions of Treatment Integrity Overview (Wing Institute Paper)

October 4, 2017

Historically, treatment integrity has been defined as implementation of an intervention as planned (Gresham, 1989). More recently, treatment integrity has been reimagined as multidimensional (Dane & Schneider, 1998). In this conceptualization of treatment integrity are four dimensions relevant to practice: (a) exposure (dosage), (b) adherence, (c) quality of delivery, and (d) student responsiveness.  It is important to understand that these dimensions do not stand alone but rather interact to impact the ultimate effectiveness of an intervention. It is important for educators to assess all dimensions of treatment integrity to assure that it is being implemented as intended.

Citation: Detrich, R., States, J. & Keyworth, R. (2017). Dimensions of Treatment Integrity Overview. Oakland, Ca. The Wing Institute




Overview of Treatment Integrity (Wing Institute paper)

October 4, 2017

For the best chance of a positive impact on educational outcomes, two conditions must be met: (a) Effective interventions must be adopted, and (b) those interventions must be implemented with sufficient quality (treatment integrity) to ensure benefit.  To date, emphasis in education has been on identifying effective interventions and less concern with implementing the interventions. The research on the implementation of interventions is not encouraging. Often, treatment integrity scores are very low and, in practice, implementation is rarely assessed. If an intervention with a strong research base is not implemented with a high level of treatment integrity, then the students do not actually experience the intervention and there is no reason to assume they will benefit from it. Under these circumstances, it is not possible to know if poor outcomes are the result of an ineffective intervention or poor implementation of that intervention. Historically, treatment integrity has been defined as implementing an intervention as prescribed. More recently, it has been conceptualized as having multiple dimensions, among them dosage and adherence which must be measured to ensure that it is occurring at adequate levels.

Citation: Detrich, R., States, J., & Keyworth, R. (2107). Overview of Treatment Integrity. Oakland, Ca. The Wing Institute.




Predatory Journals: How do you to know whom to trust?

September 6, 2017

Beall’s List of Predatory Journals and Publishers

This news item offers a list of questionable, scholarly open-access publishers. In an era in which we are bombarded with volumes of research, it becomes ever more challenging to decide which journals and publishers are reputable. This web site reviews, assesses, and provides guidelines on how to decide which are trustworthy, whether you want to submit articles, serve as an editor, or serve on an editorial board. The web site provides a list that mostly consists of open access journals, although, a few non-open access publishers whose practices match those of predatory publishers have been added to the list.

Citation: Beall, J. (2012). Predatory publishers are corrupting open access. Nature, 489(7415), 179.

Web Site: Beall’s List of Predatory Journals and Publishers



How to Use Expenditure-to-Performance Ratios to Boost Student Achievement Cost Effectively

August 1, 2017

A Guide to Calculating District Expenditure-to-Performance Ratios Using Publicly Available Data

Efficient use of educational resources is a perennial challenge for school systems. Maximizing the impact of education interventions to magnify student achievement is an important goal for all school districts. This guide examines the use of expenditure-to-performance ratios as a critical indicator for school systems to help decide which interventions make sense when education dollars are at a premium. It describes how states and districts can use available data on district expenditures and student academic achievement to calculate six district-level expenditure-to-performance ratios.

Citation: Ryan, S., Lavigne, H. J., Zweig, J. S., & Buffington, P. J. (2017). A guide to calculating district expenditure-to-performance ratios using publicly available data. (REL 2017-179). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Northeast & Islands.




Formative Assessment Overview (Wing Institute Paper)

June 13, 2017

Effective ongoing assessment, referred to in the education literature as formative assessment or progress monitoring, is indispensable in promoting teacher and student success. Feedback through formative assessment is ranked at or near the top of practices known to significantly raise student achievement. For decades, formative assessment has been found to be effective in clinical settings and, more important, in typical classroom settings. Formative assessment produces substantial results at a cost significantly below that of other popular school reform initiatives such as smaller class size, charter schools, accountability, and school vouchers. It also serves as a practical diagnostic tool available to all teachers. A core component of formal and informal assessment procedures, formative assessment allows teachers to quickly determine if individual students are progressing at acceptable rates and provides insight into where and how to modify and adapt lessons, with the goal of making sure that students do not fall behind.

Citation: States, J., Detrich, R. & Keyworth, R. (2017). Overview of Formative Assessment. Oakland, CA: The Wing Institute.




The Importance of Data Visualization in Decision Making

March 1, 2017

Sage Spotlight on Data Visualization

The February issue of Sage Publishing’s newsletter, Sage Methods Minute, presents useful guidance on understanding and managing data visualization in making effective decisions. The newsletter offers a lecture, interview, and webinar on this important but often neglected topic. Productive data-based decisions rely on the effective use of analytics and the acquisition, interpretation, and communication of meaningful patterns in data. In an increasingly complicated world in which vast quantities of data are available, it is essential that educators become astute in presenting data adapted to different audiences and in identifying deceptive data so they are able to make wise decisions in the service of educating children. The Sage Spotlight newsletter on visualization includes Tailoring Data Visualization to Reach Different Audiences by Tom Schenk; Textbooks in Data Visualization: 60 Seconds with Andy Kirk; and Webinar: Learn the Essentials of Data Visualization by Andy Kirk and Stephanie Evergreen. For those interested in additional resources on this topic, the works of Edward Tufte, professor emeritus of political science, statistics, and computer science at Yale University, and Howard Wainer, adjunct professor of statistics at the Wharton School of the University of Pennsylvania, provide insight in how to deliver information that communicates your message.

Sage February Newsletter:

Edward Tufte:

Howard Wainer:



New Evidence-based Web Site for ESSA

February 15, 2017

The Center for Research and Reform in Education at Johns Hopkins University has announced a February release for a website that reviews every math and reading program for grades K to 12 to determine which meet the strong, moderate, or promising levels of evidence defined in the Every Student Succeeds Act (ESSA). This web site is designed to provide education decision-makers at the state, district and school levels, teachers, parents, and the public with the information to ascertain which programs meet the ESSA evidence standards.

Additional commentary on this topic is available from Robert Slavin



How Not To Be Ignorant About The World

December 18, 2014

Hans Rosling, a Swedish medical doctor, academic, statistician and public speaker, has a particular talent for presenting data in a way that effectively tells a story that links critical issues in economic development, agriculture, poverty, Read More…



Examining Test Based Accountability

December 11, 2014

This opinion piece by Robert Slavin questions the effectiveness of how test-based accountability is currently been employed in the United States. Slavin doesn’t question the use of high stakes testing as a valuable tool for identification of poorly performing students and schools. Read More…