Categories for Implementation
January 22, 2019
Barriers to Implementing Classroom Management and Behavior Support Plans: An Exploratory Investigation. Ample evidence supports effective classroom management’s place in maximizing student achievement. Unfortunately, sustained implementation of classroom management strategies too often fail. This study examines obstacles encountered by 33 educators along with suggested interventions to overcome impediments to effective delivery of classroom management interventions or behavior support plans. Having the right classroom management plan isn’t enough if you can’t deliver the strategies to the students in the classroom.
Citation: Collier‐Meek, M. A., Sanetti, L. M., & Boyle, A. M. (2019). Barriers to implementing classroom management and behavior support plans: An exploratory investigation. Psychology in the Schools, 56(1), 5-17.
Link: https://onlinelibrary.wiley.com/doi/pdf/10.1002/pits.22127
December 5, 2018
Accountability policies and teacher decision making: Barriers to the use of data to improve practice. Underlying many accountability policies is the assumption that standardized test data and other common sources of data will be used to make decisions that will result in changes to instructional practices. This study examines longitudinal from nine high schools nominated as leading practitioners of Continuous Improvement (CI) practices. The researchers compared continuous improvement best practices to teachers actual use of data in making decisions. The study found teachers to be receptive, but also found that significant obstacles were interfering with the effective use of data that resulted in changes in instruction. The analysis showed cultural values and practices inconsistent with accountability policies and continuous improvement practices impede implementation. The researchers identify barriers to use of testing and other data that help to account for the less than successful results. Given the current understanding of the importance on implementation science in the effective application of any new practice, these findings are not a surprise. As our colleague, Ronnie Detrich, is quoted as saying, “Implementation is where great ideas go to die”.
Citation: Ingram, D., Louis, K. S., & Schroeder, R. G. (2004). Accountability policies and teacher decision making: Barriers to the use of data to improve practice. Teachers College Record, 106(6), 1258-1287.
Link: Accountability policies and teacher decision making: Barriers to the use of data to improve practice
December 5, 2018
A Review of Schoolwide Positive Behavior Interventions and Supports as a Framework for Reducing Disciplinary Exclusions. Schoolwide positive behavior interventions and supports (SWPBIS) is implemented in more than 23,000 schools. Several reviews have examined the impact of SWPBIS, including a meta-analysis of single-case design research. However, to date, there has not been a randomized controlled trials (RCTs) reviews on the effects of SWPBIS implementation to reduce disciplinary exclusion, including office discipline referrals and suspensions. The purpose of this study is to conduct a systematic meta-analysis of RCTs on SWPBIS. Ninety schools, including both elementary and high schools, met criteria to be included in this study. A statistically significant large treatment effect (g = −.86) was found for reducing school suspension. No treatment effect was found for office discipline referrals.
Citation: Gage, N.A., Whitford, D.K. and Katsiyannis, A., 2018. A review of schoolwide positive behavior interventions and supports as a framework for reducing disciplinary exclusions. The Journal of Special Education, p.0022466918767847.
Link: Schoolwide Positive Behavior Interventions and Supports as a Framework for Reducing Disciplinary Exclusions
October 26, 2018
Hard Words: Why aren’t kids being taught to read?
This report and podcast examines the scientific basis for how to teach reading to children. This investigation reveals how children learn to read, emphasizing the five critical components of reading instruction. Unfortunately, most teacher preparation programs ignore the science and, in some cases, actively resist it. As a result, millions of kids are being set up to fail. This American Public Media documentary assesses the current knowledge base behind sound reading practices, the positive impact of effective reading practices can have on student reading performance, and the challenges faced in implementing these practices in the face of opposition from practitioners of whole language and proponents of balanced reading instruction.
Citation:Hanford, E, (2018). Hard Words: Why aren’t kids being taught to read? American Public Media (APM). https://www.apmreports.org/story/2018/09/10/hard-words-why-american-kids-arent-being-taught-to-read
Link: https://www.apmreports.org/story/2018/09/10/hard-words-why-american-kids-arent-being-taught-to-read
June 27, 2018
Four Domains for Rapid School Improvement: An Implementation Framework
The National Implementation Research Network (NIRN) and the National Center for School Turnaround published the “Four Domains for Rapid School Improvement: An Implementation Framework” as a companion to the Center for School Turnaround’s publication of “The Four Domains for Rapid School Improvement: A Systems Framework“. This paper describes “how” to effectively implement lasting school improvement initiatives that maximize leadership, develop talent, amplify instructional transformation, and shift the culture.
Citation:Jackson, K., R., Fixsen, D., and Ward, C. (2018). Four Domains for Rapid School Improvement: An Implementation Framework. The Center on School Turnaround.
Link: https://files.eric.ed.gov/fulltext/ED583980.pdf
January 22, 2018
The impact of tier 1 reading instruction on reading outcomes for students in Grades 4–12: A meta-analysis
This meta-analysis examines the impact of 1st tier reading instruction on reading outcomes for students in grades 4-12 in an Multi-tiered System of Support (MTSS) service delivery model. 37 studies met criteria for inclusion in the meta-analysis. The study finds small, but positive effects for 1st tier reading instruction on comprehension, vocabulary, and indicates minimum evidence for struggling readers maintaining or improving reading comprehension over struggling students receiving typical instruction. Hedges’s g was used calculating effect sizes. Because of the limited number of studies examining phonics/word recognition and fluency instruction, it was not possible these critical instruction areas in this meta-analysis.
Citation: Swanson, E., Stevens, E. A., Scammacca, N. K., Capin, P., Stewart, A. A., & Austin, C. R. (2017). The impact of tier 1 reading instruction on reading outcomes for students in Grades 4–12: A meta-analysis. Reading and Writing, 30(8), 1639-1665.
Link: https://link.springer.com/article/10.1007/s11145-017-9743-3
December 18, 2017
(1) An Evaluation of a Learner Response System (2) The Effects of Financial Incentives on Standardized Testing (3) Do Teacher Observations Make Any Difference to Student Performance?
Commentary: This piece reports on three examples of studies of practices that did not produce positive results and highlights the issue of publication bias in educational research. There are powerful contingencies that shape the publication process in ways that do not always work in the best interest of science. For example, promotion and tenure committees do not give the same weight to published replication studies. Also, journals generally do not publish studies that show no effect resulting in the “file drawer problem”. The only exception to this rule is if a study shows that a widely accepted intervention is not effective. Studies that show no effect may be very experimentally rigorous but because they did not show an experimental effect the studies are relegated to the researchers file drawer. These contingencies result in a publication bias for original research that demonstrates a positive effect. This can result in efforts to systematically review the evidence for an intervention over-estimating its effectiveness. Publishing in peer-reviewed journals is a critical component needed to safeguard the quality of research but these biases reflect potential publication biases. Replication is a fundamental cornerstone of science. Replication studies demonstrate the robustness of a finding. The biases against publishing non-results is a bit more complicated. Some studies that report non-results are unimportant. For example, demonstrating that a car will not run if gas is put in the tires is unimportant. The only important demonstration is one that shows a positive relation between where the gas was put in the car and the car actually running. Other studies reporting non-results are important because they show that a variable that has been experimentally demonstrated to have an impact on student behavior does not have that effect in a replication study or under a particular set of conditions.
News Summary:
- An Evaluation of a Learner Response System: A Learner Response System (LRS) is a classroom feedback tool that is becoming increasing popular. LRS is the practice of teachers and pupils using electronic handheld devices to provide immediate feedback during lessons. Given that feedback has been found to be a powerful tool in learning, it is not surprising that LRS are being adopted. The important question remains, do LRS increase student performance. This study tests a Learner Response System using Promethean handsets to assess whether it improves student outcomes. The study found no evidence that math and reading were improved using the system for 2 years.
- The Effects of Financial Incentives on Standardized Testing: Standardized testing has increasingly been used to hold educators accountable. Incentives are often offered as a way to improve student test performance. This study examines the impact incentives for students, parents and tutors on standardized test results. The researchers provided incentives on specially designed tests that measure the same skills as the official state standardized tests; however, performance on the official tests was not incentivized. This study finds substantial improvement for performance when there were incentives on the results did not generalize to the official test. This calls into question how to effectively use incentives so they will actually produce desired outcomes.
- Do Teacher Observations Make Any Difference to Student Performance? Research strongly suggests that feedback obtained through direct observations of performance can be a powerful tool for improving teacher’s skills. This study examines a peer teacher observation method used in England. The study found no evidence that Teacher Observation improved student language and math scores.
Citation:
(1) Education Endowment Foundation (2017). Learner Response System. Education Endowment Foundation. Retrieved https://educationendowmentfoundation.org.uk/projects-and-evaluation/projects/learner-response-system/.
(2) John A. List, Jeffrey A Livingston and Susanne Neckermann. “Do Students Show What They Know on Standardized Tests?” working papers (2016) Available at: http://works.bepress.com/jeffrey_livingston/19/
(3) Education Endowment Foundation (2017). Teacher Observation. Education Endowment Foundation. Retrieved https://educationendowmentfoundation.org.uk/projects-and-evaluation/projects/teacher-observation/.
Link:
(1) https://educationendowmentfoundation.org.uk/projects-and-evaluation/projects/learner-response-system/
(2) https://works.bepress.com/jeffrey_livingston/19/
(3) https://educationendowmentfoundation.org.uk/projects-and-evaluation/projects/teacher-observation/
December 4, 2017
Framework for Improving Education Outcomes
Multitiered system of support (MTSS) is a framework for organizing service delivery. At the core of MTSS is the adoption and implementation of a continuum of evidence-based interventions that result in improved academic and behavioral outcomes for all students. MTSS is a data-based decision making approach based on the frequent screening of progress for all students and intervention for students who are not making adequate progress.
Citation: States, J., Detrich, R., and Keyworth, R. (2017). Multitiered System of Support Overview. Oakland, Ca. The Wing Institute.
Link: https://winginstitute.org/school-programs-multi-tiered-systems
October 4, 2017
The usual approach to determining if an intervention is effective for a student is to review student outcome data; however, this is only part of the task. Student data can only be understood if we know something about how well the intervention was implemented. Student data without treatment integrity data are largely meaningless because without knowing how well an intervention has been implemented, no judgments can be made about the effectiveness of the intervention. Poor outcomes can be a function of an ineffective intervention or poor implementation of the intervention. Without treatment integrity data, the is a risk that an intervention will be judged as ineffective when, in fact, the quality of implementation was so inadequate that it would be unreasonable to expect positive outcomes.
Citation: Detrich, R., States, J. & Keyworth, R. (2017). Treatment Integrity in the Problem Solving Process. Oakland, Ca. The Wing Institute
Link: https://www.winginstitute.org/treatment-integrity-problem-solving
October 4, 2017
Student achievement scores in the United States remain stagnant despite constant reform. New initiatives arise promising hope, only to disappoint after being adopted, implemented, and quickly found wanting. The cycle of reform followed by failure has had a demoralizing effect on schools, making new reform efforts more problematic. These efforts frequently fail because implementing new practices is far more challenging than expected and require that greater attention be paid to implementation. A fundamental factor leading to failure is inattention to treatment integrity. When innovations are not implemented as designed, it should not be a surprise that anticipated benefits are not forthcoming. The question is, what strategies can educators employ to increase the likelihood that practices will be implemented as designed?
Strategies designed to increase treatment integrity fall into two categories: antecedent-based strategies and consequence-based strategies. Antecedent-based strategies involve any setting event or environmental factor that happens prior to implementing the new practice and that increases the likelihood of success as well as eliminates setting events or environmental considerations that decrease the likelihood of success. Consequence-based strategies are designed to impact actions that happen after implementation of the new practice and that are likely to increase or decrease treatment integrity.
Citation: Detrich, R., States, J. & Keyworth, R. (2017). Approaches to Increasing Treatment Integrity. Oakland, Ca. The Wing Institute
Link: https://www.winginstitute.org/treatment-integrity-strategies
