December 17, 2021
At the core of any multi-tiered system of support (MTSS; e.g., School-wide positive behavior intervention or Response to Intervention) is the requirement Tier 1 or universal intervention is implemented with adequate fidelity to benefit most students. If Tier 1 interventions are not implemented with fidelity, too many students will receive more intensive Tier 2 and Tier 3 interventions. The increased intensity of intervention will also unnecessarily strain school resources. It is important to remember that MTSS are frameworks, and ultimately the benefit to students depends on adopting empirically-supported interventions and then implementing them well. Without fidelity measures, it is not possible to know if failing to respond to an intervention is a problem with the intervention or poor implementation. Often interventions are abandoned for apparent lack of effectiveness when, in fact, the intervention was not implemented with fidelity.
Fidelity is a complex construct that can be measured at different levels and different frequencies. Each measure yields different types of information. Until now, we have not known how researchers measured fidelity. This situation has been partially resolved in a recent review by Bruckman et al. (2021). Their review measured how researchers assessed treatment integrity, the frequency it was evaluated, and the level (school or individual implementer).
Bruckman and colleagues reported that measures at the school level were reported about twice as often as at the individual level and assessed once or twice per year. Treatment integrity measured at this level tells us how well the overall system is functioning with respect to the implementation of the intervention. Data at this level does not indicate if all students are receiving a well-implemented intervention or if some students are not receiving the intervention as planned. Measuring treatment integrity at the level of an individual teacher will inform if students in a particular teacher’s classroom are receiving a well-implemented intervention. Individual-level measures are essential for data-based decision-making when determining if a student should receive more intensive services at Tier 2. Low levels of fidelity would suggest that rather than increase the intensity of service for a student, it would be wise to invest in improving the individual teacher’s implementation of the intervention.
Finally, Bruckman and colleagues discussed the limitations of assessing treatment integrity once or twice a year. Such infrequent measurement does not tell us if implantation with integrity is occurring consistently or not. The challenge of assessing more frequently is that it places a high demand on resources. Considerably more research is required to develop effective and efficient methods for evaluating treatment integrity.
Citation for Article:
Buckman, M. M., Lane, K. L., Common, E. A., Royer, D. J., Oakes, W. P., Allen, G. E., … & Brunsting, N. C. (2021). Treatment Integrity of Primary (Tier 1) Prevention Efforts in Tiered Systems: Mapping the Literature. Education and Treatment of Children, 44(3), 145-168.
December 17, 2021
School-wide Positive Behavioral Interventions and Supports (SWPBIS) is one of the most widely adopted frameworks for supporting prosocial behavior in schools; however, it is not uncommon for schools to abandon it before fully implementing it. A recent review by Fox and colleagues (2021) sought to understand the facilitators and barriers to implementing SWPBIS. The study of facilitators of implementation identified adequate resources, strong fidelity of implementation, effective SWPBIS team function, and meaningful collection and use of the data. The most common barriers identified by participants in the study were staff beliefs that conflict with the philosophy of SWPBIS, poor implementation fidelity, and lack of resources. Less frequently cited barriers included lack of supporting leadership, lack of staff buy-in, and school characteristics (school size, elementary or high school).
The good news in this review is that many of the barriers can be addressed by assuring the facilitators of implementation are well established. Developing systems promoting high levels of implementation fidelity addresses the barrier of poor implementation fidelity. More challenging is resolving the conflict between teachers’ beliefs and the core philosophy of SWPBIS. It may be worth examining the roots of these ideas to understand their basis and how, specifically, they are inconsistent with SWPBIS. To some extent it may be possible to incorporate the teachers’ competing beliefs into the specific practices embedded in SWPBIS without doing harm to the core features of it. In other instances, there may be so much resistance to SWPBIS practices that implementation efforts should not be initiated until teachers’ concerns have been addressed to their satisfaction. Unless a substantial majority of teachers and administrators are willing to support the SWPBIS initiative, implementation will not be successful. This highlights the critical role exploration and adoption plays in implementation.
Link to article: https://link.springer.com/article/10.1007/s43494-021-00056-0
Fox, R. A., Leif, E. S., Moore, D. W., Furlonger, B., Anderson, A., & Sharma, U. (2021). A Systematic Review of the Facilitators and Barriers to the Sustained Implementation of School-Wide Positive Behavioral Interventions and Supports. Education and Treatment of Children, 1-22.
December 17, 2021
Classroom teachers consistently report classroom management as a significant area of concern. This is especially true for early career teachers and teachers often report it is one of the most common reasons for leaving the profession. Highly rigorous, practical, and effective pre-service and professional development training approaches are necessary to address classroom behavior challenges. A recent systematic review by Hirsch and colleagues (2021) reviewed the literature on classroom management training to determine the current status of professional development for classroom teachers. Ultimately, the authors identified eight experimental studies that met inclusion criteria. There were several interesting findings from this review. Of the experimental studies reviewed, a low number of participants reported having received prior training in classroom management. As the authors discuss, these results are not surprising since relatively few states have policy requirements for classroom teachers to receive instruction in classroom management. Stevenson and colleagues (2020) proposed the steps for improving instruction in classroom management: (1) pre-service coursework must include a course on explicit, evidence-based, culturally, and contextually relevant classroom management skills; (2) fieldwork should incorporate explicit support and coaching on classroom management; and (3) state departments of education should require training that aligns with the best practices of classroom management to support the needs of teachers and students. If these three recommendations were acted on, teachers would likely be more prepared to address the behavioral challenges in their classrooms.
A second finding from the Hirsch et al. (2021) systematic review was that there is considerable evidence to support practice-based professional development rather than the standard “train and hope” (Stokes & Baer, 1977). There are seven critical features to practice-based professional development. In the articles reviewed in this systematic review, all of the studies incorporated some elements of practice-based professional development. A somewhat surprising finding among the reviewed articles was that the length of training ranged from 15 minutes to four days. This result is likely possible because the researchers used practice-based professional development that included coaching and feedback to teach the new skills.
Hirsch and colleagues made a strong argument for the increased use of technology to support professional development, ranging from low-tech methods to telehealth. Telehealth makes it possible for teachers in rural communities to access high-quality professional development. Creating more effective and efficient professional development is necessary to scale it up.
As Hirsch et al. make clear, considerably more research on professional development is necessary. Eight articles are a small database for making policy and practice recommendations.
Link to Article: https://link.springer.com/article/10.1007/s43494-021-00042-6
Hirsch, S. E., Randall, K., Bradshaw, C., & Lloyd, J. W. (2021). Professional Learning and Development in Classroom Management for Novice Teachers: A Systematic Review. Education and Treatment of Children, 44(4), 291-307.
Stevenson, N. A., VanLone, J., & Barber, B. R. (2020). A commentary on the misalignment of teacher education and the need for classroom behavior management skills. Education and Treatment of Children, 43(4), 393-404.
Stokes, T. F., & Baer, D. M. (1977). An implicit technology of generalization 1. Journal of applied behavior analysis, 10(2), 349-367.
December 17, 2021
One of the most important decisions educators make is what reading curriculum to adopt. The consequences of that decision can have profound implications for students. Adopting a curriculum not based on the science of reading is likely to produce a generation of poor readers. Education Week recently reviewed a report from EDReports that reported two of the most commonly adopted reading curricula failed to meet their new review standards. The review covered both K-2 and grades 3-8 for Fountas and Pinnell Classroom and Units of Study from the Teachers College Reading and Writing Project. Neither program met expectations for text quality or alignment to standards. In 2019, EdWeek Research Center reported that 44% of K-2 early reading and special education teachers used Fountas and Pinnell’s Leveled Literacy Intervention, a companion intervention to Fountas and Pinnell Classroom.
Additionally, it was reported that 16% of teachers used Units of Study for Teaching Reading. Approximately 60% of K-2 and special education students are taught reading with curricula that do not meet standards for reading instruction. This is distressing given the importance of early reading on the educational trajectory for students.
Link for Ed Week article: https://www.edweek.org/teaching-learning/new-curriculum-review-gives-failing-marks-to-popular-early-reading-programs/2021/11
Kurtz, H., Lloyd, S., Harwin, A., Chen, V., & Furuya, Y. (2020). Early Reading Instruction: Results of a National Survey. Editorial Projects in Education.
November 5, 2021
Praise is generally recognized as an empirically-supported approach to improving student behavior (Simonsen, Fairbanks, Briesch, Myers, & Sugai, 2008); however, in spite of the research evidence, praise is often under-utilized in classrooms (Floress & Jenkins, 2015; Gable, Hendrickson, Shores, & Young, 1983; Sutherland, Wehby, & Copeland, 2000) highlighting the research to practice gap. Why don’t teachers implement praise more often and more consistently? Shernoff and colleagues (2020) attempted to answer this question. In this study, they recruited 41 teachers who identified praise as a professional development goal to participate in a coaching program with the goal of increasing praise. After the study was completed, the teachers were asked about facilitators (helpful factors) and barriers (obstacles) to using praise. During the study, the teachers slowly increased the frequency and quality of praise over a three-month period. This suggests that it takes time to make practice changes and it may be more complex to implement praise than is generally considered. The teachers identified a number of facilitators to using praise including feedback to students without having to criticize them, positive student reactions, and deliberate planning and reminders (planning how to use praise in the context of a specific lesson). Teachers also identified barriers to using praise including it interferes with instruction, conflicts with education, training and beliefs, and the context dependent nature of praise. Using praise in classrooms is an innovation when there is initially a very low level. From an implementation science perspective, the process leading to adoption can be complex and influenced by factors that are unrelated to the intervention. For example, if an innovation conflicts with a teacher’s education, training, and beliefs, then the innovation will likely be met with resistance. One way to reduce the resistance to the innovation is to have someone that is credible to the teacher champion the intervention rather than outside consultants, trainers, or researchers. Often the most credible person to a teacher is another teacher. This highlights that introducing interventions that are seemingly simple is not a simple process.
Citation: Shernoff, E. S., Lekwa, A. L., Reddy, L. A., & Davis, W. (2020). Teachers’ use and beliefs about praise: A mixed-methods study. School Psychology Review, 49(3), 256-274.
- Floress, M. T., & Jenkins, L. N. (2015). A preliminary investigation of kindergarten teachers’ use of praise in general education classrooms. Preventing School Failure: Alternative Education for Children and Youth, 59(4), 253–262. doi:10.1177/0198742917709472.
- Gable, R. A., Hendrickson, J. M., Shores, R. E., & Young, C. C. (1983). Teacher-handicapped child classroom interactions. Teacher Education and Special Education: The Journal of the Teacher Education Division of the Council for Exceptional Children, 6(2), 88–95. doi:10.1177/019874299301800405.
- Simonsen, B., Fairbanks, S., Briesch, A., Myers, D., & Sugai, G. (2008). Evidence-based practices in classroom management: Considerations for research to practice. Education and Treatment of Children, 3, 351–380. doi:10.1353/etc.0.0007.
- Sutherland, K. S., Wehby, J. H., & Copeland, S. R. (2000). Effect of varying rates of behavior-specific praise on the ontask behavior of students with EBD. Journal of Emotional and Behavioral Disorders, 8(1), 2–8. doi:10.1177/ 106342660000800101.
November 5, 2021
Disruptive behavior is one of the biggest challenges facing classroom teachers today. Many of the students with the most disruptive behavior are classified as having emotional and behavioral disorders or at risk of developing them. These students take up a disproportionate amount of classroom time, reducing time spent of instruction. Generally, these students have not been responsive to class-wide behavior management approaches and require more individualized and intensive intervention. This raises the question what are the effective practices that will benefit the student? A recent review by Riden and colleagues attempted to answer this question through a systematic review of the literature. They identified eight practices that met critieria to be considered evidence-based: (check in/check out (2) functional assessment-based intervention (3) group contingencies (4) peer- mediated interventions (5) self-management (6) self-regulated strategy development for writing (7) token economies (8) video modeling. Another eleven practices were identified as promising and include: (1) praise (2) opportunities to respond (3) behavior contracting (4) cooperative learning (5) goal setting (6) good behavior game (7) high probability requests (8) instructional choice (9) self-determination (10) social skills (11) time out. It is important to recognize that practices described as promising may well be effective but the empirical data base is not yet strong enough to warrant inclusion as evidence-based. These practices should be considered when selecting approaches for addressing significant behavior problems.
These data are important because they can guide educators about which practices to adopt when addressing the behavior problems posed by disruptive students.
Citation: Riden, B. S., Kumm, S., & Maggin, D. M. (2021). Evidence-Based Behavior Management Strategies for Students With or At Risk of EBD: A Mega Review of the Literature. Remedial and Special Education, 07419325211047947.
November 5, 2021
One of the great challenges in education is training all staff to implement interventions. There is considerable reliance on para-professionals, especially in special education, to support students. Many of the para-professionals have minimal training in educational practices. In many cases, the training that does occur is the traditional didactic model and there is little evidence that it produces the outcomes it is supposed to yield. An alternative model of training that holds great promise is coaching; however, there are limitations to it because it often relies on outside coaches which makes it cost-prohibited for many districts. A recent report by Sallese and Vannest (2021) offers an alternative that may make coaching more cost-effective. In their research, they utilized classroom teachers to coach the para-professionals working in the classroom to increase the use of behavior specific praise. Many teachers report that they have little or no pre-service or in-service training focused on paraprofessional training and support (Douglas, Chapin & Nolan, 2016). To address this issue, the teachers were provided a manual to guide their coaching efforts. The components of the coaching package included self-monitoring, performance feedback, goal setting, modeling, and action planning. In surveys of paraprofessionals one of the most cited concerns is lack of training and support in behavior management (Mason, et al., 2021). Behavior specific praise has been identified as an evidence-based component of classroom behavior management (Simonsen, Fairbanks, Briesch, Myers, & Sugai (2008); however, it has often been a challenge to increase behavior specific praise and maintain it over time. In this study, all four of the para-professionals that received coaching increased their rate of behavior specific praise. In addition, 100% of the participants agreed that the procedures were appropriate and feasible in terms of time and effort required to implement.
This was a small-scale study but it holds promise as a method for coaching implementers to carry out effective practices. From an implementation perspective, this provides a cost-effective approach to increase the internal capacity of a system to implement adopted practices. Building internal capacity is critical if effective interventions are to be sustained over generations of implementers.
Citation: Sallese, M. R., & Vannest, K. J. (2021). Effects of a Manualized Teacher-Led Coaching Intervention on Paraprofessional Use of Behavior-Specific Praise. Remedial and Special Education, 07419325211017298.
- Douglas, S. N., Chapin, S. E., & Nolan, J. F. (2016). Special education teachers’ experiences supporting and supervising paraeducators: Implications for special and general education settings. Teacher Education and Special Education, 39(1), 60–74. https://doi.org/gf86tz
- Mason, R. A., Gunersel, A. B., Irvin, D. W., Wills, H. P., Gregori, E., An, Z. G., & Ingram, P. B. (2021). From the frontlines: Perceptions of paraprofessionals’ roles and responsibilities. Teacher Education and Special Education, 44(2), 97–116. https://doi.org/fwn6
- Simonsen, B., Fairbanks, S., Briesch, A., Myers, D., & Sugai, G. (2008). Evidence-based practices in classroom management: Considerations for research to practice. Education and treatment of children, 351-380.
November 5, 2021
One of the persistent problems in education is the gap between what we know about effective educational practices and the practices that are frequently used in public schools. Many of these practices do not have empirical support. The challenge for all educators is how do we close the gap? The flow of research to practice is often perceived as being a one way flow from researchers that develop effective interventions and disseminate them to practitioners who are expected to adopt them (Ringeisen, Henderson, & Hoagwood, 2003). Ringeisen et al., argue that this is not likely to result in widespread adoption of effective practices. McLaughlin and colleagues (1997) have made the argument that having an array of effective practices is not sufficient for closing the research to practice gap. In many instances, the practices developed by researchers are not a good contextual fit for the school settings because training and experience requirements for implementers are unreasonable within the school setting, the resources necessary for implementation are not present, and the time demands to implement are unrealistic. If the dominant model of disseminating empirically-supported interventions is not impacting the research to practice gap, what should we do? The goal is important but we need effective alternatives to the common approach. Recently a report from the William T. Grant Foundation, (Farrell, Penuel, Coburn, Daniel, Steup (2021) entitled, Research-Practice Partnerships in Education: The State of the Field. In this report, the authors define research-practice partnerships as “intentionally organized to connect diverse forms of expertise and shift power relations in the research endeavor to ensure that all partners have a say in the joint work.” This is a significant shift from usual practice in the development and dissemination of effective practices. There are five principles associated with these partnerships: (1) they are long term collaborations (2) they work toward educational improvement or equitable transformation (3) they feature engagement with research as a leading activity (4) they are intentionally organized to bring together a diversity of expertise (5) they employ strategies to shift power relations in research endeavors to ensure that all participants have a say. This is an important shift. Practitioners are now partners with researchers. It is a movement away from the researcher as expert model to a model in which practitioners are equally expert as researchers. Each is an expert in different domains of improving educational practices.
If practitioners are involved from the beginning in guiding research then the practices are more likely to be seen as usable by educators when considering interventions to adopt. The development of research-practice partnerships has the potential to increase the adoption of empirically-supported practices.
Citation: Farrell, C.C., Penuel, W.R., Coburn, C., Daniel, J., & Steup, L. (2021). Research-practice partnerships in education: The state of the field. William T. Grant Foundation.
References: McLaughlin, M. J., & Leone, P. E., Meisel, S., & Henderson, K. (1997). Strengthen school and community capacity. Journal of Emotional and Behavioral Disorders, 5(1), 15-24.
Ringeisen, H., Henderson, K., & Hoagwood, K. (2003). Context matters: Schools and the “research to practice gap” in children’s mental health. School Psychology Review, 32(2), 153-168.
October 5, 2021
A review of the evidence for real-time performance feedback to improve instructional practice. Performance feedback, including real time performance feedback, has been implemented across many contexts including educational settings. Sinclair, Gesel, LeJeune, and Lemons (2020) evaluated 32 studies that met their inclusion criteria to determine the effectiveness of real-time performance feedback in improving the instructional practices of educators. Interestingly, all but two of the studies utilized bug-in-the-ear technology in which the teacher wore an ear piece and a coach provided immediate feedback to the teacher as implementation of an intervention was occurring. In this review, teachers implementing both academic instruction and behavior management interventions were considered. Based on this review, the authors concluded that real time performance feedback was an evidence-based practice and could be used as a method for improving the performance of pre-service teachers, teachers, and paraprofessionals. The bug-in-the-ear technology offers several advantages. First, it is less intrusive than other methods for providing real time feedback. With current technology, the coach can view implementation in the classroom without being in the classroom. Technologies such as Go Pro and Swivl have sufficient flexibility for the coach to get a good sample of what is happening in the classroom. A second advantage is that because the feedback is immediate it is more likely to be effective compared to when the feedback is delayed. Finally, the bug-in-the-ear technology is time saving because the feedback is delivered in real-time. Brief follow-up meetings to discuss issues related to the intervention can be scheduled. Interestingly, bug-in-the-ear technology has been around for decades but has been under-utilized in educational settings. An analysis of the barriers to utilizing this technology more broadly is warranted. The potential for impact on implementation of an intervention is significant.
Citation: Sinclair, A. C., Gesel, S. A., LeJeune, L. M., & Lemons, C. J. (2020). A review of the evidence for real-time performance feedback to improve instructional practice. The Journal of Special Education, 54(2), 90-100.
October 4, 2021
Electronically Delivered Support to Promote Intervention Implementation Fidelity: A Research Synthesis. Fundamental to any intervention outcome is the fidelity of implementation of the intervention. The ultimate goal of implementation science is to assure that innovations are implemented well enough for students to benefit. Failure to implement well can minimize the effectiveness of even the most powerful intervention. One of the challenges involved in insuring high quality implementation is that most approaches are resource intensive and often are not seen as feasible in school settings even though failure to achieve adequate implementation fidelity may result in a very poor benefit to cost ratio. One possible alternative is to utilize technology to reduce the resource demands. Fallon and colleagues (2021) conducted a systematic review to evaluate the effectiveness of technology-based supports to promote implementation fidelity. For the purposes of this review, “electronically delivered implementation supports (EDIS) was support delivered to an implementer electronically (e.g., via email, social media, video conferencing) for the purpose of improving educators’ implementation fidelity of a student intervention.” Fifteen studies met inclusion criteria and were judged to be of sufficient methodological rigor to warrant further analysis. The electronically delivered implementation supports ranged from video modeling, electronically delivered performance feedback, emailed intervention prompts, coaching via video conference, and online training modules. All of the studies were based on single participant designs. Since there are no agreed upon methods for calculating effect sizes for single participant designs, the authors calculated several different effect sizes (Tau-U, Standard Mean Difference, Hedges’ g, and a variation of Hedges’ g. In most of the studies, the effect sizes ranged from moderate to large regardless of the calculation method used. After completing the review, the authors provided guidance to educators about when to use the various methods of electronically delivered implementation supports. This article is a valuable resource to any educator considering implementing an intervention but is concerned about the resource requirements required for insuring high quality implementation. This article suggests that technology-based alternatives can be effective in supporting implementation and may reduce the overall demands on resources.
Citation: Fallon, L. M., Collier-Meek, M. A., Famolare, G. M., DeFouw, E. R., & Gould, K. M. (2020). Electronically Delivered Support to Promote Intervention Implementation Fidelity: A Research Synthesis. School Psychology Review, 1-16.