Education Drivers

Instructional Effectiveness

Publications

TITLE
SYNOPSIS
CITATION
Overview of Treatment Integrity

For the best chance of a positive impact on educational outcomes, two conditions must be met: (a) Effective interventions must be adopted, and (b) those interventions must be implemented with sufficient quality (treatment integrity) to ensure benefit.  To date, emphasis in education has been on identifying effective interventions and less concern with implementing the interventions. The research on the implementation of interventions is not encouraging. Often, treatment integrity scores are very low and, in practice, implementation is rarely assessed. If an intervention with a strong research base is not implemented with a high level of treatment integrity, then the students do not actually experience the intervention and there is no reason to assume they will benefit from it. Under these circumstances, it is not possible to know if poor outcomes are the result of an ineffective intervention or poor implementation of that intervention. Historically, treatment integrity has been defined as implementing an intervention as prescribed. More recently, it has been conceptualized as having multiple dimensions, among them dosage and adherence which must be measured to ensure that it is occurring at adequate levels.

Detrich, R., States, J., & Keyworth, R. (2107). Overview of Treatment Integrity. Oakland, Ca. The Wing Institute.

Summative Assessment Overview

Summative assessment is an appraisal of learning at the end of an instructional unit or at a specific point in time. It compares student knowledge or skills against standards or benchmarks. Summative assessment includes midterm exams, final project, papers, teacher-designed tests, standardized tests, and high-stakes tests. 

States, J., Detrich, R. & Keyworth, R. (2018). Overview of Summative Assessment. Oakland, CA: The Wing Institute. https://www.winginstitute.org/assessment-summative

 

Data Mining

TITLE
SYNOPSIS
CITATION
What is the experience level of school principals, in the role of principal?

The purpose of this analysis is to examine data from the National Schools and Staffing Survey on principal years of experience on the job.

Keyworth, R. (2014). What is the experience level of school principals, in the role of principal? Retrieved from what-is-experience-level834.

 

Presentations

TITLE
SYNOPSIS
CITATION
Now What? The Current State of Principal Preparation, Evaluation, and Support
This paper examines the current state of principal development in the context of best practices, including: evidence-based curriculum, well-trained instructors, effective coaching, and ongoing feedback and support.
Keyworth, R. (2015). Now What? The Current State of Principal Preparation, Evaluation, and Support [Powerpoint Slides]. Retrieved from 2015-calaba-presentation-randy-keyworth.

 

Student Research

TITLE
SYNOPSIS
CITATION
Evaluating a Multimedia Professional Development Package for Improving Implementation of Evidence-Based Instructional Practices

Three 8th grade English teachers participated in this single-case multiple baseline experiment. These teachers were observed daily during classes that were inclusive to students with disabilities. Observations were conducted using the Classroom Teaching Scan (www.classroomteachingscan.com/ctscan/). Within the Classroom Teaching Scan, a checklist of quality indicators for modeling was the primary dependent variable. Additionally, observations were scored using the Protocol for Language Arts Teaching Observations (PLATO, 2017). Participating students responded to curriculum-based measurement writing prompts throughout the study.

Minor changes in performance on the PLATO and CBM measures were demonstrated. However, these measures were descriptive in nature, not experimental. Therefore, more research over a sustained period of time is necessary to determine the effect of this professional development package on distal measures of teacher quality and student outcomes.

Elwood, J.R. (2017). Evaluating a Multimedia Professional Development Package for Improving Implementation of Evidence-Based Instructional Practices:Oakland, CA: The Wing Institute. https://www.winginstitute.org.

 

TITLE
SYNOPSIS
CITATION
Evaluating Principals: Balancing Accountability with Professional Growth
The goal of this paper is to provide policymakers with recommendations for the design and implementation of strong principal development and evaluation systems. States and local school systems that pursue these ideas can use principal evaluation to drive a powerful vision of principal effectiveness and, by consequence, improve outcomes for all students.
(2010). Evaluating Principals: Balancing Accountability with Professional Growth. New Leaders for New Schools.
Principal Evaluation Handbook
New Leaders has recently published a new principal evaluation model. It includes seven modules: (1) Overview of the New Leaders Principal Evaluation Model, (2) The Principal Evaluation Rubric, (3) Setting a Principal Practice Goal + Strategic Planning, (4) Identifying Evidence, (5) Direct Observation of Principal Practice, (6) Collecting and Mapping Evidence to the Principal Practice Rubric, and (7) Providing Actionable Feedback.
(2012). Principal Evaluation Handbook. New Leaders
Putting Principal Evaluation into Practice
Overview New Leaders has recently published a new principal evaluation model. It includes seven modules: (1) Overview of the New Leaders Principal Evaluation Model, (2) The Principal Evaluation Rubric, (3) Setting a Principal Practice Goal + Strategic Planning, (4) Identifying Evidence, (5) Direct Observation of Principal Practice, (6) Collecting and Mapping Evidence to the Principal Practice Rubric, and (7) Providing Actionable Feedback.
(2012). Putting Principal Evaluation into Practice. New Leaders
Principal Evaluation Rubric
Overview New Leaders has recently published a new principal evaluation model. It includes seven modules: (1) Overview of the New Leaders Principal Evaluation Model, (2) The Principal Evaluation Rubric, (3) Setting a Principal Practice Goal + Strategic Planning, (4) Identifying Evidence, (5) Direct Observation of Principal Practice, (6) Collecting and Mapping Evidence to the Principal Practice Rubric, and (7) Providing Actionable Feedback.
(2012). Putting Principal Evaluation into Practice. New Leaders
A New Approach to Principal Preparation: Innovative Programs Share Their Practices and Lessons Learned

This paper examines a number of promising principal preparation programs to identify lessons for improving the impact of principals on student perrmance.

A new approach to principal preparation: Innovative programs share their practices and lessons learned. Rainwater Leadership Alliance, 2010.

Rethinking the Use of Tests: A Meta-Analysis of Practice Testing

This meta-analysis examined the effects of practice tests versus non-testing learning conditions on student performance. Research demonstrates that students who take practice tests often outperform students in non-testing learning conditions such as restudying, practice, filler activities, or no presentation of the material. Results reveal that practice tests are more beneficial for learning than restudying and all other comparison conditions.

Adesope, O. O., Trevisan, D. A., & Sundararajan, N. (2017). Rethinking the Use of Tests: A Meta-Analysis of Practice Testing. Review of Educational Research, 0034654316689306.

Is the three-term contingency trial a predictor of effective instruction?

Two experiments are reported which test the effect of increased three-term contingency trials on students' correct and incorrect math responses. The results warrant further research to test whether or not rates of presentation of three-term contingency trials are predictors of effective instruction.

Albers, A. E., & Greer, R. D. (1991). Is the three-term contingency trial a predictor of effective instruction?. Journal of Behavioral Education1(3), 337-354.

Instructional Coaching: Professional development strategies that improve instruction

Two key strategies are central to the Annenberg Institute's work on professional development systems: professional learning communities (small groups of teachers, administrators, community members, and others who work together to improve professional practice); and instructional coaching (school-based, educator-led professional learning for groups of teachers in specific content areas). This package includes two publications describing these strategies and what we have learned about using them effectively.

Annenburg Institute for School Reform. (2004). Instructional Coaching: Professional development strategies that improve instruction.Retrieved from: http://www.annenberginstitute.org/publications/professional-development-strategies-professional-learning-communitiesinstructional-coac

 

A Synthesis of Empirical Research on Teaching Mathematics to Low-Achieving Students

This meta-analysis synthesized research on the effects of interventions to improve mathematics achievement of students considered at risk for academic failure. It found that effective interventions included providing teachers and students with student performance data; using peer tutors; providing clear, specific feedback to parents on children's mathematics success; and using explicit instruction to teach math.

Baker, S., Gersten, R., & Lee, D. S. (2002). A synthesis of empirical research on teaching mathematics to low-achieving students. The Elementary School Journal, 51-73.

Obtaining Validation from Graduates on a Restructured Principal Preparation Program

This study used within-program comparison of follow-up survey responses from two sets of program graduates from a university-based leadership preparation program to determine differences in program features and outcome measures.

Ballenger, J., Alford, B., McCune, S. L., & McCune, E. D. (2010). Obtaining validation from graduates on a restructured principal preparation program. Jsl Vol 19-N5, 19 533.

Questioning the Author: An approach for enhancing student engagement with text

Noting that elementary-school students often fail to understand many of the ideas presented in school textbooks, this book presents the Questioning the Author (QtA) strategy, which is designed to establish student interactions with text and build greater understanding by teaching students to question the ideas presented in the text while they are reading. The book presents many examples of QtA in action as children engage with narrative and expository texts to construct meaning. After a foreword and an introduction, chapters in the book are: (1) "What Is Questioning the Author and How Was It Developed?"; (2) "Queries"; (3) "Planning"; (4) "Discussion"; (5) "Implementation"; and (6) "Where Has Questioning the Author Been and Where Is It Going?" Contains 66 references. (RS)

Beck, I. L., & McKeown, M. G., Hamilton, R. L., & Kugan, L. (1997). Questioning the Author: An approach for enhancing student engagement with text.Newark, DE: International Reading Association.

 

Development and validation of the clarity indicators scale

This study was conducted to create a reliable and valid low- to medium-inference, multidimensional measure of instructor clarity from seminal work across several academic fields. The five factors were explored in regards to their ability to predict the outcomes. Implications for instructional communication researchers are discussed.

Bolkan, S. (2017). Development and validation of the clarity indicators scale. Communication Education66(1), 19-36.

Progress Over a Decade in Preparing More Effective School Principals

Over the past 10 years, the Southern Regional Education Board has helped states and public universities across the region evaluate their state policies for preparing school principals who are leaders of instruction. This benchmark report reviews the past decade and looks at 10 learning-centered leadership indicators to gauge how far states have come and how far they need to go in selecting, preparing and supporting leaders of change.

Bottoms, G., Egelson, P., & Bussey, L. H. (2012). Progress over a decade in preparing more effective school principals. Atlanta, GA: Southern Regional Education Board.

An Analysis of Principal Preparation Programs at Pennsylvania State Schools

This study examined principal preparation programs within the Pennsylvania State System of Higher Education to ascertain whether their programs were structured in a way that would equip principal candidates with the leadership roles deemed essential for 21st century school leadership.

Burks, Karlin. (2014). An Analysis of Principal Preparation Programs at Pennsylvania State Schools. Seton Hall University Dissertations and Theses (ETDs) Paper 1934.

Impact of the script in a supplemental reading program on instructional opportunities for student practice of specified skills

This study sought to investigate the impact of a supplemental program’s script on the rate of on-task and off-task instructional opportunities offered by the instructor for students to practice the specific skills targeted in lesson exercises.

Cooke, N. L., Galloway, T. W., Kretlow, A. G., & Helf, S. (2011). Impact of the script in a supplemental reading program on instructional opportunities for student practice of specified skills. The Journal of Special Education45(1), 28-42.

Rethinking Leadership: The Changing Role of Principal Supervisors

This report consists of two parts: a survey of 67 public school systems district staff serving as principal supervisors and on-site analysis of six districts pre-service training and support systems for new principals.

Corcoran, A., et al. (2013). Rethinking Leadership: The Changing Role of Principal Supervisors. The Wallace Foundation.

On the teachability of communication strategies.

This article describes what communication strategies are and provides an overview of the teachability issue, discussing the arguments for and against strategy instruction, and suggests three possible reasons for the existing controversy. 

Dörnyei, Z. (1995). On the teachability of communication strategies. TESOL quarterly29(1), 55-85.

The Effectiveness of Principal Training and Formal Principal Mentoring Programs

The purpose of this study was to determine principals' perceptions of how effective mentoring programs and university-based principal preparation programs are in developing the skills necessary to carry out the 13 critical success factors identified by the Southern Regional Education Board (SREB). A review of the literature addressed what it means to be an effective principal and what an effective mentoring program should look like.

Dodson, R. B. (2006). The Effectiveness of Principal Training and Formal Principal Mentoring Programs.

School Improvement Grants: Implementation and Effectiveness

The Institute of Education Sciences (IES) recently released a summary report of the impact of School Improvement Grants (SIG). The American Recovery and Reinvestment Act of 2009 provided states and school districts with $3 Billion for SIG. By accepting SIG grants states agreed to implement one of four interventions to improve the lowest performing schools: transformation, turnaround, restart, or closure. The goals of SIG were to improve practices in four main areas: (1) adopting comprehensive instructional reform strategies, (2) developing and increasing teacher and principal effectiveness, (3) increasing learning time and creating community-oriented schools, and (4) having operational flexibility and receiving support. The report finds minimal positive effects from the grants and no evidence that SIG had significant impacts on math and reading scores, graduation rates, or increased college enrollment.

 

Dragoset, L., Thomas, J., Herrmann, M., Deke, J., James-Burdumy, S., Graczewski, C., … & Giffin, J. (2017). School Improvement Grants: Implementation and Effectiveness (No. 76bce3f4bb0944f29a481fae0dbc7cdb). Mathematica Policy Research.

 

Learner Response System. Education Endowment Foundation

A Learner Response System (LRS) is a classroom feedback tool that is becoming increasing popular. LRS is the practice of teachers and pupils using electronic handheld devices to provide immediate feedback during lessons. Given that feedback has been found to be a powerful tool in learning, it is not surprising that LRS are being adopted. The important question remains, do LRS increase student performance. This study tests a Learner Response System using Promethean handsets to assess whether it improves student outcomes. The study found no evidence that math and reading were improved using the system for 2 years.

Education Endowment Foundation (2017). Learner Response System. Education Endowment Foundation. Retrieved https://educationendowmentfoundation.org.uk/projects-and-evaluation/projects/learner-response-system/.

What We Know About Literal and Inferential Comprehension in Reading

In 2000, the National Reading Panel identified five practice elements with a sufficient evidence base to be deemed essential for mastery of reading (National Institute of Child Health and Human Development, 2000). These elements consist of systematic teaching of phonemic awareness, phonics instruction, vocabulary, fluency, and exposure to reading comprehension strategies. This meta-analysis of 25 studies evaluates the impact of inference instruction in grades K-12. The study reported that inference instruction had an effect size d=0.58 on general comprehension and d= 0.68 on literal comprehension. These are “moderate to large” effects of instruction on general comprehension and to making inferences for both skilled and less skilled readers. The pattern differed for the literal measure, however, with skilled readers showing almost no gain but unskilled readers showing sizable gains. 

Elleman, A. M. (2017). Examining the impact of inference instruction on the literal and inferential comprehension of skilled and less skilled readers: A meta-analytic review. Journal of Educational Psychology, 109(6), 761-781.

Effective college teaching from the students' and faculty's view: Matched or mismatched priorities?

Thirty-one studies were located in each of which students and faculty specified the instructional characteristics they considered particularly important to good teaching and effective instruction. 

Feldman, K. A. (1988). Effective college teaching from the students' and faculty's view: Matched or mismatched priorities?. Research in Higher Education28(4), 291-329.

Do students benefit from longer school days? Regression discontinuity evidence from Florida’s additional hour of literacy instruction

This research examines the impact of longer school days on student achievement. This study attempts to fill in gaps in the evidence-base on this topic. Although this study finds positive outcomes for additional reading instruction, it is important to note that for achieving maximum results it is important to pair evidence-based reading instruction practices with the additional instruction time in order to achieve maximum results.

 

Figlio, D., Holden, K. L., & Ozek, U. (2018). Do students benefit from longer school days? Regression discontinuity evidence from Florida’s additional hour of literacy instruction. Economics of Education Review67, 171-183.

Coaching middle-level teachers to think aloud improves comprehension instruction and student reading achievement
 
 
In an effort to improve student achievement, a group of middle-school teachers at an underperforming school developed a school wide literacy plan. As part of the plan, they agreed to model their thinking while reading aloud. Eight teachers were selected for coaching related to think alouds in which they exposed students to comprehension strategies that they used while reading. The achievement of their students was compared with the achievement of students whose teachers participated in the ongoing professional development but who were not coached. Results indicate that the coached teachers changed their instructional practices and that student achievement improved as a result. 

Fisher, D., Frey, N., & Lapp, D. (2011). Coaching middle-level teachers to think aloud improves comprehension instruction and student reading achievement. The Teacher Educator, 46(3), 231-243.

Active learning increases student performance in science, engineering, and mathematics

This meta-analysis examined the impact of lecturing as compared to active methods of instruction on learning and course performance. The effect sizes indicate that on average, student performance on examinations and concept inventories increased by 0.47 SDs under active learning (n = 158 studies), and that the odds ratio for failing was 1.95 under traditional lecturing (n = 67 studies).

Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences, 111(23), 8410-8415.

The Principal Internship: How Can We Get It Right?

In this report, the Southern Regional Education Board evaluates some 60 internship programs within its states and finds them lacking in a number of ways, including failing to provide real experiences in leadership. It urges policymakers, universities and school districts to create apprenticeships that better prepare aspiring principals for the demands they will face.

Fry, B., Bottoms, G., & O’neill, K. (2005). The principal internship: How can we get it right. Atlanta, GA: Southern Regional Education Board.

How Principals Learn to Lead: The Comparative Influence of On-the-Job Experiences, Administrator Credential Programs, and the ISLLC Standards in the Development of Leadership Expertise among Urban Public School Principals

This study examines the influence of administrator credential programs, on-the-job experiences, and the standards in the development of urban public school principals.

Fultz, M. Assessing the Relationship Between Administrator Preparation Programs and Job Performance.

Assisting Students Struggling with Mathematics: Response to Intervention (RtI) for Elementary and Middle Schools

The What Works Clearinghouse (WWC) guide recommends practices to be in the repertoire of mathematics instructors for maximizing the effectiveness of mathematics instruction.

Gersten, R., Beckman S., Clarke B., Foegen A., Marsh L., Star J., and Witzel B. (2009). Assisting Students Struggling with Mathematics: Response to Intervention (RtI) for Elementary and Middle Schools (NCEE 2009-4060) Washington D.C.: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.

Effects of quantity of instruction on time spent on learning and achievement.

This article evaluates the extent to which quantity of instruction influences time spent on self‐
study and achievement. The results suggest that time spent on self‐study is primarily a function of the degree of time allocated to instruction. 

Gijselaers, W. H., & Schmidt, H. G. (1995). Effects of quantity of instruction on time spent on learning and achievement. Educational Research and Evaluation1(2), 183-201.

When Evidence-based Literacy Programs Fail.

This study examines the implementation of Leveled Literacy Intervention (LLI) for struggling readers that had been proven to work in early grades. The findings highlight the importance of considering context and implementation, in addition to evidence of effectiveness, when choosing an intervention program. Not only do schools need to adopt programs supported by evidence, but equally educators need to implement them consistently and effectively if students are to truly benefit from an intervention.

Gonzalez, N. (2018). When evidence-based literacy programs fail. Phi Delta Kappan, 100(4), 54–58. https://doi.org/10.1177/0031721718815675

Academic engagement: Current perspectives on research and practice.

A brief perspective is offered on the development and validation of one enabler—engagement in academic responding—and recent findings are provided of an effort to bridge the gap between research and practice by employing this knowledge in Title 1 elementary schools to improve instruction.

Greenwood, C. R., Horton, B. T., & Utley, C. A. (2002). Academic engagement: current perspectives in research and practice. School Psychology Review31(3).

Adolescent trust in teachers: Implications for behavior in the high school classroom

This study examined teachers' relational approach to discipline as a predictor of high school students' behavior and their trust in teacher authority. 

Gregory, A., & Ripski, M. B. (2008). Adolescent trust in teachers: Implications for behavior in the high school classroom. School Psychology Review37(3), 337.

Research on Mathematics Instruction with Students with Significant Cognitive Disabilities: Has Anything Changed?

This research examines mathematics instruction for learners of significant cognitive disabilities. This study builds on the previous meta-analysis by Browder et al. (2008) and has added an additional 29 studies. The purpose of this literature review was to identify research of teaching mathematics skills published since 2006 and to evaluate the evidence of instructional practices used in these studies. The review also attempts to examine if any progress has been made in implementing five strands of mathematics instruction identified in the National Council for Teachers of Mathematics (NCTM; (2000) recommendations. The five strands for effective instruction of mathematics are: (1) Number and Operations, (2) Algebra, (3) Geometry, (4) Measurement, (5) Data Analysis and Probability. The criteria for quality of research developed by Horner and colleagues for single subject designed research was used to review the studies (Horner et al., 2005). These standards require that to be included in this review a minimum of five single-case studies must be conducted by a minimum of three different researchers across a minimum of three different geographical regions with no less than 20 participants be required for a practice to be considered evidence-based. The data from both reviews were combined as well as they were compared. The results show more studies since 2008 taught skills from Number and Operations, Geometry, and Algebra. Additionally, the study found that the teaching of Measurement decreased and Data Analysis and Probability remained unchanged. The systematic analysis conducted by the study of specific instructional practices found systematic instruction, in vivo instruction, system of least prompts strategy, constant time delay strategy, and task-analytic instruction met criteria for being considered evidence-based practices for teaching mathematics to learners with significant cognitive disabilities.

Hudson, M. E., Rivera, C. J., & Grady, M. M. (2018). Research on Mathematics Instruction with Students with Significant Cognitive Disabilities: Has Anything Changed?. Research and Practice for Persons with Severe Disabilities, 1540796918756601.

Four Domains for Rapid School Improvement: An Implementation Framework. The Center on School Turnaround.

The National Implementation Research Network (NIRN) and the National Center for School Turnaround published the “Four Domains for Rapid School Improvement: An Implementation Framework” as a companion to the Center for School Turnaround’s publication of “The Four Domains for Rapid School Improvement: A Systems Framework“. This paper describes “how” to effectively implement lasting school improvement initiatives that maximize leadership, develop talent, amplify instructional transformation, and shift the culture.

Jackson, K., R., Fixsen, D., and Ward, C. (2018). Four Domains for Rapid School Improvement: An Implementation Framework. The Center on School Turnaround.

Assessing the cost of instructional coaching.

this study presents and apply a framework for measuring the cost of coaching programs to 3 schools. Then the study discusses strategies for reducing the average cost of instructional coaching. Finally, it estimates the costs of traditional approaches to professional development for comparison with instructional coaching.

Knight, D. S. (2012). Assessing the cost of instructional coaching. Journal of Education Finance, 52-80.

Instructional coaching

This article discusses instructional coaching as well as the eight factors that can increase the likelihood that coaching will be a real fix for a school. 

Knight, J. (2006). Instructional Coaching. School Administrator63(4), 36.

Using in-service and coaching to increase teachers’ accurate use of research-based strategies

Increasing the accurate use of research-based practices in classrooms is a critical issue. Professional development is one of the most practical ways to provide practicing teachers with training related to research-based practices. This study examined the effects of in-service plus follow-up coaching on first grade teachers’ accurate delivery of three research-based strategies during math instruction. Teachers were trained to use a combination of whole-class instruction strategies, including model-lead-test for introducing new concepts and correcting errors, choral responding, and response cards. Results indicated that all teachers improved their delivery of the strategies after the in-service, with a second level of growth achieved after coaching. Improvements also generalized to untrained math sessions. Teachers reported very high levels of satisfaction with the training model.

Kretlow, A. G., Cooke, N. L., & Wood, C. L. (2012). Using in-service and coaching to increase teachers’ accurate use of research-based strategies. Remedial and Special Education, 33, 348-361.

Using in-service and coaching to increase kindergarten teachers' accurate delivery of group instructional units

Early intervention is key to preventing academic failure and referral to special education. General educators are responsible for providing primary instruction for students at risk for failure; however, the training they receive related to specific instructional strategies for these students is often insufficient (e.g., 1-day workshops). Alternative forms of professional development that include a combination of in-service and follow-up support have shown more promise in promoting changes in teaching behaviors. This study examined the effects of in-service support plus coaching on kindergarten teachers' accurate delivery of group instructional units in math. Teachers were trained to use a combination of whole-class instruction strategies, including model-lead-test for introducing new concepts and correcting errors, choral responding, and response cards. Results indicated that all teachers improved their delivery of instruction after the in-service training, with a second level of growth achieved after coaching. Teachers also reported high levels of satisfaction using the strategies. 

Kretlow, A. G., Wood, C. L., & Cooke, N. L. (2011). Using in-service and coaching to increase kindergarten teachers' accurate delivery of group instructional units. The Journal of Special Education, 44,234-246.

 

A Theory-Based Meta-Analysis of Research on Instruction.

This research synthesis examines instructional research in a functional manner to provide guidance for classroom practitioners.

Marzano, R. J. (1998). A Theory-Based Meta-Analysis of Research on Instruction.

 

More Than Content: The Persistent Cross-Subject Effects of English Language Arts Teachers’ Instruction

This paper investigates the long-term impact of teachers' value-added effects on student learning over multiple years and across subject areas. The study finds that the durable effects of English Language Arts (ELA) teachers are more likely to generalizable across subjects than the instructional effects of math teachers.

Master, B., Loeb, S., & Wyckoff, J. (2017). More Than Content: The Persistent Cross-Subject Effects of English Language Arts Teachers’ Instruction. Educational Evaluation and Policy Analysis, 0162373717691611.

Promoting Educator Effectiveness: The Effects of Two Key Strategies

The National Center for Education Evaluation, a division of the Institute of Education Sciences has released a new research brief that evaluated two strategies for improving educator effectiveness as measured by improvements in student outcomes.  The two strategies evaluated were performance feedback to educators about several dimensions of their performance for a period of two years and a pay-for-performance system that was in place for four years.  In the performance feedback project teachers were given feedback four times per year on their classroom practices and principals received feedback two times per year.  The impact on student outcomes were small. The pay-for-performance study teachers were eligible for performance bonuses based on their ratings across multiple dimensions of their performance.  The students in the pay-for-performance schools outperformed the students in the control group schools in both math and reading. The overall benefit of the gains by the students in the pay-for-performance schools was estimated to be 3-4 weeks.  Again, this is a relatively small impact. It was noted that the quality of implementation may have reduced the impact of the two projects.

National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences (March 2018).  Promoting Educator Effectiveness: The Effects of Two Key Strategies.

This study is a meta-analysis to identify the subject matter practice elements required to effectively teach reading to children

This study is a meta-analysis to identify the subject matter practice elements required to effectively teach reading to children

National Reading Panel (US), National Institute of Child Health, & Human Development (US). (2000). Report of the national reading panel: Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction: Reports of the subgroups. National Institute of Child Health and Human Development, National Institutes of Health.

Taking Charge of Principal Preparation-A Guide to NYC Leadership Academy's Aspiring Principals Program

This report provides an overview of NYCLA’s flagship principal preparation program. Intended to help others involved in principal preparation think through important elements of principal preparation, including candidate selection, developing experiential learning opportunities, and funding, staffing and sustaining the program, the guide shares NYCLA’s successes and lessons learned during the 11 years we have delivered the Aspiring Principals Program in New York City, as well as through our work with various state and district partners nationally to adapt the program.

NYC Leadership Academy (2014). Taking Charge of Principal Preparation-A Guide to NYC Leadership Academy's Aspiring Principals Program.  Retrieved from http://www.nycleadershipacademy.org/news-and-resources/tools-and-publications/pdfs/app-guide-full-guide.

11 Million Days Lost: Race, Discipline, and Safety at U.S. Public Schools: Part I

This descriptive summary is one of the first reviews to examine the number of days of “lost instruction” resulting from student suspensions. The study examines the total number of days lost nationwide, disparities among different student subgroups, and differences across individual states. The impact of loss of instruction due to suspensions has a lifelong impact on students, including: lower graduation rates (Rumberger and Losen, 2017), increased involvement in the juvenile justice system (Mowicki, 2018), and arrests as adults Rosenbaum (2018).

 

Russell W. Rumberger and Daniel J.Losen, The Hidden Cost of California’s Harsh School Discipline, The Civil Rights Project at UCLA, (2017) Retrieved from http://www.schooldisciplinedata.org/ccrr/docs/CostofSuspensionReportFinal.pdf

Janet Rosenbaum (2018). Educational and Criminal Justice Outcomes 12 Years After School Suspension. Youth & Society.

Jacqueline M. Mowicki, Discipline Disparities for Black Students, Boys and Students with Disabilities, GAO (March 2018). Retrieved from http://www.gao.gov/assets/700/690828.pdf

The Impacts of Reading Recovery at Scale: Results From the 4-Year i3 External Evaluation

A recent large-scale evaluation of Reading Recovery, a supplemental reading program for young struggling readers, supports previous research that found it to be effective.  In a 4 year, federally funded project, almost 3,500 students in 685 schools found that generally students benefitted from the intervention. Students receiving Reading Recovery receive supplemental services in a 1:1 instructional setting for 30 minutes 5 days a week from an instructor trained in Reading Recovery.  In the study reported here, students who received Reading Recovery had effect sizes of .35-.37 relative to a control group across a number of measures of reading.  These represent moderate effect sizes and account for about a 1.5 month increase in skill relative to the control group.  Even though the research supports the efficacy of the intervention, it also raises questions about its efficiency.  The schools that participated in the study served about 5 students and the estimated cost per student has ranged from $2,000-$5,000.  These data raise questions about the wisdom of spending this much money per student for growth of about a month and a half.

Sirinides, P., Gray, A., & May, H. (2018). The Impacts of Reading Recovery at Scale: Results From the 4-Year i3 External Evaluation. Educational Evaluation and Policy Analysis, 0162373718764828.

What One Hundred Years of Research Says About the Effects of Ability Grouping and Acceleration on K–12 Students’ Academic Achievement: Findings of Two Second-Order Meta-Analyses

This paper analyzed the results of research on the effects of ability grouping and acceleration on students' academic achievement. Nineteen meta-analyses were met criteria for inclusion for the review. Results were found for improved academic achievement within-class grouping, cross-grade grouping by subject, and grouping for the gifted. No positive effects were identified for between-class grouping. The results were consistent regardless of whether students were high, medium, or low achievers. The study found acceleration appeared to have a positive, moderate, and statistically significant impact on students’ academic achievement.

Steenbergen-Hu, S., Makel, M. C., & Olszewski-Kubilius, P. (2016). What One Hundred Years of Research Says About the Effects of Ability Grouping and Acceleration on K–12 Students’ Academic Achievement: Findings of Two Second-Order Meta-Analyses. Review of Educational Research, 86(4), 849-899.

A Meta-Analysis of Direct Instruction

A soon to be published meta-analysis of Direct Instruction (DI) curricula that reviews research on DI curricula between 1966-2016 reports that DI curricula produced moderate to large effect sizes across the curriculum areas reading, math, language, and spelling.  The review is notable because it reviews a much larger body of DI research than has occurred in the past and covers a wide range of experimental designs (from single subject to randomized trials).  328 studies were reviewed and almost 4,000 effects were considered.  Given the variability in research designs and the breadth of the effects considered, it suggests that DI curricula produce robust results.  There was very little decline during maintenance phases of the study and greater exposure to the curricula resulted in greater effects.

Stockard, J., Wood, T. W., Coughlin, C. & Khoury, C. R. (in press), Review of Educational Research.  DOI: 10.3102/0034654317751919

 

Comparing and validating methods of reading instruction using behavioural and neural findings in an artificial orthography.

The results of this study confirm that early literacy instruction is most effective when focused on print-to-sound relationships (phonics) rather than on meaning. The benefits of print-to-sound training were found to be superior to print-to-meaning training for these reasons: (a) Reading aloud trained words learned phonetically was faster and more accurate, (b) generalization in reading aloud untrained words was faster, and (c) comprehension of written words was more accurate earlier in learning.

Taylor, J. S. H., Davis, M. H., & Rastle, K. (2017, April 20). Comparing and validating methods of reading instruction using behavioural and neural findings in an artificial orthography. Journal of Experimental Psychology: General. Advance online publication.

Live webcam coaching to help early elementary classroom teachers provide effective literacy instruction for struggling readers: The Targeted Reading Intervention

This study evaluated whether the Targeted Reading Intervention (TRI), a classroom teacher professional development program delivered through webcam technology literacy coaching, could provide rural classroom teachers with the instructional skills to help struggling readers progress rapidly in early reading.

Vernon-Feagans, L., Kainz, K., Hedrick, A., Ginsberg, M., & Amendum, S. (2013). Live webcam coaching to help early elementary classroom teachers provide effective literacy instruction for struggling readers: The Targeted Reading Intervention. Journal of Educational Psychology105(4), 1175.

Productive teaching

This literature review examines the impact of various instructional methods

Walberg H. J. (1999). Productive teaching. In H. C. Waxman & H. J. Walberg (Eds.) New directions for teaching, practice, and research (pp. 75-104). Berkeley, CA: McCutchen Publishing.

Meta-Analysis of the Effects of Academic Interventions and Modifications on Student Behavior Outcomes

This meta-analysis of single case designed studies examines the effect of academic interventions on student behavior. The academic interventions examined included modifying task difficulty, instruction in reading, mathematics, or writing and contingent reinforcement for academic performance. The study concluded that these interventions produced positive effects on student behavior issues observed in the classroom. The effects were observed to have a moderate effect size ranging from 0.42 to 0.64. The effects were stronger for increasing student time on task than for reducing disruptive behavior, but both showed positive impacts. This research strengthens the available evidence that well-designed instruction is effective component in creating an effective classroom climate.

Warmbold-Brann, K., Burns, M. K., Preast, J. L., Taylor, C. N., & Aguilar, L. N. (2017). Meta-Analysis of the Effects of Academic Interventions and Modifications on Student Behavior Outcomes. School Psychology Quarterly. DOI: 10.1037/spq0000207

A meta-analysis of the effects of direct instruction in special education

Studies of the effectiveness of Direct Instruction programs with special education students 
were examined in a meta-analysis comparison. To be included, the outcomes had to be 
compared with outcomes for some other treatment to which students were assigned prior to 
any interventions. Not one of 25 studies showed results favoring the comparison groups. 
Fifty-three percent of the outcomes significantly favored DI with an average magnitude of 
effect of. 84 standard deviation units. The effects were not restricted to a particular handicapping condition, age group or skill area. 

White, W. A. T. (1988). A meta-analysis of the effects of direct instruction in special education. Education and Treatment of Children, 11(4), 364–374.

 

Can reading comprehension be taught

Daniel Willingham and Gail Lovette's interpretation of the effect of comprehension instruction is that it signals to students the significance of inferential thinking. Willingham and Lovette conclude that practicing inferences does not lead to increases in general inferencing for the following reasons; inferencing depends on the particular text, and whatever cognitive processes contribute to inferencing are already well practiced in oral language as we are constantly drawing inferences in daily conversation.

Willingham, D. T., & Lovette, G. (2014). Can reading comprehension be taught. Teachers College Record116, 1-3

Troubleshooting Behavioral Interventions: A Systematic Process for Finding and Eliminating Problems

This article describes a systematic process for finding and resolving problems with classroom-based behavioral interventions in schools. 

Witt, J. C., VanDerHeyden, A. M., & Gilbertson, D. (2004). Troubleshooting behavioral interventions: A systematic process for finding and eliminating problems. School Psychology Review33, 363-383.

Impacts of New Leaders on Student Achievement in Oakland
This report presents our examination of impacts of principals trained by New Leaders on student achievement in Oakland Unified School District (OUSD). For the last 10 years, New Leaders has partnered with OUSD in efforts to improve the recruitment and training of effective principals and school leaders. This report fills an important gap in the literature, as little research has been done looking at the impacts of principal training programs on student achievement.
Booker, K., Thomas, J. (2014). Impacts of New Leaders on Student Achievement in Oakland. Mathematica Policty Research
Gateways to the Principalship State Power to Improve the Quality of School Leaders
This paper analyzes state policies for principal preparation approval and certification in 16 states, eight of which are "lagging," and eight "leading" in efforts to ensure effective leaders.
Cheney, G. R., & Davis, J. (2011). Gateways to the Principalship: State Power to Improve the Quality of School Leaders. Center for American Progress.
School Principals and School Performance
This paper uses data from New York City to estimate how the characteristics of school principals relate to school performance, as measured by students' standardized exam scores and other outcomes. There is little evidence of any relationship between school performance and principal education and pre-principal work experience, but some evidence that experience as an assistant principal at the principal's current school is associated with higher performance among inexperienced principals.
Clark, D., Martorell, P., & Rockoff, J. (2009). School Principals and School Performance. Working Paper 38. National Center for Analysis of longitudinal data in Education research.
Measuring Principal Performance: How Rigorous Are Commonly Used Principal Performance Assessment Instruments?
This study reviewed eight publicly available formative assessments of principal performance, focusing on the tool’s approach, time requirement, content and construct validity, and reliability.
Condon, C., & Clifford, M. (2010). Measuring Principal Performance: How Rigorous Are Commonly Used Principal Performance Assessment Instruments? A Quality School Leadership Issue Brief. Learning Point Associates.
Developing District-wide Expertise in Leaders’ Ability to Analyze and Improve Instructional Practice
This manuscript reports on two district’s efforts to improve the instructional leadership of principals, and lead teacher coaches in collaboration with an external support provider.
Copland, M. A., & Blum, D. (2007). Developing District-wide Expertise in Leaders’ Ability to Analyze and Improve Instructional Practice.
The Nature of Leadership Development
This book provides insight into the streams that are driving leadership theory and practice today. The Nature of Leadership, Second Edition provides students with an updated and complete yet concise handbook that solidifies and integrates the vast and disparate leadership literature.
Day, D., Antonakis, J. (2012). The Nature of Leadership. SAGE.
Schools Can’t Wait: Accelerating The Redesign of University Principal Preparation Programs
This report of 22 universities principal preparation programs identifies weaknesses in the training and offers recommendations for states to address the problems.
Fry, B., et al. (2006). Schools Can’t Wait: Accelerating The Redesign of University Principal Preparation Programs. Southern Regional Education Board.
TITLE
SYNOPSIS
Balefire Labs

Balefire Labs provides an online educational app review service for mobile apps. It helps teachers and parents to find the highest quality educational apps for kids, ages 0-19 years. It uses rigorous, science-based, review criteria and publishes a detailed rubric on its site.

Best Evidence Encyclopedia

The Best Evidence Encyclopedia is a free web site created by the Johns Hopkins University School of Education's Center for Data-Driven Reform in Education (CDDRE) under funding from the Institute of Education Sciences, U.S. Department of Education. It is intended to give educators and researchers fair and useful information about the strength of the evidence supporting a variety of programs available for students in grades K-12.

The National Center on Time and Learning

The National Center on Time & Learning has been dedicated to expanding and improving learning time in school to improve student achievement.

The University of Chicago Consortium on School Research

UChicago Consortium was created in 1990 after the passage of the Chicago School Reform Act that decentralized governance of the city's public schools. Researchers at the University of Chicago joined with researchers from the school district and other organizations to form UChicago Consortium with the imperative to study this landmark restructuring and its long-term effects

What Works Clearinghouse (WWC)

The goal of the WWC is a resource for informed education decision-making. The WWC identifies evidence-based practice, program, or policy, and disseminates summary information on the WWC website.

Back to Top