Schools in the United States now spend more than $2 billion each year on education technology. But what are schools getting in return for this significant investment in technology learning? Robert Slavin examines the results from five studies designed to answer this question.
Slavin, R. (2019). A Powerful Hunger for Evidence-Proven Technology. Baltimore, MD: Robert Slavin’s Blog. https://robertslavinsblog.wordpress.com/2019/11/14/a-powerful-hunger-for-evidence-proven-technology/.
This review of the research on secondary reading programs focuses on 69 studies that used random assignment (n=62) or high-quality quasi-experiments (n=7) to evaluate outcomes of 51 programs on widely accepted measures of reading.The study found programs using one-to-one and small-group tutoring (+0.14 to +0.28 effect size), cooperative learning (+0.10 effect size), whole-school approaches including organizational reforms such as teacher teams (+0.06 effect size), and writing-focused approaches (+0.13 effect size) showed positive outcomes. Individual approaches in a few other categories also showed positive impacts. The findings are important suggesting interventions for secondary readers to improve struggling student’s chances of experiencing greater success in high school and better opportunities after graduation.
Citation: Baye, A., Lake, C., Inns, A. & Slavin, R. E. (2018, January). A Synthesis of Quantitative Research on Reading Programs for Secondary Students. Baltimore, MD: Johns Hopkins University, Center for Research and Reform in Education.
The purpose of this study was to evaluate the efficacy of a classroom-teacher-delivered reading intervention for struggling readers called the Targeted Reading Intervention (TRI), designed particularly for kindergarten and first-grade teachers and their struggling students in rural, low-wealth communities.
Amendum, S. J., Vernon-Faegans, L. V., & Ginsberg, M. C. (2011). The effectiveness of a technologically facilitated classroom-based early reading intervention. The Elementary School Journal, 112, 107-131.
Providing children with nutritious school meals continues to be a topic of interest in education policy. It has been argued that a healthy diet can have a positive impact on childhood obesity as well as student achievement. However, there is little empirical evidence to support this popular intervention. A working paper released by the National Bureau of Economic Research offers insights to help policy makers decide whether to select this option as a way to make a difference in children’s lives. The study, a natural experiment conducted in California public schools, uses a difference-in-difference regression statistical analysis of data from a treatment group and a control group at two or more different time periods, pre-treatment and post-treatment. The study found no evidence to support a reduction in obesity, but it did discover that introducing healthy meals was associated with a 0.036 standard deviation increase in test scores. The improved student achievement, although small, makes nutritious school lunches a viable cost-effective intervention that can boost test scores. It is important to remember that this is a correlational study and thus cannot establish a cause and effect relationship between healthy eating and student achievement.
Anderson, M. L., Gallagher, J., & Ritchie, E. R. (2017). School lunch quality and academic performance (NBER Working Paper No. 23218). Cambridge, MA: National Bureau of Economic Research.
This article discuss about study that measures the effect of offering healthier public school lunches on end of year academic test scores for public school students in California. This study focus on school-specific differences in lunch quality over-time. The study shows increasing the nutritional quality of school meals appears to be promising, cost-effective way to improve student learning.
Anderson. M. L., Gallagher, J., Ritchie. E. R. (2017). How the Quality of School Lunch Affects Students' Academic Performance. Brookings Institution. Retrieved from https://www.brookings.edu/blog/brown-center-chalkboard/2017/05/03/how-the-quality-of-school-lunch-affects-students-academic-performance/
Professional judgment is required whenever conditions are uncertain. This article provides an analysis of professional judgment and describes sources of error in decision making.
Barnett, D. W. (1988). Professional judgment: A critical appraisal. School Psychology Review., 17(4), 658-672.
A rationale and model for changing assessment efforts in schools from simple description to the integration of information from multiple sources for the purpose of designing interventions are described.
Christenson, S. L., & Ysseldyke, J. E. (1989). Scientific practitioner: Assessing student performance: An important change is needed. Journal of School Psychology, 27(4), 409-425.
This practice guide provides five recommendations for improving students’ mathematical problem solving in grades 4 through 8. The manual is geared toward teachers, math coaches, other educators, and curriculum developers who want to improve the mathematical problem solving of students.
Clearinghouse, W. W. Improving Mathematical Problem Solving in Grades 4 Through 8. U.S. Department of Education, Institute of Education Science (IES) NCEE 2012-4055.
a mini-series from School Psychology Review about Academic Enablers to Improve Student Performance: Considerations for Research and Practice.
DiPerna, J., & Elliott, S. N. (2002). Promoting academic enablers to improve student performance: Considerations for research and practice [Special issue]. School Psychology Review, 31(3).
The book discusses the analytical and policy challenges that face health systems in seeking to allocate resources efficiently and fairly. New chapters include 'Principles of economic evaluation' and 'Making decisions in healthcare'.
Drummond, M. F., Sculpher, M. J., Claxton, K., Stoddart, G. L., & Torrance, G. W. (2015). Methods for the economic evaluation of health care programmes. Oxford university press.
In 2000, the National Reading Panel identified five practice elements with a sufficient evidence base to be deemed essential for mastery of reading (National Institute of Child Health and Human Development, 2000). These elements consist of systematic teaching of phonemic awareness, phonics instruction, vocabulary, fluency, and exposure to reading comprehension strategies. This meta-analysis of 25 studies evaluates the impact of inference instruction in grades K-12. The study reported that inference instruction had an effect size d=0.58 on general comprehension and d= 0.68 on literal comprehension. These are “moderate to large” effects of instruction on general comprehension and to making inferences for both skilled and less skilled readers. The pattern differed for the literal measure, however, with skilled readers showing almost no gain but unskilled readers showing sizable gains.
Elleman, A. M. (2017). Examining the impact of inference instruction on the literal and inferential comprehension of skilled and less skilled readers: A meta-analytic review. Journal of Educational Psychology, 109(6), 761-781.
This study examines the issue of extending the school day and year. The positive effects of having more time for teaching through longer class periods, individual assistance, and tutoring sessions, students spend more time on task than do students in schools operating on a conventional schedule. Studying these schools also reveals that despite the benefits there are hefty challenges to extending the school day.
Farbman, D. and Kaplan, C. (2005). Time for a change: The promise of extended-time schools for promoting student achievement. Boston, MA: Massachusetts.
In this book, Gambrill examines the importance of critical thinking, the biases that all of us are prone to, and ways to improve our judgments.
Gambrill, E. (2006). Critical thinking in clinical practice: Improving the quality of judgments and decisions. John Wiley & Sons. Retrieved from http://books.google.com/books?hl=en&lr=&id=z8Hils1vn4kC&oi=fnd&pg=PR7&dq=Gambrill+Critical+Thinking&ots=T8-fSWM4Gk&sig=ykQYQtOXgQwfzrOhC_zghsp3__w
This workbook provides exercises for improving the decision-making of helping professionals.
Gambrill, E., & Gibbs, L. (2009). Critical thinking for helping professionals: A skills-based workbook. Oxford University Press on Demand. Retrieved from http://books.google.com/books?hl=en&lr=&id=RsITDAAAQBAJ&oi=fnd&pg=PR15&dq=Gambrill+Critical+Thinking&ots=MREzsHeI5n&sig=L5KzgI1WBdNU2O3n9tlu2e18sF4
This report: (1) investigates the declining state of the educational system in America, as measured by high school student performance in the United States and other countries; (2) identifies specific problem areas; and (3) offers multiple recommendations for improvement
Gardner, D. P. (1983). A Nation At Risk: The Imperative For Educational Reform. An Open Letter to the American People. A Report to the Nation and the Secretary of Education.
The Rand Corporation just released its report evaluating the impact of the Principal Pipeline Initiative (PPI),a project supported by the Wallace Foundation to create and implement a strategic process for school leadership talent management. This report documents what the PPI districts were able to accomplish, describing the implementation of the PPI and its effects on student achievement, other school outcomes, and principal retention.
Gates, Susan M., Matthew D. Baird, Benjamin K. Master, and Emilio R. Chavez-Herrerias, Principal Pipelines: A Feasible, Affordable, and Effective Way for Districts to Improve Schools, Santa Monica, Calif.: RAND Corporation, RR-2666-WF, 2019.
This paper discusses the effectiveness of research‐based educational approaches on
Gersten, R. (2001). Sorting out the roles of research in the improvement of practice. Learning Disabilities Research & Practice, 16(1), 45-50.
Conventional wisdom holds that heuristics and biases lead to flawded decision making. This paper makes the case that under some conditions they actually make decision-making more efficient.
Gigerenzer, G., & Brighton, H. (2009). Homo heuristicus: Why biased minds make better inferences. Topics in Cognitive Science, 1, 107-143. doi:10.1111/j.1756-8765.2008.01006.
The current paper describes how policymakers and educators might employ professional accountability to address professional practice, using examples from schools that are already doing so.
Gill, B. P., (2017). Professional Accountability as a Tool for Continuous Improvement in the Age of ESSA. Retrieved from https://aefpweb.org/sites/default/files/webform/42/professional%20accountability%20for%20AEFP.pdf
This meta-analysis examines 23 studies for student access to curriculum by assessing the gap in reading achievement between general education peers and students with disabilities (SWD). The study finds that SWDs performed more than three years below peers. The study looks at the implications for changing this pictures and why current policies and practices are not achieving the desired results.
Gilmour, A. F., Fuchs, D., & Wehby, J. H. (2018). Are students with disabilities accessing the curriculum? A meta-analysis of the reading achievement gap between students with and without disabilities. Exceptional Children. Advanced online publication. doi:10.1177/0014402918795830
This study examines adoption and implementation of the US Department of Education's new policy, the `Principles of Effectiveness', from a diffusion of innovations theoretical framework. In this report, we evaluate adoption in relation to Principle 3: the requirement to select research-based programs.
Hallfors, D., & Godette, D. (2002). Will the “principles of effectiveness” improve prevention practice? Early findings from a diffusion study. Health Education Research, 17(4), 461–470.
This paper reports on the analysis of state statutes and department of education regulations in fifty states for changes in teacher evaluation in use since the passage of No Child Left Behind Act of 2001.
Hazi, H. M., & Rucinski, D. A. (2009). Teacher evaluation as a policy target for improved student learning: A fifty-state review of statute and regulatory action since NCLB. education policy analysis archives, 17, 5.
A study of 27 promising programs reveals 8 common reasons that educational innovations fail, including disenchanted practitioners; departure of innovation supporters; lack of personnel training; disappearing funding; inadequate supervision; and lack of accountability, administrative support, and termination consequences. Innovations succeed by avoiding overload, complementing school mission, and securing board approval
Latham, G. (1988). The birth and death cycles of educational innovations. Principal, 68(1), 41-43.
This meta-analysis of behavior management strategies includes single-subject designed studies of 838 students from 22 studies for K-12 classrooms. The study finds the behavior management strategies are highly effective for improving student conduct. Interventions that used an individual or group contingency demonstrated large effects and were the most common behavior management strategies used. The study finds few studies included diverse populations other than African-American students.They also find a need to improve upon the quality of available studies on the classroom management strategies.
Long, A. C. J., Miller, F. G., & Upright, J. J. (2019). Classroom management for ethnic–racial minority students: A meta-analysis of single-case design studies. School Psychology, 34(1), 1-13. http://dx.doi.org/10.1037/spq0000305
The What Works Clearinghouse (WWC) identified five studies of NBPTS certification that both fall within the scope of the Teacher Training, Evaluation, and Compensation topic area and meet WWC group design standards.
Mathematica Policy Research (2018). What Works Clearinghouse Intervention Report: National Board for Professional Teaching Standards Certification. Washington, DC: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress (NAEP). Retrieved from https://ies.ed.gov/ncee/wwc/Docs/InterventionReports/wwc_nbpts_021318.pdf.
The National Center for Education Evaluation, a division of the Institute of Education Sciences has released a new research brief that evaluated two strategies for improving educator effectiveness as measured by improvements in student outcomes.
National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences (March 2018). Promoting Educator Effectiveness: The Effects of Two Key Strategies.
Over the past twenty years many reading interventions have been proposed. One of these, “Book Flooding”, proposes that providing an enriched environment in which books are present and readily available can improve reading. Much of the research on this topic has focused on exposing children in the early grades to storybooks. Given the greater importance on reading complex text in meeting new reading standards, this study examines the impact of book flooding of books that stress academic words and technical terms. This quasi-experimental study examines the influence of a book distribution program targeted at enhancing children’s exposure to information books. The research examined whether a flood of information books in early childhood settings could affect growth in language, content-related vocabulary, and concepts of comprehending information text. The study concludes there were no significant effects on student outcomes and that book distribution programs on their own need to be reevaluated if they are to improved student reading performance.
Neuman, S. B. (2017). The Information Book Flood: Is Additional Exposure Enough to Support Early Literacy Development?. The Elementary School Journal, 118(1), 1-27.
US President Obama has launched one of the world’s most ambitious education reform agendas. Under the heading “Race to the Top”, this agenda encourages US states to adopt internationally benchmarked standards and assessments that prepare students for success in college and the workplace: recruit, develop, reward, and retain effective teachers and principals.
Organisation for Economic Co-operation and Development (OECD). (2011). Lessons from PISA for the United States–Strong performers and successful reformers in education. OECD Publishing. doi:http://dx.doi.org/10.1787/9789264096660-en
This research synthesis examines randomized controlled trials and quasi-experimental research on the mathematics achievement outcomes for elementary school programs. The best outcomes were found for tutoring programs. The findings suggest that programs emphasizing personalization, engagement, and motivation are most impactful in elementary mathematics instruction.
Pellegrini, M., Lake, C., Inns, A, & , Slavin, R. (2018). Effective programs in elementary mathematics: A best-evidence synthesis. Best Evidence Encyclopedia. Retrieved from http://www.bestevidence.org/word/elem_math_Oct_8_2018.pdf
Effects of a 1-semester professional development (PD) intervention that included expert coaching with Head Start teachers were investigated in a randomized controlled trial with 88 teachers and 759 children.
Powell, D. R., Diamond, K. E., Burchinal, M. R., & Koehler, M. J. (2010). Effects of an early literacy professional development intervention on Head Start teachers and children. Journal of Educational Psychology, 102, 299-312.
The authors use research-based "impact modeling" to show how a strategic approach to recruiting and supporting rookie teachers could yield as much as 4.2 extra months of student learning. We provide 5 recommendations for school systems to leverage their investment in structures that provide rookie teachers with both shelter and development.
Rosenberg, D., & Miles, K.H. (2018). Growing Great Teachers: How School System Leaders Can Use Existing Resources to Better Develop, Support, and Retain New Teachers--and Improve Student Outcomes. Retrieved from https://files.eric.ed.gov/fulltext/ED593368.pdf
This report breaks out key steps in the school identification and improvement process, focusing on (1) a diagnosis of school needs; (2) a plan to improve schools; and (3) evidenced-based interventions that work.
School Intervention That Work: Targeted Support for Low-Performing Students. (2017). Alliance For Excellent Education. Retrieved from https://all4ed.org/reports-factsheets/schoolinterventions/
A recent large-scale evaluation of Reading Recovery, a supplemental reading program for young struggling readers, supports previous research that found it to be effective. In a 4 year, federally funded project, almost 3,500 students in 685 schools found that generally students benefitted from the intervention. Students receiving Reading Recovery receive supplemental services in a 1:1 instructional setting for 30 minutes 5 days a week from an instructor trained in Reading Recovery. In the study reported here, students who received Reading Recovery had effect sizes of .35-.37 relative to a control group across a number of measures of reading. These represent moderate effect sizes and account for about a 1.5 month increase in skill relative to the control group. Even though the research supports the efficacy of the intervention, it also raises questions about its efficiency. The schools that participated in the study served about 5 students and the estimated cost per student has ranged from $2,000-$5,000. These data raise questions about the wisdom of spending this much money per student for growth of about a month and a half.
Sirinides, P., Gray, A., & May, H. (2018). The Impacts of Reading Recovery at Scale: Results From the 4-Year i3 External Evaluation. Educational Evaluation and Policy Analysis, 0162373718764828.
This report, completed by the Center on Education Policy, attempts to provide an initial snapshot of the number and percentages of schools each states has identified low performing. It provides an early look at a very diverse set of guidelines. The data show a wide range of results in terms of the percentage of schools identified as low performing. The overall range is 3% to 99%, with individual states spread out fairly evenly in between. Eight states identified over 40% of their public schools as low performing, eleven states 20%–40%, fifteen states 11%–19%, and thirteen states 3%–10%. Even with the limitations of the data listed above, this data suggests inconsistent standards across states.
Stark Renter, D., Tanner, K., Braun, M. (2019). The Number of Low-Performing Schools by State in Three Categories (CSI, TSI, and ATSI), School Year 2018-19. A Report of the Center on Education Policy
Summative assessment is an appraisal of learning at the end of an instructional unit or at a specific point in time. It compares student knowledge or skills against standards or benchmarks. Summative assessment includes midterm exams, final project, papers, teacher-designed tests, standardized tests, and high-stakes tests.
States, J., Detrich, R. & Keyworth, R. (2018). Overview of Summative Assessment. Oakland, CA: The Wing Institute. https://www.winginstitute.org/assessment-summative
This meta-analysis examines the impact of 1st tier reading instruction on reading outcomes for students in grades 4-12 in an Multi-tiered System of Support (MTSS) service delivery model. 37 studies met criteria for inclusion in the meta-analysis. The study finds small, but positive effects for 1st tier reading instruction on comprehension, vocabulary, and indicates minimum evidence for struggling readers maintaining or improving reading comprehension over struggling students receiving typical instruction. Hedges’s g was used calculating effect sizes. Because of the limited number of studies examining phonics/word recognition and fluency instruction, it was not possible these critical instruction areas in this meta-analysis.
Swanson, E., Stevens, E. A., Scammacca, N. K., Capin, P., Stewart, A. A., & Austin, C. R. (2017). The impact of tier 1 reading instruction on reading outcomes for students in Grades 4–12: A meta-analysis. Reading and Writing, 30(8), 1639-1665.
In the research reported here, the authors study one approach to teacher evaluation: practice-based assessment that relies on multiple, highly structured classroom observations conducted by experienced peer teachers and administrators.
Taylor, E. S., & Tyler, J. H. (2012a). Can teacher evaluation improve teaching? Evidence of systematic growth in the effectiveness of mid-career teachers. Education Next, 12(4), 79–84. Retrieved from http://educationnext.org/can-teacher-evaluation-improve-teaching/
"The Mirage" describes the widely held perception among education leaders that they already know how to help teachers improve, and that they could achieve their goal of great teaching in far more classrooms if they just applied what they knew more widely.
TNTP. (2015). The Mirage: Confronting the truth about our quest for teacher development. Retrieved from: https://tntp.org/publications/view/the-mirage-confronting-the-truth-about-our-quest-for-teacher-development
This paper examines the types of research to consider when evaluating programs, how to know what “evidence’ to use, and continuums of evidence (quantity of the evidence, quality of the evidence, and program development).
Twyman, J. S., & Sota, M. (2008). Identifying research-based practices for response to intervention: Scientifically based instruction. Journal of Evidence-Based Practices for Schools, 9(2), 86-101.
Daniel Willingham and Gail Lovette's interpretation of the effect of comprehension instruction is that it signals to students the significance of inferential thinking. Willingham and Lovette conclude that practicing inferences does not lead to increases in general inferencing for the following reasons; inferencing depends on the particular text, and whatever cognitive processes contribute to inferencing are already well practiced in oral language as we are constantly drawing inferences in daily conversation.
Willingham, D. T., & Lovette, G. (2014). Can reading comprehension be taught. Teachers College Record, 116, 1-3
This article compares the relative cost-effectiveness of the five policies, using best-evidence estimates drawn from available data regarding the effectiveness and costs of rapid assessment, increased spending, voucher programs, charter schools, and accountability, using a conservative methodology for calculating the relative effectiveness of the rapid assessment.
Yeh, S. S. (2007). The cost-effectiveness of five policies for improving student achievement. American Journal of Evaluation, 28(4), 416-436.