Categories for Decision Making
November 14, 2019
Using Resource and Cost Considerations to Support Educational Evaluation: Six Domains. Assessing cost, along with the effectiveness of an initiative is common in public policy decision-making, but is frequently missing in education decision-making. Understanding the cost-effectiveness of an intervention is essential if educators are to maximize the impact of an intervention given limited budgets. Education is full of examples of practices, such as class-size reduction and accountability through high-stakes testing, that produce minimal results while consuming significant resources. It is vital for those making critical decisions to understand which practice is best suited to meet the needs of the school and the schools’ students that can be implemented using the available resources. The best way to do this is through the use of a cost-effectiveness analysis (CEA).
A CEA requires an accurate estimation of all added resources needed to implement the new intervention. Costs commonly associated with education interventions include; added personnel, professional development, classroom space, technology, and expenses to monitor effectiveness. The second variable essential to a CEA is the selection of a practice supported by research. In the past twenty years, a significant increase in the quality and quantity of research supporting different education practices has occurred. A CEA compares the extra expenditures required to implement a new intervention to current practices against targeted education outcomes. Examples of educational outcomes are standardized test scores, graduation rates, or student grades.
The focus of this essay is on which economic methods can complement and enhance impact evaluations. The authors propose the use of six domains to link intervention effectiveness to the best technique needed to determine which practice is the most cost-effective choice. The six domains outlined in the paper are outcomes, treatment comparisons, treatment integrity, the role of mediators, test power, and meta-analysis. This paper provides examples of how analyzing the costs associated with these domains can complement and augment practices in evaluating research in the field of education.
Citation: Belfield, C. R., & Brooks Bowden, A. (2019). Using Resource and Cost Considerations to Support Educational Evaluation: Six Domains. Educational Researcher, 48(2), 120-127.
October 24, 2019
Small class sizes for improving student achievement in primary and secondary schools: a systematic review. This Campbell Collaboration systematic review examines the impact of class size on academic achievement. The review summarizes findings from 148 reports from 41 countries. Reducing class size is viewed by many educators as an essential tool for improving student performance, and is especially popular among teachers. But smaller class sizes come at a steep cost. Education policymakers see increasing class size as a way to control education budgets. Despite the real policy and practice implications, the research on the educational effects of class‐size differences on student performance is mixed. This meta-analysis suggests, at best only, a small impact on reading achievement. The study finds a small negative effect on mathematics. Given the fact that class size reduction is minimally effective while being costly, aren’t there better solutions that are both cost-effective, benefits students, and can help teachers be successful in a very challenging profession?
Citation: Filges, T., Sonne‐Schmidt, C. S., & Nielsen, B. C. V. (2018). Small class sizes for improving student achievement in primary and secondary schools: a systematic review. Campbell Systematic Reviews, 14(1), 1-107.
October 24, 2019
State Department of Education Support for Implementation Issues Faced by School Districts during the Curriculum Adoption Process. The results of this systematic review of the websites of all 50 of the departments of education in the United States show that relatively few states provide state-created curriculum evaluation tools in the areas of English/language arts and mathematics, and only one state provides a curriculum evaluation tool that thoroughly addresses issues of implementation. In the area of English/language arts, the implementation issue most commonly addressed is fit of an instructional program with the district. Evidence demonstrating the effectiveness of a curriculum and the district’s capacity to effectively implement a curriculum are the next two most frequently addressed implementation-related issues. In the area of mathematics, fit with the district is also the most commonly addressed implementation-related issue. The next two most frequently addressed implementation-related issues are supports for the personnel implementing the curriculum and the capacity of the district to successfully implement. Only one state provided a state-created evaluation tool that thoroughly addressed all aspects of implementation as defined by The Hexagon Tool. Interestingly, this tool was generic. It was not designed to be used with English/language arts or mathematics curricula specifically, but with a variety of innovations that districts may consider adopting.
Citation: Rolf, R., R. (2019). State Department of Education Support for Implementation Issues Faced by School Districts during the Curriculum Adoption Process. Oakland, CA: The Wing Institute. https://www.winginstitute.org/student-research-2019.
September 27, 2019
Visible Learning Insights. This book by John Hattie and Klaus Zierer is written for teachers, education researchers, and anyone interested in the latest research on the efficacy of education practices. This research offers an overview of 1,400 meta-analyses and continues to build on the work John Hattie began with his book, Visible Learning, published over a decade ago that provided educators with a synthesis of 800 meta-analyses.
Citation: Hattie, J., & Zierer, K. (2019). Visible Learning Insights. Routledge.
September 26, 2019
Evidence-Based Practices in a Changing World: Reconsidering the Counterfactual in Educational Research. Much of current education research relies on experimental and quasi-experimental designs to establish causality and to ascertain the effectiveness of practices. These research designs rely on a comparison between a treatment group and a control group. Current methods assume the population from which control groups are unchanging in behavior or performance. This paper challenges this notion and finds that populations and study samples often change over time and sometimes do so dramatically. The authors examine data from 5 randomized control trials of the efficacy of Kindergarten Peer-Assisted Learning Strategies, a supplemental, peer-mediated reading program. The paper finds a significant increase in the performance of control students over time, suggesting the need for a more nuanced understanding of current research practices in the identification of evidence-based practices.
Citation: Lemons, C. J., Fuchs, D., Gilbert, J. K., & Fuchs, L. S. (2014). Evidence-based practices in a changing world: Reconsidering the counterfactual in education research. Educational Researcher, 43(5), 242-252.
July 29, 2019
Answering the Bell: High School Start Times and Student Academic Outcomes. Research in the area of health and sleep has encouraged educators and policymakers to look to delaying school starting times as an intervention with the potential to improve achievement and other relevant student outcomes. At this time, studies conducted on starting school days at a later time show mixed results. Although, a sufficient number of studies exist to suggest that moving back the start time of school can contribute to improving lagging student performance. This research finds starting school later is associated with reduced suspensions and higher course grades. These studies suggest disadvantaged students may especially benefit from delayed starting times. This study attempts to fill in the research gap on the topic of later start times as much of the earlier research has been conducted using small sample sizes. To increase the sample size needed to confirm previous research, Bastin and Fuller use statewide student-level data from North Carolina to estimate start time effects for all students and traditionally disadvantaged students. Statewide achievement results were mixed, with positive and negative associations found between start times and high school students’ test scores. Bastin and Fuller counsel for further research to increase confidence that later start times predictably produce desired outcomes. Studies of sufficient rigor, using multiple populations, and across different settings are required to address remaining issues and possible unintended consequences associated with changing start times.
Citation: Bastian, K. C., & Fuller, S. C. (2018). Answering the Bell: High School Start Times and Student Academic Outcomes. AERA Open, 4(4), 2332858418812424.
May 31, 2019
The Condition of Education 2019 Newly Released. The Condition of Education 2019 is produced by The Institute for Education Sciences (IES) National Center for Education Statistics (NCES). This annual publication is one of the best ongoing sources for tracking and analyzing important developments and trends in education over time using the latest available data. It is essential that education systems have feedback mechanisms at the macro level. The 2019 report presents 50 key indicators on important topics and trends in U.S. education. These descriptive and performance indicators focus on family characteristics, such as educational attainment and economic outcomes; participation in education at all levels (preprimary, elementary, secondary and post secondary); and several contextual aspects of education, including international comparisons, at both the elementary and secondary education level and the postsecondary education level. Each year the report “spotlights” new supplemental indicators to provide more in-depth analyses on critical topics. In 2019 the spotlights were on: “Early Childhood Care Arrangements”; “Choices and Costs Characteristics of Public School Teachers Who Completed Alternative Route to Certification Programs”; and “Trends in Student Loan Debt for Graduate School Completers.”
Characteristics of Public School Teachers Who Completed Alternative Route to Certification Programs”; and “Trends in Student Loan Debt for Graduate School Completers.”
Citation: McFarland, J., Hussar, B., Zhang, J., Wang, X., Wang, K., Hein, S., Diliberti, M., Forrest Cataldi, E., Bullock Mann, F., and Barmer, A. (2019). e Condition of Education 2019 (NCES 2019-144). U.S. Department of Education. Washington, DC: National Center for Education Statistics. Retrieved [date] from https://nces.ed.gov/ pubsearch/pubsinfo.asp?pubid=2019144.
May 8, 2019
Active Student Responding (ASR) is a powerful set of low cost strategies teachers can use to improve student achievement. ASR occurs when a student makes a response by answering questions or responding in a variety of ways that communicates the student’s understanding of the content being taught during the lesson. The more opportunities the student has to respond, the increased likelihood the student is learning. Increasing active responses allows teachers to rapidly assess performance. As opportunities to respond increase so does opportunities for praise and corrective feedback that results in accelerated learning. Attending and being on-task are insufficient ways for teachers to know if learning is occurring. For a teacher to know if a student is actually learning a written, action, or oral response is required. The more opportunities to respond the more quickly students master lessons. ASR strategies are designed to engage all students regardless of class size and ASR avoids the common problem of having only high achievers answer questions while low achievers remain silent, thus escaping detection. Examples of ASR strategies include; guided notes, response slates, response cards, and choral responding.
Citation: States, J., Detrich, R. & Keyworth, R. (2019). Active Student Responding (ASR) Overview.Oakland, CA: The Wing Institute. https://www.winginstitute.org/instructional-delivery-student-respond
April 16, 2019
Performance feedback is a practice used to improve performance. Principals give feedback to teachers to clarify expectations and to provide information for increasing administrative, instructional, behavior management, and personal competency skills. More than seven meta-analyses conducted since 1980 support feedback as one of the most powerful tools for improving performance. To deliver useful feedback, principals need current and accurate information on student performance and a teacher’s instructional skills. Research finds that principals depend on unreliable sources of data such as “walk-throughs,” brief informal observations that provide snapshots of classroom activities but are not designed for performance improvement. Principals should replace traditional walk-throughs with more effective feedback practices, such as coaching, that are better suited to improving specific teaching skills. For the best results, feedback must meet these four conditions: (1) It is objective, reliable, measurable, and specific; (2) it provides information about what was done well, what needs improvement, and how to improve; (3) it is delivered frequently and immediately following performance; and (4) it is about performance rather than personal characteristics.
Citation: Cleaver, S., Detrich, R. & States, J. (2019). Overview of Performance Feedback. Oakland, CA: The Wing Institute. https://www.winginstitute.org/teacher-evaluation-feedback.
April 15, 2019
Proceed With Caution: Using Web-Based Resources for Instructing Students With and at Risk for EBD. This article examines issues relating to the use of websites popular with educators. Today educators often rely on social media platforms such as Pinterest and Teachers Pay Teachers for information for solving problems encountered in the classroom. These sites can offer teachers helpful information, but also come with potential risks that educators need to consider when utilizing such resources. This article offers guidelines for maximizing the usefulness of such sites and for avoiding many of the pitfall educators may face. The authors suggest educators first identify and learn the critical elements of effective practices from trustworthy sources and then use sites such as Pinterest and Teachers Pay Teachers to facilitate implementation.
Citation: Beahm, L. A., Cook, B. G., & Cook, L. (2019). Proceed With Caution: Using Web-Based Resources for Instructing Students With and at Risk for EBD. Beyond Behavior, 28(1), 13-20.