Categories for Decision Making
January 23, 2020
Why is this question important? Given the limited resources that are available for the education of children, it is important to select interventions that have the greatest impact we can afford. Using Stuart Yeh’s effectiveness cost ratio formula, a rough comparison can be drawn comparing class size reduction with other educational interventions.
Citation: Yeh, S. S. (2007). The Cost-Effectiveness of Five Policies for Improving Student Achievement, American Journal of Evaluation, 28(4), 416-436.
January 15, 2020
Overview of Value-Added Research in Education: Reliability, Validity, Efficacy, and Usefulness. Value-added modeling (VAM) is a statistical approach that provides quantitative performance measures for monitoring and evaluating schools and other aspects of the education system. VAM comprises a collection of complex statistical techniques that use standardized test scores to estimate the effects of individual schools or teachers on student performance. Although the VAM approach holds promise, serious technical issues have been raised regarding VAM as a high-stakes instrument in accountability initiatives. The key question remains: Can VAM scores of standardized test scores serve as a proxy for measuring teaching quality? To date, research on the efficacy of VAM is mixed. There is a body of research that supports VAM, but there is also a body of studies suggesting that model estimates are unstable over time and subject to bias and imprecision. A second issue with VAM is the sole use of standardized tests as a measure of student performance. Despite these valid concerns, VAM has been shown to be valuable in performance improvement efforts when used cautiously in combination with other measures of student performance such as end-of-course tests, final grades, and structured classroom observations.
Citation: Cleaver, S., Detrich, R. & States, J. (2020). Overview of Value-Added Research in Education: Reliability, Validity, Efficacy, and Usefulness. Oakland, CA: The Wing Institute. https://www.winginstitute.org/staff-value-added.
December 16, 2019
On the Reality of Dyslexia. This paper assesses research on the topic of dyslexia. Willingham’s piece is in response to comments made by literacy researcher, Dick Allington, in which he questions the legitimacy of the label, dyslexia. Answering this question is more than an academic exercise as having a clearer understanding of dyslexia is crucial if educators are to understand why 10% of students struggle to master reading, the skill essential to success in academic learning. Willingham highlights the etiology of the disorder, and he concludes that the ability to read is the product of the home environment, instruction at school, and genetics within the child. Dyslexia is a problem in the child’s ability to successfully master the skills of reading and is closely related to fluency in language. Dyslexia is not like measles in which you are ill, or you aren’t. Dyslexia is more like high blood pressure where individuals fall on a bell curve. Falling somewhere on the bell curve is supported by the hypothesis that the disorder is the complex interaction between multiple causes. Although it does not have a single source, dyslexia is successfully remediated through evidence-based language and reading instruction.
Citation: Willingham, D. (2019). On the Reality of Dyslexia. Charlottesville, VA.http://www.danielwillingham.com/daniel-willingham-science-and-education-blog/on-the-reality-of-dyslexia?utm_source=feedburner&utm_medium=email&utm_campaign=Feed%3A+nbspDanielWillingham-DanielWillinghamScienceAndEducationBlog+%28Daniel+Willingham%27s+Science+and+Education+Blog%29.
Link: On the Reality of Dyslexia
November 14, 2019
Using Resource and Cost Considerations to Support Educational Evaluation: Six Domains. Assessing cost, along with the effectiveness of an initiative is common in public policy decision-making, but is frequently missing in education decision-making. Understanding the cost-effectiveness of an intervention is essential if educators are to maximize the impact of an intervention given limited budgets. Education is full of examples of practices, such as class-size reduction and accountability through high-stakes testing, that produce minimal results while consuming significant resources. It is vital for those making critical decisions to understand which practice is best suited to meet the needs of the school and the schools’ students that can be implemented using the available resources. The best way to do this is through the use of a cost-effectiveness analysis (CEA).
A CEA requires an accurate estimation of all added resources needed to implement the new intervention. Costs commonly associated with education interventions include; added personnel, professional development, classroom space, technology, and expenses to monitor effectiveness. The second variable essential to a CEA is the selection of a practice supported by research. In the past twenty years, a significant increase in the quality and quantity of research supporting different education practices has occurred. A CEA compares the extra expenditures required to implement a new intervention to current practices against targeted education outcomes. Examples of educational outcomes are standardized test scores, graduation rates, or student grades.
The focus of this essay is on which economic methods can complement and enhance impact evaluations. The authors propose the use of six domains to link intervention effectiveness to the best technique needed to determine which practice is the most cost-effective choice. The six domains outlined in the paper are outcomes, treatment comparisons, treatment integrity, the role of mediators, test power, and meta-analysis. This paper provides examples of how analyzing the costs associated with these domains can complement and augment practices in evaluating research in the field of education.
Citation: Belfield, C. R., & Brooks Bowden, A. (2019). Using Resource and Cost Considerations to Support Educational Evaluation: Six Domains. Educational Researcher, 48(2), 120-127.
October 24, 2019
Small class sizes for improving student achievement in primary and secondary schools: a systematic review. This Campbell Collaboration systematic review examines the impact of class size on academic achievement. The review summarizes findings from 148 reports from 41 countries. Reducing class size is viewed by many educators as an essential tool for improving student performance, and is especially popular among teachers. But smaller class sizes come at a steep cost. Education policymakers see increasing class size as a way to control education budgets. Despite the real policy and practice implications, the research on the educational effects of class‐size differences on student performance is mixed. This meta-analysis suggests, at best only, a small impact on reading achievement. The study finds a small negative effect on mathematics. Given the fact that class size reduction is minimally effective while being costly, aren’t there better solutions that are both cost-effective, benefits students, and can help teachers be successful in a very challenging profession?
Citation: Filges, T., Sonne‐Schmidt, C. S., & Nielsen, B. C. V. (2018). Small class sizes for improving student achievement in primary and secondary schools: a systematic review. Campbell Systematic Reviews, 14(1), 1-107.
October 24, 2019
State Department of Education Support for Implementation Issues Faced by School Districts during the Curriculum Adoption Process. The results of this systematic review of the websites of all 50 of the departments of education in the United States show that relatively few states provide state-created curriculum evaluation tools in the areas of English/language arts and mathematics, and only one state provides a curriculum evaluation tool that thoroughly addresses issues of implementation. In the area of English/language arts, the implementation issue most commonly addressed is fit of an instructional program with the district. Evidence demonstrating the effectiveness of a curriculum and the district’s capacity to effectively implement a curriculum are the next two most frequently addressed implementation-related issues. In the area of mathematics, fit with the district is also the most commonly addressed implementation-related issue. The next two most frequently addressed implementation-related issues are supports for the personnel implementing the curriculum and the capacity of the district to successfully implement. Only one state provided a state-created evaluation tool that thoroughly addressed all aspects of implementation as defined by The Hexagon Tool. Interestingly, this tool was generic. It was not designed to be used with English/language arts or mathematics curricula specifically, but with a variety of innovations that districts may consider adopting.
Citation: Rolf, R., R. (2019). State Department of Education Support for Implementation Issues Faced by School Districts during the Curriculum Adoption Process. Oakland, CA: The Wing Institute. https://www.winginstitute.org/student-research-2019.
September 27, 2019
Visible Learning Insights. This book by John Hattie and Klaus Zierer is written for teachers, education researchers, and anyone interested in the latest research on the efficacy of education practices. This research offers an overview of 1,400 meta-analyses and continues to build on the work John Hattie began with his book, Visible Learning, published over a decade ago that provided educators with a synthesis of 800 meta-analyses.
Citation: Hattie, J., & Zierer, K. (2019). Visible Learning Insights. Routledge.
September 26, 2019
Evidence-Based Practices in a Changing World: Reconsidering the Counterfactual in Educational Research. Much of current education research relies on experimental and quasi-experimental designs to establish causality and to ascertain the effectiveness of practices. These research designs rely on a comparison between a treatment group and a control group. Current methods assume the population from which control groups are unchanging in behavior or performance. This paper challenges this notion and finds that populations and study samples often change over time and sometimes do so dramatically. The authors examine data from 5 randomized control trials of the efficacy of Kindergarten Peer-Assisted Learning Strategies, a supplemental, peer-mediated reading program. The paper finds a significant increase in the performance of control students over time, suggesting the need for a more nuanced understanding of current research practices in the identification of evidence-based practices.
Citation: Lemons, C. J., Fuchs, D., Gilbert, J. K., & Fuchs, L. S. (2014). Evidence-based practices in a changing world: Reconsidering the counterfactual in education research. Educational Researcher, 43(5), 242-252.
July 29, 2019
Answering the Bell: High School Start Times and Student Academic Outcomes. Research in the area of health and sleep has encouraged educators and policymakers to look to delaying school starting times as an intervention with the potential to improve achievement and other relevant student outcomes. At this time, studies conducted on starting school days at a later time show mixed results. Although, a sufficient number of studies exist to suggest that moving back the start time of school can contribute to improving lagging student performance. This research finds starting school later is associated with reduced suspensions and higher course grades. These studies suggest disadvantaged students may especially benefit from delayed starting times. This study attempts to fill in the research gap on the topic of later start times as much of the earlier research has been conducted using small sample sizes. To increase the sample size needed to confirm previous research, Bastin and Fuller use statewide student-level data from North Carolina to estimate start time effects for all students and traditionally disadvantaged students. Statewide achievement results were mixed, with positive and negative associations found between start times and high school students’ test scores. Bastin and Fuller counsel for further research to increase confidence that later start times predictably produce desired outcomes. Studies of sufficient rigor, using multiple populations, and across different settings are required to address remaining issues and possible unintended consequences associated with changing start times.
Citation: Bastian, K. C., & Fuller, S. C. (2018). Answering the Bell: High School Start Times and Student Academic Outcomes. AERA Open, 4(4), 2332858418812424.
May 31, 2019
The Condition of Education 2019 Newly Released. The Condition of Education 2019 is produced by The Institute for Education Sciences (IES) National Center for Education Statistics (NCES). This annual publication is one of the best ongoing sources for tracking and analyzing important developments and trends in education over time using the latest available data. It is essential that education systems have feedback mechanisms at the macro level. The 2019 report presents 50 key indicators on important topics and trends in U.S. education. These descriptive and performance indicators focus on family characteristics, such as educational attainment and economic outcomes; participation in education at all levels (preprimary, elementary, secondary and post secondary); and several contextual aspects of education, including international comparisons, at both the elementary and secondary education level and the postsecondary education level. Each year the report “spotlights” new supplemental indicators to provide more in-depth analyses on critical topics. In 2019 the spotlights were on: “Early Childhood Care Arrangements”; “Choices and Costs Characteristics of Public School Teachers Who Completed Alternative Route to Certification Programs”; and “Trends in Student Loan Debt for Graduate School Completers.”
Characteristics of Public School Teachers Who Completed Alternative Route to Certification Programs”; and “Trends in Student Loan Debt for Graduate School Completers.”
Citation: McFarland, J., Hussar, B., Zhang, J., Wang, X., Wang, K., Hein, S., Diliberti, M., Forrest Cataldi, E., Bullock Mann, F., and Barmer, A. (2019). e Condition of Education 2019 (NCES 2019-144). U.S. Department of Education. Washington, DC: National Center for Education Statistics. Retrieved [date] from https://nces.ed.gov/ pubsearch/pubsinfo.asp?pubid=2019144.