Categories for Policy Initiatives
November 15, 2019
A Powerful Hunger for Evidence-Proven Technology. The technology industry and education policymakers have touted the benefits of computers in learning. As a consequence, schools in the United States now spend more than $2 billion each year on education technology. But what are schools getting in return for this significant investment in technology learning? Robert Slavin examines the results from five studies designed to answer this question. Slavin concludes that the impact of technology-infused instruction on reading, mathematics, and science in elementary and secondary schools is very small. His analysis finds a study-weighted average across these five reviews to be a +0.05 effect size. This effect size appears to be an insignificant return on investment for such a substantial allocation of resources. Slavin concludes that how software is designed is at the heart of the problem. Commercial companies most often develop education technology. Given technology companies are market-driven, education software developers value profit margins, attractiveness, ease of use, low cost, trends, and fads, over evidence of efficacy. Slavin proposes a solution to improve upon this current model needs to include boosting the incentives to technology developers for creating products based on rigorous research and proven technology-based programs. Regardless of how best to solve the problem, educators need to take seriously this call to address this issue.
Citation: Slavin, R. (2019). A Powerful Hunger for Evidence-Proven Technology. Baltimore, MD: Robert Slavin’s Blog. https://robertslavinsblog.wordpress.com/2019/11/14/a-powerful-hunger-for-evidence-proven-technology/.
November 14, 2019
Using Resource and Cost Considerations to Support Educational Evaluation: Six Domains. Assessing cost, along with the effectiveness of an initiative is common in public policy decision-making, but is frequently missing in education decision-making. Understanding the cost-effectiveness of an intervention is essential if educators are to maximize the impact of an intervention given limited budgets. Education is full of examples of practices, such as class-size reduction and accountability through high-stakes testing, that produce minimal results while consuming significant resources. It is vital for those making critical decisions to understand which practice is best suited to meet the needs of the school and the schools’ students that can be implemented using the available resources. The best way to do this is through the use of a cost-effectiveness analysis (CEA).
A CEA requires an accurate estimation of all added resources needed to implement the new intervention. Costs commonly associated with education interventions include; added personnel, professional development, classroom space, technology, and expenses to monitor effectiveness. The second variable essential to a CEA is the selection of a practice supported by research. In the past twenty years, a significant increase in the quality and quantity of research supporting different education practices has occurred. A CEA compares the extra expenditures required to implement a new intervention to current practices against targeted education outcomes. Examples of educational outcomes are standardized test scores, graduation rates, or student grades.
The focus of this essay is on which economic methods can complement and enhance impact evaluations. The authors propose the use of six domains to link intervention effectiveness to the best technique needed to determine which practice is the most cost-effective choice. The six domains outlined in the paper are outcomes, treatment comparisons, treatment integrity, the role of mediators, test power, and meta-analysis. This paper provides examples of how analyzing the costs associated with these domains can complement and augment practices in evaluating research in the field of education.
Citation: Belfield, C. R., & Brooks Bowden, A. (2019). Using Resource and Cost Considerations to Support Educational Evaluation: Six Domains. Educational Researcher, 48(2), 120-127.
October 24, 2019
Small class sizes for improving student achievement in primary and secondary schools: a systematic review. This Campbell Collaboration systematic review examines the impact of class size on academic achievement. The review summarizes findings from 148 reports from 41 countries. Reducing class size is viewed by many educators as an essential tool for improving student performance, and is especially popular among teachers. But smaller class sizes come at a steep cost. Education policymakers see increasing class size as a way to control education budgets. Despite the real policy and practice implications, the research on the educational effects of class‐size differences on student performance is mixed. This meta-analysis suggests, at best only, a small impact on reading achievement. The study finds a small negative effect on mathematics. Given the fact that class size reduction is minimally effective while being costly, aren’t there better solutions that are both cost-effective, benefits students, and can help teachers be successful in a very challenging profession?
Citation: Filges, T., Sonne‐Schmidt, C. S., & Nielsen, B. C. V. (2018). Small class sizes for improving student achievement in primary and secondary schools: a systematic review. Campbell Systematic Reviews, 14(1), 1-107.
October 22, 2019
Teacher turnover is an enormous burden on education systems, both in terms of student achievement and dollars. High turnover necessitates the recruitment of large numbers of novice teachers, whom research shows are less skilled. This situation is exacerbated by a steady exodus of veteran teachers opting to move from challenging assignments in poorer performing schools with higher percentages of lower socio-economic students to preferred assignments more affluent areas. The high rate of turnover destabilizes the system, forcing diversion of valuable resources from ongoing improvements to recruitment, hiring, and training of novice instructors. Teachers seem to be particularly at risk for higher turnover at the beginning of their careers. Nearly half of teachers leave within 5 years of entering the profession. Efforts to improve retention have been inadequate as evidenced by steadily increasing departures from the profession. This tendency toward turnover is even more striking in private schools than in public schools. Turnover represents a major obstacle to long-term stability, diverts valuable resources, and derails many efforts at reform.
Citation: Donley, J., Detrich, R., States, J., & Keyworth, R. (2019). Teacher Retention Analysis Overview. Oakland, CA: The Wing Institute. https://www.winginstitute.org/teacher-retention-strategies
October 15, 2019
The effect of charter schools on student achievement. Charter schools increasingly play a prominent role in educating students in the United States. Given the vast resources allocated to charter schools, it is imperative the question is asked, How effective are these schools in comparison to traditional public schools? This meta-analysis focuses on student math and reading performance. The authors found an overall effect size for elementary school reading and math of 0.02 and 0.05 and middle school math of 0.055. Effects were not statistically meaningful for middle school reading and high school math and reading. The study offers compelling evidence that charters under-perform traditional public schools in some locations, grades, and subjects, and out-perform traditional public schools in other geographical locations, grades, and subjects. The mixed results are not surprising as there is no set management model, quality of personnel, curricula, or pedagogy that distinguishes charter schools from public schools. The study did find a small positive effect size for KIPP charter schools. The absence of significant achievement gains attributed to charter schools should concern school systems considering expanding the number of charter schools as a solution to underperforming schools.
Citation: Betts, J. R., & Tang, Y. E. (2019). The effect of charter schools on student achievement. School choice at the crossroads: Research perspectives, 67-89.
October 7, 2019
Public Accountability and Nudges: The Effect of an Information Intervention on the Responsiveness of Teacher Education Programs to External Ratings. Teacher preparation programs (TEP) have received a great deal of research and policy attention as a potential driver of improvements in teacher quality and improved student outcomes. This paper presents research on external ratings of teacher education programs (TEPs) produced by the National Council on Teacher Quality (NCTQ) and the results of an experiment to improve school rankings by providing information and feedback to TEPs about how to improve their ratings. Research suggests that higher education institutions are responsive to feedback and public ratings. The college rankings published by U.S. News and World Report (USNWR) shows how powerful ratings can be in changing higher education practices such as admissions requirements, financial aid disbursements, and policies. NCTQ released the first ranking of TEP’s in 2014 as a part of an ongoing effort to rate teacher training nationally. The intervention relied upon providing targeted information about specific programmatic changes that would improve the rating for a randomly selected sample of elementary teacher education programs. Average program ratings improved between 2013 and 2016, but we find no evidence that the information intervention increased program responsiveness to NCTQ’s rating effort. The results show that the experimental group had lower ratings than the control group in 2016.
Citation: Goldhaber, D., & Koedel, C. (2019). Public Accountability and Nudges: The Effect of an Information Intervention on the Responsiveness of Teacher Education Programs to External Ratings. American Educational Research Journal, 0002831218820863.
October 1, 2019
Single‐track year‐round education for improving academic achievement in U.S. K‐12 schools: Results of a meta‐analysis. This Campbell Collaboration systematic meta-analysis examines the impact of reducing summer breaks on student academic performance. Research shows that students experience a loss in math and reading performance over summer breaks. Year‐round education (YRE) redistributes schooldays to shorten summer. This study finds that students at single‐track YRE schools show modestly higher achievement in both math and reading. The higher achievement is similar to estimates of summer learning loss. The effects were similar for all students. Given that summer learning loss is thought to be greater among students from disadvantaged groups, the estimated impact for low‐income and minority students were found to unexpectedly be about the same magnitude or smaller than for the full sample.
Citation: Fitzpatrick, D., & Burns, J. (2019). Single‐track year‐round education for improving academic achievement in US K‐12 schools: Results of a meta‐analysis. Campbell Systematic Reviews, 15(3), e1053.
Link: https://onlinelibrary.wiley.com/doi/full/10.1002/cl2.1053 and https://onlinelibrary.wiley.com/doi/epdf/10.1002/cl2.1053
September 30, 2019
Do Pay-for-Grades Programs Encourage Student Cheating? Evidence from a randomized experiment. Pay-for-grades programs are designed to increase student academic performance. One of the claims of those opposing such incentive systems is monetary incentives may lead to academic cheating. This randomized controlled study of 11 Chinese primary schools examines the effects of pay-for-grades programs on academic fraud. The study found widespread cheating behavior for students regardless of being in the control or experimental group, but no overall increase in the level of cheating for students in the pay-for-grades program. The authors conclude that educators need to be on the lookout for academic dishonesty, especially on standardized tests, while using moderate incentives to encourage student learning did not lead to increased levels of gaming the system.
Citation: Li, T., & Zhou, Y. (2019). Do Pay-for-Grades Programs Encourage Student Academic Cheating? Evidence from a Randomized Experiment. Frontiers of Education in China, 14(1), 117-137.
August 29, 2019
The Feasibility of Collecting School-Level Finance Data: An Evaluation of Data from the School-Level Finance Survey (SLFS) School Year 2014–15. Few things are more complicated nor critical than collecting accurate and meaningful data on school finances at the individual school level. It is complicated because of the sheer size of the education system, diversity of spending categories, differing state laws and regulations governing finances, and accounting systems not designed for this task. It is critical because the education system puts high value on equitable and adequate funding for all students. Tracking spending at the individual school level is also a requirement of the recently enacted Every Student Succeeds Act.
This research and development report field-tested a new model for collection of finance data at the school level—the School- Level Finance Survey (SLFS). The pilot SLFS, collected for fiscal year (FY) 14 (school year 2013–14) and FY 15 (school year 2014–15), was designed to evaluate whether the survey is a viable, efficient, and cost-effective method to gather comparable school-level finance data. The results suggest that, regardless of the inherent challenges, it is highly feasible to collect and report on school-level finance data with acceptable accuracy. It also projects improved response rates and the increased availability of complete, accurate, and comparable finance data at the school level as the number of states participating in the SLFS increases and the collection continues to expand.
Citation: Cornman, S.Q., Reynolds, D., Zhou, L., Ampadu, O., D’Antonio, L., Gromos, D., Howell, M., and Wheeler, S. (2019). The Feasibility of Collecting School-Level Finance Data: An Evaluation of Data From the School- Level Finance Survey (SLFS) School Year 2014–15 (NCES 2019-305). U.S. Department of Education. Washington, DC: National Center for Education Statistics.
August 29, 2019
Characteristics of Public and Private Elementary and Secondary School Principals in the United States: Results From the 2017–18 National Teacher and Principal Survey First Look: The National Teacher and Principal Survey (NTPS) collects data from public and private K-12 schools, principals, and teachers across the United States. Its data provides critical data on core topics such as school characteristics and services, principal and teacher demographics, and teacher preparation. The most recent 2017-18 report examined public (traditional), charter, and private school principals in terms of: race/ethnicity, age, highest college degree, salary, years experience (as a principal and at their current school), level of influence on decision-making, and experience with evaluations. A few of the more notable points include:
• Twenty-seven percent of school principals are 55 or older. This represents a significant number of principals who likely to retire in five years.
• The average salary for school principals is $ 92,900.
• Over ninety percent (91.7%) of school principals have a Master’s Degree or higher.
• Almost half (44.3%) of school principals have less than three years experience in their current schools.
• Seventy percent of school principals received evaluations in the selected year (79% in traditional public schools, 69% in charter schools, and 51% in private schools).
Citation: Taie, S., and Goldring, R. (2019). Characteristics of Public and Private Elementary and Secondary School Principals in the United States: Results From the 2017–18 National Teacher and Principal Survey First Look (NCES 2019- 141). U.S. Department of Education. Washington, DC: National Center for Education Statistics.