Categories for Structural Interventions
July 10, 2018
The Effect of Voucher Programs on Student Achievement: A Meta-Analysis
News Summary: This meta-analysis examines the effectiveness of school voucher programs for improving student achievement. The research is of special interest for three important reasons; (1) it examines an important policy issue relevant to educators and the public, (2) it is an example of a study designed to replicate previous research on an important topic, and (3) it highlights the importance of examining the cost effectiveness associated with implementing practices in real world settings.
The use of school vouchers as a means to improve the quality of education has been an attractive, although controversial. It has been touted as a way to use public funding to overcome academic deficits in school systems. Enabling parents to exercise choice as to where to send a child to school is enticing. It appeals to the belief that exercising “control” over where your child can go to school will have an impaction the quality of education. The important question is, does this type of structural intervention produce both statistically significant, but more importantly do school vouchers produce socially significant academic gains?
This study tries to answer this question by replicating a previous meta-analysis of school voucher programs by Shakeel, Anderson, and Wolf (2016). The studies included in both meta-analyses required that to be included in the research individual studies must use randomized control trials of school voucher programs reporting quantitative measures of reading and/or math performance. The Wing Institute chose to include this new item to feature the need for replicating research as means to increase confidence in the results and broaden our knowledge base on a topic. The results indicated that, compared to the original study, this meta-analysis obtained smaller effect sizes with larger standard errors. This is not surprising as most replication of original research report smaller effects. The results of this research although somewhat smaller are consistent with the Shakeel, Anderson, and Wolf (2016) study. More importantly, both studies found that although the effects were for the most part positive they were very small ranging from 0.080 effect size on reading to 0.135 effect size on math.
It is important to note these results are also similar to those found in the Yeh, S. S. (2007) research on this topic. Yeh goes beyond reporting on effect size to examine the importance of putting effect size in the context of cost-effectiveness. It becomes quite clear when one asks the question, are school vouchers a cost effective intervention designed to deliver significant change, this intervention is found wanting. Yeh in his research concludes that educators and the public will be better served by adopting a practice such as formative assessment, that has a greater effective size (0.90) and can be implemented at a much smaller cost.
Citation:
(1) Bennett, M., Banerjee, H. L., Doan, L. N., Geib, T., and Burley, A. (2018). The Effect of Voucher Programs on Student Achievement: A Meta-Analysis. AERA Conference New York, NY. 10.302/1302823.
(2) Shakeel, M., Anderson, K., & Wolf, P. (2016). The participant effects of private school vouchers across the globe: A meta-analytic and systematic review.
(3) Yeh, S. S. (2007). The cost-effectiveness of five policies for improving student achievement. American Journal of Evaluation, 28(4), 416-436.
Link:
(1) http://www.aera.net/Publications/Online-Paper-Repository/AERA-Online-Paper-Repository/Owner/997930
(2) https://files.eric.ed.gov/fulltext/ED567044.pdf
(3)http://c2.derrytsd.schoolwires.net/cms/lib/PA09000080/Centricity/ModuleInstance/1496/cost_effect_of_rapid_assess_by_Yeh_(2007).pdf
July 2, 2018
Preparing for Life after High School: The Characteristics and Experiences of Youth in Special Education. Findings from the National Longitudinal Transition Study 2012. Volume 2: Comparisons across Disability Groups. Full Report
News Summary:The United States has committed to improving the lives of students with disabilities for over 40 years. Since the advent of Federal Law PL 94-142 in 1975 that mandated a free and appropriate education for all students regardless of ability and six reauthorizations of legislation, the federal government has emphasized the need to prepare students with disabilities for post-secondary education, careers, and independent living. The federal investment in funding special education services exceeds $15 Billion annually. It is reasonable to ask, are student with disabilities substantially benefiting from these efforts? The National Longitudinal Transition Study (NLTS) provides the most recent data on youth with disabilities and efforts to address their needs. The study used surveys in 2012 and 2013 on a nationally representative set of nearly 13,000 students. The student included were mostly those with an individualized education program (IEP) and expected to receive special education services. The data reveal participation in key transition activities by youth with an IEP and their parents have declined, although they are just as likely to have gone to an IEP meeting. The findings from this report suggest a closer examination of current practices is warranted with a focus on achieving the stated outcomes the laws were designed to remedy.
Citation: Lipscomb, S., Hamison, J., Liu Albert, Y., Burghardt, J., Johnson, D. R., & Thurlow, M. (2018). Preparing for Life after High School: The Characteristics and Experiences of Youth in Special Education. Findings from the National Longitudinal Transition Study 2012. Volume 2: Comparisons across Disability Groups. Full Report. NCEE 2017-4018. National Center for Education Evaluation and Regional Assistance.
Link: https://files.eric.ed.gov/fulltext/ED580934.pdf
February 7, 2018
The Information Book Flood: Is Additional Exposure Enough to Support Early Literacy Development?
News Summary: Over the past twenty years many reading interventions have been proposed. One of these, “Book Flooding”, proposes that providing an enriched environment in which books are present and readily available can improve reading. Much of the research on this topic has focused on exposing children in the early grades to storybooks. Given the greater importance on reading complex text in meeting new reading standards, this study examines the impact of book flooding of books that stress academic words and technical terms. This quasi-experimental study examines the influence of a book distribution program targeted at enhancing children’s exposure to information books. The research examined whether a flood of information books in early childhood settings could affect growth in language, content-related vocabulary, and concepts of comprehending information text. The study concludes there were no significant effects on student outcomes and that book distribution programs on their own need to be reevaluated if they are to improved student reading performance.
Citation: Neuman, S. B. (2017). The Information Book Flood: Is Additional Exposure Enough to Support Early Literacy Development?. The Elementary School Journal, 118(1), 1-27.
Link: http://www.journals.uchicago.edu/doi/pdfplus/10.1086/692913
January 22, 2018
Homework and academic achievement: A meta-analytic review of research
This study from Turkey looks at the effect of homework on students’ academic achievement. This meta-analysis attempts to answer to the question: “What kind of effect does homework assignment have on students’ academic achievement levels?” The effect size of homework on student achievement was determined to be a small effect size, d = 0.229. Despite the significant time students spend on homework, this study supports previous research that suggest these efforts only have a modest effect. Successful use of homework requires making smart choices about when to assign homework and how much to assign. To maximize the impact, teachers must grade homework assignments promptly and provide timely feedback. Unless these rules are followed, homework unsurprisingly achieves poor results (See Wing Institute Homework Abstract).
Citation: Bas, G., Senturk, C., & Cigerci, F. M. (2017). Homework and academic achievement: A meta-analytic review of research. Issues in Educational Research, 27(1), 31-50.
Link: http://www.iier.org.au/iier27/bas.html
December 14, 2017
Impacts of Early Childhood Education on Medium-and Long-Term Educational Outcomes
This meta-analysis examines over 50 years of data on the impact of early childhood interventions designed to improve student performance. As schools look for initiatives that can make a difference improving important social outcomes, early childhood education (ECE), as a structural intervention, appears to offer results that last beyond the first few years of elementary school. This study finds ECE has a positive effect on reducing special education placements (effect size = 0.33), reduces grade retention (effect size = 0.26), and increases high school graduation rates (effect size = 0.24). Although, these are considered to be small effect sizes they have an impact improving large numbers of student’s education experiences while reducing overall education expenditures.
Citation: McCoy, D. C., Yoshikawa, H., Ziol-Guest, K. M., Duncan, G. J., Schindler, H. S., Magnuson, K., Yang, R., Koepp, A., & Shonkoff, J. P. (2017). Impacts of Early Childhood Education on Medium-and Long-Term Educational Outcomes. Educational Researcher, 46(8), 474-487.
Link: http://journals.sagepub.com/doi/abs/10.3102/0013189X17737739
August 8, 2017
A Meta-Analysis of Class Sizes and Ratios in Early Childhood Education Programs: Are Thresholds of Quality Associated With Greater Impacts on Cognitive, Achievement, and Socioemotional Outcomes?
This meta-analysis of early childhood education programs in the U.S. examines the impact of class size and child to teacher ratios on the cognitive, achievement, and socioemotional outcomes for children. The study found no socially significant relationship with cognitive and achievement effect size for either class size or child to teacher ratios. The only notable improvement in effect size was found when child to teacher ratios were lower than 7.5-1.
Citation: Bowne, J. B., Magnuson, K. A., Schindler, H. S., Duncan, G. J., & Yoshikawa, H. (2017). A Meta-Analysis of Class Sizes and Ratios in Early Childhood Education Programs: Are Thresholds of Quality Associated With Greater Impacts on Cognitive, Achievement, and Socioemotional Outcomes?. Educational Evaluation and Policy Analysis, 0162373716689489.
Link: http://journals.sagepub.com/doi/abs/10.3102/0162373716689489
June 15, 2017
School Lunch Quality and Academic Performance
Providing children with nutritious school meals continues to be a topic of interest in education policy. It has been argued that a healthy diet can have a positive impact on childhood obesity as well as student achievement. However, there is little empirical evidence to support this popular intervention. A working paper released by the National Bureau of Economic Research offers insights to help policy makers decide whether to select this option as a way to make a difference in children’s lives. The study, a natural experiment conducted in California public schools, uses a difference-in-difference regression statistical analysis of data from a treatment group and a control group at two or more different time periods, pre-treatment and post-treatment. The study found no evidence to support a reduction in obesity, but it did discover that introducing healthy meals was associated with a 0.036 standard deviation increase in test scores. The improved student achievement, although small, makes nutritious school lunches a viable cost-effective intervention that can boost test scores. It is important to remember that this is a correlational study and thus cannot establish a cause and effect relationship between healthy eating and student achievement.
Citation: Anderson, M. L., Gallagher, J., & Ritchie, E. R. (2017). School lunch quality and academic performance (NBER Working Paper No. 23218). Cambridge, MA: National Bureau of Economic Research.
Link: http://www.nber.org/papers/w23218.pdf
June 14, 2017
Creating Supportive School Cultures for Beginning Teachers: Mitigating the Cultural Contextual Factors
This systematic literature review examines factors that affect new teachers as they become employed. It suggests that new teachers experience significant stress as they transition from teacher preparation programs to everyday teaching. The pressures experienced by novice instructors are evidenced by high rates of turnover during the first 5 years of teaching. This initial period entails a steep learning curve as new teachers try to match and adjust what they learned in pre-service training with the unique cultural practices of their new placements. To mitigate the challenges new teachers face, schools have implemented an array of practices subsumed under the heading “teacher induction.” Unfortunately, the literature review finds that the lack of standardization of practices including mentoring to coaching has reduced the overall impact of teacher induction and the ability of schools to effectively align the education philosophy and cultural values of new teachers with the strategies, procedures, needs, and resources of the schools where the new teachers are beginning their careers.
Citation: Kutsyuruba, B, Walker, K. D., and Gooden, L., (2017). Creating supportive school cultures for beginning teachers: Mitigating the cultural contextual factors. International Journal of Educational Organization and Leadership, 24(2), 1–18.
Link: https://ir.library.carleton.ca/pub/26850/
May 31, 2017
Learning in One-to-One Laptop Environments: A Meta-Analysis and Research Synthesis
A recently released paper on the impact of computers on student achievement has concluded that providing each student with a computer has a modest but positive effect on student achievement. This meta-analysis included 10 studies from more than 15 years of research on the topic. The researchers found that one-to-one laptop programs have a statistically significant positive role in improving student test scores in English/language arts, writing, math, and science. This research runs counter to previous studies, which viewed computer impact as overstated and computers as underused in schools. The new study found that providing students with their own computers has a positive impact by making it easier to share drafts of projects and thus increasing access to feedback. The researchers also concluded that more research is needed as only 10 rigorous studies were available for inclusion in the meta-analysis.
Citation: Zheng, B., Warschauer, M., Lin, C. H., & Chang, C. (2016). Learning in one-to-one laptop environments: A meta-analysis and research synthesis. Review of Educational Research, 86(4), 1052–1084.
Link: http://journals.sagepub.com/doi/abs/10.3102/0034654316628645
May 15, 2017
Academic achievement of students without special educational needs in inclusive classrooms: A meta-analysis
A newly released multi-national meta-analysis examined the impact of including special education needs students (SEN) in classrooms with students without special needs. Inclusion of SEN students in general education classrooms has been broadly practiced for more than 30 years and is seen as a method for improving SEN student performance. Previous research on the impact of the practice on non-SEN students produced mixed results, prompting the authors to conduct a meta-analysis to estimate the effect size of the relationship between inclusion education and academic achievement of students without special needs. A total of 47 studies with more than 4,800,000 students met the criteria for the meta-analysis. The studies had to be quantitative, assess academic achievement, and measure student academic achievement in at least one of the following subjects: language, mathematics, science, biology, or a foreign language. Also, the participants had to be in grades K–12. The meta-analysis concluded that attending inclusive classrooms is positively, though weakly, associated with the academic achievement of students without special needs. The effect size was determined to be 0.12. This compares with the effect sizes John Hattie found for many popular structural interventions: educational expenditure (0.23), class size (0.21), ability grouping (0.12), within-class grouping (0.16), and charter schools (0.20).
Citation: Szumski, G., Smogorzewska, J., & Karwowski, M. (2017). Academic achievement of students without special educational needs in inclusive classrooms: A meta-analysis. Educational Research Review, 21, 33–54.
Link: http://www.sciencedirect.com/science/article/pii/S1747938X17300131
