Categories for Education Outcomes
July 10, 2018
The Effect of Voucher Programs on Student Achievement: A Meta-Analysis
News Summary: This meta-analysis examines the effectiveness of school voucher programs for improving student achievement. The research is of special interest for three important reasons; (1) it examines an important policy issue relevant to educators and the public, (2) it is an example of a study designed to replicate previous research on an important topic, and (3) it highlights the importance of examining the cost effectiveness associated with implementing practices in real world settings.
The use of school vouchers as a means to improve the quality of education has been an attractive, although controversial. It has been touted as a way to use public funding to overcome academic deficits in school systems. Enabling parents to exercise choice as to where to send a child to school is enticing. It appeals to the belief that exercising “control” over where your child can go to school will have an impaction the quality of education. The important question is, does this type of structural intervention produce both statistically significant, but more importantly do school vouchers produce socially significant academic gains?
This study tries to answer this question by replicating a previous meta-analysis of school voucher programs by Shakeel, Anderson, and Wolf (2016). The studies included in both meta-analyses required that to be included in the research individual studies must use randomized control trials of school voucher programs reporting quantitative measures of reading and/or math performance. The Wing Institute chose to include this new item to feature the need for replicating research as means to increase confidence in the results and broaden our knowledge base on a topic. The results indicated that, compared to the original study, this meta-analysis obtained smaller effect sizes with larger standard errors. This is not surprising as most replication of original research report smaller effects. The results of this research although somewhat smaller are consistent with the Shakeel, Anderson, and Wolf (2016) study. More importantly, both studies found that although the effects were for the most part positive they were very small ranging from 0.080 effect size on reading to 0.135 effect size on math.
It is important to note these results are also similar to those found in the Yeh, S. S. (2007) research on this topic. Yeh goes beyond reporting on effect size to examine the importance of putting effect size in the context of cost-effectiveness. It becomes quite clear when one asks the question, are school vouchers a cost effective intervention designed to deliver significant change, this intervention is found wanting. Yeh in his research concludes that educators and the public will be better served by adopting a practice such as formative assessment, that has a greater effective size (0.90) and can be implemented at a much smaller cost.
(1) Bennett, M., Banerjee, H. L., Doan, L. N., Geib, T., and Burley, A. (2018). The Effect of Voucher Programs on Student Achievement: A Meta-Analysis. AERA Conference New York, NY. 10.302/1302823.
(2) Shakeel, M., Anderson, K., & Wolf, P. (2016). The participant effects of private school vouchers across the globe: A meta-analytic and systematic review.
(3) Yeh, S. S. (2007). The cost-effectiveness of five policies for improving student achievement. American Journal of Evaluation, 28(4), 416-436.
July 2, 2018
Preparing for Life after High School: The Characteristics and Experiences of Youth in Special Education. Findings from the National Longitudinal Transition Study 2012. Volume 2: Comparisons across Disability Groups. Full Report
News Summary:The United States has committed to improving the lives of students with disabilities for over 40 years. Since the advent of Federal Law PL 94-142 in 1975 that mandated a free and appropriate education for all students regardless of ability and six reauthorizations of legislation, the federal government has emphasized the need to prepare students with disabilities for post-secondary education, careers, and independent living. The federal investment in funding special education services exceeds $15 Billion annually. It is reasonable to ask, are student with disabilities substantially benefiting from these efforts? The National Longitudinal Transition Study (NLTS) provides the most recent data on youth with disabilities and efforts to address their needs. The study used surveys in 2012 and 2013 on a nationally representative set of nearly 13,000 students. The student included were mostly those with an individualized education program (IEP) and expected to receive special education services. The data reveal participation in key transition activities by youth with an IEP and their parents have declined, although they are just as likely to have gone to an IEP meeting. The findings from this report suggest a closer examination of current practices is warranted with a focus on achieving the stated outcomes the laws were designed to remedy.
Citation: Lipscomb, S., Hamison, J., Liu Albert, Y., Burghardt, J., Johnson, D. R., & Thurlow, M. (2018). Preparing for Life after High School: The Characteristics and Experiences of Youth in Special Education. Findings from the National Longitudinal Transition Study 2012. Volume 2: Comparisons across Disability Groups. Full Report. NCEE 2017-4018. National Center for Education Evaluation and Regional Assistance.
June 27, 2018
Four Domains for Rapid School Improvement: An Implementation Framework
The National Implementation Research Network (NIRN) and the National Center for School Turnaround published the “Four Domains for Rapid School Improvement: An Implementation Framework” as a companion to the Center for School Turnaround’s publication of “The Four Domains for Rapid School Improvement: A Systems Framework“. This paper describes “how” to effectively implement lasting school improvement initiatives that maximize leadership, develop talent, amplify instructional transformation, and shift the culture.
Citation:Jackson, K., R., Fixsen, D., and Ward, C. (2018). Four Domains for Rapid School Improvement: An Implementation Framework. The Center on School Turnaround.
May 1, 2018
The National Assessment of Educational Progress (NAEP) has been assessing what American students know and can do in various subject areas since 1969. Subjects include: mathematics, reading, science, writing, the arts, civics, economics, geography, U.S. history, and in Technology and Engineering Literacy (TEL). NAEP generates the Nation’s Report Card every two years summarizing test scores by subject area, grade levels (4th, 8th, and 12th), student demographics, school characteristics, etc. With very few exceptions, the 2017 national results did not show any gains in test scores from previous years. One of the more critical metrics is the percentage of students who score at or above proficiency. Proficiency is the target academic performance benchmark for students. The 2017 results show that few students are at or above proficiency in all subjects. In reading, only 37% of fourth graders, 36% of eighth graders, and 357% of twelfth grader were at or above proficiency. In math, each subsequent assessed grade showed lower percentages of students at or above proficiency (4thgrade at 40%, 8thgrade at 34%, 12thgrade at 25%). Such scores are disappointing given the efforts made towards school improvement.
Citation: Institute of Education Sciences. (2018). The Nation’s Report Card, 2017. NCES 2006-451. National Center for Education Statistics.
April 23, 2018
A Synthesis of Quantitative Research on Reading Programs for Secondary Students
This review of the research on secondary reading programs focuses on 69 studies that used random assignment (n=62) or high-quality quasi-experiments (n=7) to evaluate outcomes of 51 programs on widely accepted measures of reading. Reading performance of students in America’s middle and high schools is one of the most important problems in education. In 2015, the National Assessment of Educational Progress (NAEP; NCES, 2016) reported that only 34% of eighth graders scored at or above proficiency. At the twelfth-grade level only 37% of students scored at or above proficiency. Given the importance of mastering reading for success in school it is important that schools adopt programs to help bridge the gap between current performance and expectations.
The study found programs using one-to-one and small-group tutoring (+0.14 to +0.28 effect size), cooperative learning (+0.10 effect size), whole-school approaches including organizational reforms such as teacher teams (+0.06 effect size), and writing-focused approaches (+0.13 effect size) showed positive outcomes. Individual approaches in a few other categories also showed positive impacts. These include programs emphasizing social studies/science, structured strategies, and personalized and group/personalization rotation approaches for struggling readers.
An important finding was programs providing a daily extra period of reading and those utilizing technology were no more effective, on average, than programs that did not provide these resources. This might not be as surprising as it appears on the surface. If students are struggling in reading, just placing them in a setting to read may only produce additional frustration and failure for those currently struggling with reading.
A few commonalities among programs that achieved positive outcomes are worth noting. One of these factors was that programs with positive outcomes tended to emphasize student motivation, student-to-student and student-to-teacher relationships, and social-emotional learning. An additional factor found in many of the promising programs is individualizing the intervention.
The findings are important suggesting interventions for secondary readers to improve struggling student’s chances of experiencing greater success in high school and better opportunities after graduation. Although, these effect sizes are small, given the large number of participants, smaller effect sizes would be anticipated and still may be of interest to school administrators.
Citation: Baye, A., Lake, C., Inns, A. & Slavin, R. E. (2018, January). A Synthesis of Quantitative Research on Reading Programs for Secondary Students. Baltimore, MD: Johns Hopkins University, Center for Research and Reform in Education.
April 10, 2018
Call for Action to Prevent Gun Violence in the United States of America Interdisciplinary Group on Preventing School and Community Violence
In response to the continuing gun violence in American schools, an interdisciplinary group of 19 scholars are proposing an eight-point plan to prevent future tragedies that have become common place in the nation. This one-page position statement proposes a public health approach to protecting children as well as adults from gun violence involves three levels of prevention: (1) universal approaches promoting safety and well-being for everyone; (2) practices for reducing risk and promoting protective factors for persons experiencing difficulties; and (3) interventions for individuals where violence is present or appears imminent.
Citation: Astor, R. et al. (2018). Call for Action to Prevent Gun Violence in the United States of America. University of Virginia.
April 10, 2018
Discipline Disparities for Black Students, Boys, and Students with Disabilities
The US Government Accountability Office has recently released a new report evaluating the disproportionality in discipline in K-12 grades. The racial and gender gap persists in spite of efforts to remediate. African-American youth, boys, and individuals with disabilities are more likely to receive any type of discipline than are individuals in our sub-groups than would be predicted on the basis of their percentage of the population. In this evaluation, the disproportionality existed even though economic level of the student was controlled for. Previously, it had been argued that the disproportionality was a function of poverty rather than race and gender. This study challenges that argument. These data highlight that as a society we still have a great deal of work to do to overcome racial and gender biases in this country.
Citation: United States Governmental Accountability Office (2018). K-12 education: A guide for schools (GAO publication-18-258). Retrieved from https://www.gao.gov/products/GAO-18-258
April 3, 2018
Why Do School Psychologists Cling to Ineffective Practices? Let’s Do What Works.
This article examines the impact of poor decision making in school psychology, with a focus on determining eligibility for special education. Effective decision making depends upon the selection and correct use of measures that yield reliable scores and valid conclusions, but traditional psychometric adequacy often comes up short. The author suggests specific ways in which school psychologists might overcome barriers to using effective assessment and intervention practices in schools in order to produce better results.
Citation: VanDerHeyden, A. M. (2018, March). Why Do School Psychologists Cling to Ineffective Practices? Let’s Do What Works. In School Psychology Forum, Research in Practice(Vol. 12, No. 1, pp. 44-52). National Association of School Psychologists.
March 22, 2018
Promoting Educator Effectiveness: The Effects of Two Key Strategies
The National Center for Education Evaluation, a division of the Institute of Education Sciences has released a new research brief that evaluated two strategies for improving educator effectiveness as measured by improvements in student outcomes. The two strategies evaluated were performance feedback to educators about several dimensions of their performance for a period of two years and a pay-for-performance system that was in place for four years. In the performance feedback project teachers were given feedback four times per year on their classroom practices and principals received feedback two times per year. The impact on student outcomes were small. After the first year, there was a statistically significant difference between students in math but not in reading in the feedback schools compared to the schools that served as the control group as measured by end of year scores. At the end of the second year there were no statistically significant effects for either reading or math. The net gain in math achievement for the students in the feedback schools was about 4 weeks compared to the control group.
The pay-for-performance study teachers were eligible for performance bonuses based on their ratings across multiple dimensions of their performance. The students in the pay-for-performance schools outperformed the students in the control group schools in both math and reading. Statistically significant scores were obtained in reading beginning the first year and each subsequent year through the third year. Students math scores in the pay-for-performance schools achieved statistically significant scores only at the end of the third year. There was no additional benefit in reading or math for pay-for-performance in the fourth year. The overall benefit of the gains by the students in the pay-for-performance schools was estimated to be 3-4 weeks. Again, this is a relatively small impact. It was noted that the quality of implementation may have reduced the impact of the two projects. Across both studies, there were discrepancies between how the programs were intended to be implemented and how they were actually implemented. Further limitations of the studies are that the performance feedback for teachers was only four times per year. This is generally considered to be far too infrequent to have meaningful impact. In the pay-for-performance study, 40% of the teachers reported that they were not aware they were eligible for bonuses, limiting the motivational properties of the bonus system. Given these results it is clear that we must continue searching for effective approaches to improving educator performance and ways to assure high quality implementation.
Citation: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences (March 2018). Promoting Educator Effectiveness: The Effects of Two Key Strategies.
March 21, 2018
The Effectiveness of School-Based Mental Health Services for Elementary-Aged Children: A Meta-Analysis
News Summary: This meta-analysis examines the effects of school-based mental health services for elementary school-age children delivered by school personnel. Forty-three controlled trials evaluating 49,941 elementary school-age children met criteria for inclusion in this study. The study used a randomized, between-subjects, controlled comparison or quasi-experimental design using matched samples to minimize selection bias. The study finds school-based mental health services had a small to medium effect size (Hedges g = 0.39) in decreasing mental health problems. The largest effect size was for targeted intervention, (Hedges g = 0.76), followed by selective prevention (Hedges g = 0.67) compared with universal prevention (Hedges g = 0.29). Interventions integrated into student’s academic instruction using contingency management were found to have positive impacts (Hedges g = 0.57), and interventions implemented multiple times per week (Hedges g = 0.50) were also shown to have a notable impact for improving student’s lives. These results are promising considering the normal barriers that impede students from receiving mental health care outside of school and the fact 80% of mental health service are provided in schools by personnel who are readily available and are shown to be effective in addressing student’s mental health needs (Ringeisen, Henderson, and Hoagwood, 2003).
- Targeted Intervention: interventions provided only to students identified as having mental health problems.
- Universal prevention: interventions provided to all students in a classroom
- Selective prevention: interventions provided only to students at risk for mental health problems according to a teacher referral or mental health screening
Citation: Sanchez, A. L., Cornacchio, D., Poznanski, B., Golik, A. M., Chou, T., & Comer, J. S. (2018). The effectiveness of school-based mental health services for elementary-aged children: a meta-analysis. Journal of the American Academy of Child & Adolescent Psychiatry, 57(3), 153-165.