Categories for Education Outcomes
November 14, 2019
Using Resource and Cost Considerations to Support Educational Evaluation: Six Domains. Assessing cost, along with the effectiveness of an initiative is common in public policy decision-making, but is frequently missing in education decision-making. Understanding the cost-effectiveness of an intervention is essential if educators are to maximize the impact of an intervention given limited budgets. Education is full of examples of practices, such as class-size reduction and accountability through high-stakes testing, that produce minimal results while consuming significant resources. It is vital for those making critical decisions to understand which practice is best suited to meet the needs of the school and the schools’ students that can be implemented using the available resources. The best way to do this is through the use of a cost-effectiveness analysis (CEA).
A CEA requires an accurate estimation of all added resources needed to implement the new intervention. Costs commonly associated with education interventions include; added personnel, professional development, classroom space, technology, and expenses to monitor effectiveness. The second variable essential to a CEA is the selection of a practice supported by research. In the past twenty years, a significant increase in the quality and quantity of research supporting different education practices has occurred. A CEA compares the extra expenditures required to implement a new intervention to current practices against targeted education outcomes. Examples of educational outcomes are standardized test scores, graduation rates, or student grades.
The focus of this essay is on which economic methods can complement and enhance impact evaluations. The authors propose the use of six domains to link intervention effectiveness to the best technique needed to determine which practice is the most cost-effective choice. The six domains outlined in the paper are outcomes, treatment comparisons, treatment integrity, the role of mediators, test power, and meta-analysis. This paper provides examples of how analyzing the costs associated with these domains can complement and augment practices in evaluating research in the field of education.
Citation: Belfield, C. R., & Brooks Bowden, A. (2019). Using Resource and Cost Considerations to Support Educational Evaluation: Six Domains. Educational Researcher, 48(2), 120-127.
November 7, 2019
What Do Surveys of Program Completers Tell Us About Teacher Preparation Quality? Identifying which teacher preparation programs produce highly qualified teachers is understood to be a means to improve the effectiveness of teacher preparation programs (TTP). One proposed method for measuring TTP effectiveness is surveying recent graduate’s satisfaction with the training received. Research suggests that analysis of the surveys correlate satisfaction with teacher classroom performance, evaluation ratings, and retention data. If correct, this data offers schools a wealth of information to aid in deciding which pre-service programs to focus recruiting efforts. It also suggests that surveys can provide data for holding TTP accountable.
But much of the available research lacks sufficient rigor. This paper uses survey data from teachers in the state of North Carolina to gauge graduate’s satisfaction with TTP training to raise the validity and reliability of the study’s findings. The study concludes perceptions of preparation programs are modestly associated with the effectiveness and retention of first and second-year teachers. The researchers find, on average, those who feel better prepared to teach are more effective and more likely to remain in teaching. These results indicate that surveys of preparation program graduate satisfaction be monitored to assure validity and reliability of polling, given the interest accreditation bodies, state agencies, and teacher preparation programs show in using this data for high stakes decision making. The results also imply that surveys alone do not provide sufficient data to identify which programs offer the best teacher training.
Citation: Bastian, K. C., Sun, M., & Lynn, H. (2018). What do surveys of program completers tell us about teacher preparation quality? Journal of Teacher Education, November 2019.
October 25, 2019
A Quantitative Synthesis of Research on Writing Approaches in Grades 2 to 12. This paper looks at randomized or well-matched control group research on outcomes of writing programs. The average effect size for those writing programs reviewed was a small effect size of 0.18. The writing programs fell into three categories; those that focus on the writing process, cooperative learning, or interactions between reading and writing.
The core characteristics of programs that produced the best writing outcomes include:
• Use of cooperative learning
• Structured approaches that give students step-by-step guides to writing
• Programs that teach students to assess their own and peer writing,
• Programs that balance writing with reading
• Programs that attempt to build students’ motivation to write and enjoy self-expression
• Programs that teach writing conventions (e.g., grammar, punctuation, usage) explicitly, but in the context of creative writing
• Programs that provide extensive professional development to teachers, in which teachers experience the writing strategies they will employ
Citation: Slavin, R. E., Lake, C., Inns, A., Baye, A., Dachet, D., & Haslam, J. (2019). A Quantitative Synthesis of Research on Writing Approaches in Grades 2 to 12. Best Evidence Encyclopedia.
October 24, 2019
Small class sizes for improving student achievement in primary and secondary schools: a systematic review. This Campbell Collaboration systematic review examines the impact of class size on academic achievement. The review summarizes findings from 148 reports from 41 countries. Reducing class size is viewed by many educators as an essential tool for improving student performance, and is especially popular among teachers. But smaller class sizes come at a steep cost. Education policymakers see increasing class size as a way to control education budgets. Despite the real policy and practice implications, the research on the educational effects of class‐size differences on student performance is mixed. This meta-analysis suggests, at best only, a small impact on reading achievement. The study finds a small negative effect on mathematics. Given the fact that class size reduction is minimally effective while being costly, aren’t there better solutions that are both cost-effective, benefits students, and can help teachers be successful in a very challenging profession?
Citation: Filges, T., Sonne‐Schmidt, C. S., & Nielsen, B. C. V. (2018). Small class sizes for improving student achievement in primary and secondary schools: a systematic review. Campbell Systematic Reviews, 14(1), 1-107.
October 22, 2019
Teacher turnover is an enormous burden on education systems, both in terms of student achievement and dollars. High turnover necessitates the recruitment of large numbers of novice teachers, whom research shows are less skilled. This situation is exacerbated by a steady exodus of veteran teachers opting to move from challenging assignments in poorer performing schools with higher percentages of lower socio-economic students to preferred assignments more affluent areas. The high rate of turnover destabilizes the system, forcing diversion of valuable resources from ongoing improvements to recruitment, hiring, and training of novice instructors. Teachers seem to be particularly at risk for higher turnover at the beginning of their careers. Nearly half of teachers leave within 5 years of entering the profession. Efforts to improve retention have been inadequate as evidenced by steadily increasing departures from the profession. This tendency toward turnover is even more striking in private schools than in public schools. Turnover represents a major obstacle to long-term stability, diverts valuable resources, and derails many efforts at reform.
Citation: Donley, J., Detrich, R., States, J., & Keyworth, R. (2019). Teacher Retention Analysis Overview. Oakland, CA: The Wing Institute. https://www.winginstitute.org/teacher-retention-strategies
October 15, 2019
The effect of charter schools on student achievement. Charter schools increasingly play a prominent role in educating students in the United States. Given the vast resources allocated to charter schools, it is imperative the question is asked, How effective are these schools in comparison to traditional public schools? This meta-analysis focuses on student math and reading performance. The authors found an overall effect size for elementary school reading and math of 0.02 and 0.05 and middle school math of 0.055. Effects were not statistically meaningful for middle school reading and high school math and reading. The study offers compelling evidence that charters under-perform traditional public schools in some locations, grades, and subjects, and out-perform traditional public schools in other geographical locations, grades, and subjects. The mixed results are not surprising as there is no set management model, quality of personnel, curricula, or pedagogy that distinguishes charter schools from public schools. The study did find a small positive effect size for KIPP charter schools. The absence of significant achievement gains attributed to charter schools should concern school systems considering expanding the number of charter schools as a solution to underperforming schools.
Citation: Betts, J. R., & Tang, Y. E. (2019). The effect of charter schools on student achievement. School choice at the crossroads: Research perspectives, 67-89.
October 15, 2019
Risks and rewards of school-based mentoring relationships: A reanalysis of the student mentoring program evaluation. School-based mentoring programs are a widely funded intervention adopted as a means to positively impact student conduct and achievement. Despite widespread support, meta-analyses indicate that the effects of school-based mentoring programs are small. The authors hypothesize that the poor outcomes found in previous research happen because mentors do not develop high-quality relationships with the student mentees. To examine this issue, the authors reanalyzed a large randomized trial on this topic. The study concludes that a high-quality relationship has its benefits, but the effect size on academic and behavioral outcomes was near zero and confirmed the original findings. These results suggest that schools proceed cautiously before adopting school-based mentoring programs.
Citation: Lyons, M. D., & McQuillin, S. D. (2018). Risks and rewards of school-based mentoring relationships: A reanalysis of the student mentoring program evaluation. School Psychology Quarterly
October 14, 2019
Supporting Appropriate Student Behavior Overview. Proactive classroom management strategies promote appropriate behavior and reduce or prevent misbehavior. Reinforcement is at the core of most proactive strategies. It is defined as a consequence that follows a behavior and increases the frequency of that behavior. Contingent praise is a versatile strategy based on reinforcement. Through positive statements delivered by the teacher, contingent praise acknowledges appropriate conduct and informs students what they did well. A rule of thumb is to maintain a 4:1 ratio of positive praise to corrective statements. Teachers should avoid the trap of becoming overly critical, which can damage the student-teacher relationship and lead to increased misbehavior. Other reinforcement-based strategies that use praise as well as material reinforcers include class-wide group contingencies, point systems, and behavior contracts. Material reinforcers commonly used in schools include tangible reinforcers (stickers, toys, food), preferred activities (games, computer time), and privileges (running errands, distributing papers). Additional proactive strategies are classroom rules and procedures, a structured environment, active supervision, and effective instruction; see classroom management drivers.
Citation: Guinness, K., Detrich, R., Keyworth, R. & States, J. (2019). Overview of Supporting Appropriate Behavior. Oakland, CA: The Wing Institute. https://www.winginstitute.org/classroom-appropriate-behaviors.
October 7, 2019
Public Accountability and Nudges: The Effect of an Information Intervention on the Responsiveness of Teacher Education Programs to External Ratings. Teacher preparation programs (TEP) have received a great deal of research and policy attention as a potential driver of improvements in teacher quality and improved student outcomes. This paper presents research on external ratings of teacher education programs (TEPs) produced by the National Council on Teacher Quality (NCTQ) and the results of an experiment to improve school rankings by providing information and feedback to TEPs about how to improve their ratings. Research suggests that higher education institutions are responsive to feedback and public ratings. The college rankings published by U.S. News and World Report (USNWR) shows how powerful ratings can be in changing higher education practices such as admissions requirements, financial aid disbursements, and policies. NCTQ released the first ranking of TEP’s in 2014 as a part of an ongoing effort to rate teacher training nationally. The intervention relied upon providing targeted information about specific programmatic changes that would improve the rating for a randomly selected sample of elementary teacher education programs. Average program ratings improved between 2013 and 2016, but we find no evidence that the information intervention increased program responsiveness to NCTQ’s rating effort. The results show that the experimental group had lower ratings than the control group in 2016.
Citation: Goldhaber, D., & Koedel, C. (2019). Public Accountability and Nudges: The Effect of an Information Intervention on the Responsiveness of Teacher Education Programs to External Ratings. American Educational Research Journal, 0002831218820863.
October 7, 2019
Decades of data attest to high rates of teacher turnover. Almost half of new teachers leave the profession within 5 years. For the past 10 years, turnover has leveled off at a disconcerting 16% per year. High turnover impedes student performance and diverts resources away from efforts to improve schools. It places large numbers of inexperienced, less effective teachers in classrooms, resulting in increased recruiting, hiring, and training budgets. With effective retention, the United States could save a meaningful portion of the $2.2 billion spent annually on replacing teachers. Research shows that increases in teacher turnover consistently correspond with decreases in achievement in core academic subjects. Attrition disproportionately affects schools with the greatest needs, low-achieving and high-poverty schools. Chronic turnover also negatively impacts a school’s culture, increasing student disciplinary problems and principal turnover. It damages collegiality, adding chaos and complexity to schoolwide operations and perpetuating new cycles of turnover. Effective interventions can remediate this situation, but they require administrators’ long-term commitment to improve the learning environment and working conditions.
Citation: Donley, J., Detrich, R, Keyworth, R., & States, J. (2019). Teacher Turnover Impact. Oakland, CA: The Wing Institute. https://www.winginstitute.org/teacher-retention-turnover