Categories for Education Outcomes
January 7, 2020
School Principal Competencies. Research has consistently shown that principals play a critical role in determining the quality of teaching, and in turn, student learning and achievement. Recent meta-analytic reviews suggest that effective principals are highly competent in the following areas: 1) establishing and conveying the school’s vision, goals and expectations by modeling aspirational practices and promoting data use for continuous improvement; 2) building teachers’ professional capacity by providing targeted and job-embedded professional development, protecting instructional time, and selecting educators who are the “right fit” for the school; 3) creating a supportive organization for learning by sharing and distributing leadership, understanding and building on diversity, and strategically acquiring and allocating resources; 4) facilitating a high-quality student learning experience by developing and monitoring curriculum, instruction, and assessment, and creating learning environments that are personalized, safe, and orderly; and 5) connecting with external partners who can support fulfillment of school goals, and building productive and collaborative relationships with families. While these principal competencies are relevant for a range of school contexts, leaders operating in varying school environments (e.g., high/low poverty, urban/rural) must ultimately determine how best to enact them to optimize teaching and learning.
Citation: Donley, J., Detrich, R., States, J., & Keyworth, (2020). Principal Competencies. Oakland, CA: The Wing Institute. https://www.winginstitute.org/principal-competencies-research
Link: https://www.winginstitute.org/principal-competencies-research
December 17, 2019
Practice Outpacing Policy? Credit Recovery in American School Districts. Traditionally, students who failed classes could get credit by taking the class over during summer school. In a relatively brief period of time, most high schools have a undergone a complete and rapid transformation in this area, shifting to “credit recovery” programs to help at-risk students earn credits towards graduation for classes they have failed. These programs typically offer online alternatives to students rather than having them retake the failed class. In the 2015-16 school year approximately three out of four high schools offered credit recovery programs with about 6% of all students participating. This growth in credit recovery programs, rapid expansion of new online models, and numerous cases of misuse to inflate graduation rates highlight the need for research into the implementation and effectiveness of this intervention.
This study examined the policies and practices of a representative sample of high schools to identify how they structure their credit recovery programs. It found: Over 95% of the credit recovery program had online components (58% online with some in-person instruction, 37% were exclusively online). Seventy-eight percent used just one online credit recovery provider to manage their system. Eighty-seven percent of districts offer credit recovery programs year-round. Forty percent of the districts limited credit recovery to courses
Traditionally, students who failed classes could get credit by taking the course over during summer school. In a relatively brief time, most high schools have undergone a complete and rapid transformation in this area, shifting to “credit recovery” programs to help at-risk students earn credits towards graduation for classes they have failed. These programs typically offer online alternatives to students rather than having them retake the failed class. In the 2015-16 school year, approximately three out of four high schools offered credit recovery programs with, about 6% of all students participating. This growth in credit recovery programs, a rapid expansion of new online models, and numerous cases of misuse to inflate graduation rates highlight the need for research into the implementation and effectiveness of this intervention
This study examined the policies and practices of a representative sample of high schools to identify how they structure their credit recovery programs. It found: Over 95% of the credit recovery program had online components (58% online with some in-person instruction, 37% were exclusively online). Seventy-eight percent used just one online credit recovery provider to manage their system. Eighty-seven percent of districts offer credit recovery programs year-round. Forty percent of the districts limited credit recovery to courses required for graduation. Only 16% limited the option to twelfth graders, allowing students from all high school grades to avail themselves of the option. The study concluded that “many districts’ policies allow lots of flexibility for student access and assessment with relatively little constraint. Taken individually, these policies could be justifiable but taken together, and they leave credit recovery programs ripe for abuse.”
Citation: Malkus, N. (2019). Practice Outpacing Policy? Credit Recovery in American School Districts. American Enterprise Institute.
Web Address: https://www.aei.org/wp-content/uploads/2019/11/Practice-Outpacing-Policy.pdf
December 17, 2019
A Review of the Literature on Principal Turnover. When it comes to variables that impact student achievement, school principals consistently come up second, behind teachers. They play a critical role in supporting teachers and students to ensure a school’s success. Hiring and maintaining qualified school principals is as important as it is challenging. Yet, principal turnover remains high, impacting the consistency of services and school improvement efforts.
This paper examines research on what we know about the causes and impact of principal turnover. The author is interested in answering three questions: how principal turnover is defined and measured, identifying the causes of turnover, and understanding the consequences of high rates of turnover. The review of the literature shows that there are insufficient numbers and quality of studies on this topic. The current knowledge base consists of case studies and correlational research. The fact there is a shortage of experimental research is not surprising given challenges confronted in implementing research on principal turnover. Because of this, the author was unable to conduct a meta-analysis for this paper. It appears that principal turnover is closely related to student achievement and teacher turnover, but the research is not sufficient to make definitive conclusions at this time. Snodgrass concludes the field needs additional quality studies on this topic.
Citation: Snodgrass Rangel, V. (2018). A review of the literature on principal turnover. Review of Educational Research, 88(1), 87-124.
Link: https://journals.sagepub.com/doi/pdf/10.3102/0034654317743197
December 16, 2019
On the Reality of Dyslexia. This paper assesses research on the topic of dyslexia. Willingham’s piece is in response to comments made by literacy researcher, Dick Allington, in which he questions the legitimacy of the label, dyslexia. Answering this question is more than an academic exercise as having a clearer understanding of dyslexia is crucial if educators are to understand why 10% of students struggle to master reading, the skill essential to success in academic learning. Willingham highlights the etiology of the disorder, and he concludes that the ability to read is the product of the home environment, instruction at school, and genetics within the child. Dyslexia is a problem in the child’s ability to successfully master the skills of reading and is closely related to fluency in language. Dyslexia is not like measles in which you are ill, or you aren’t. Dyslexia is more like high blood pressure where individuals fall on a bell curve. Falling somewhere on the bell curve is supported by the hypothesis that the disorder is the complex interaction between multiple causes. Although it does not have a single source, dyslexia is successfully remediated through evidence-based language and reading instruction.
Citation: Willingham, D. (2019). On the Reality of Dyslexia. Charlottesville, VA.http://www.danielwillingham.com/daniel-willingham-science-and-education-blog/on-the-reality-of-dyslexia?utm_source=feedburner&utm_medium=email&utm_campaign=Feed%3A+nbspDanielWillingham-DanielWillinghamScienceAndEducationBlog+%28Daniel+Willingham%27s+Science+and+Education+Blog%29.
Link: On the Reality of Dyslexia
December 16, 2019
PISA 2018 Results (Volume I): What Students Know and Can Do. Benchmark Indicators are critical tools to help education stakeholders track their education system’s performance over time, in comparison to other similar level education systems (state, national, international), and by student groups (ethnicity, disabilities, socioeconomic status, etc.). One of the most respected tools for benchmarking system performance is the Program for International Student Assessment (PISA), which tests 15-year-old-students across nearly 80 countries and educational systems in reading, mathematics, and science. The results from the most recent testing (2018) were just released. The report itself has an enormous amount of data. A summary of key findings follows.
Performance Over Time
U.S. test performance, despite small fluctuations, has been virtually flat over the past twelve to eighteen years (depending on the subject area). The reading performance score was 504 in 2000 and 505 in 2018. Math performance got worse, dropping from 483 in 2003 to 478 in 2018. And science performance has remained the same over the last four testing periods, remaining at 502 between 2009 and 2018. Consistency is not inherently a bad thing depending on how well a system is performing. However, PISA data suggests that the U.S. system is significantly underperforming compared to other international systems (see following). Consistency is also a problem when one considers that unprecedented investments in school reform efforts during this time period (e.g. No Child Left Behind, School Improvement Grants, Race to the Top, and Every Student Succeeds) have failed to move the needle in any significant way.
Performance Compared to Other Countries
U.S performance in reading ranked thirteenth among participating nations in reading, thirty-eighth in math, and twelfth in science. These represent a slight improvement in rankings from the 2015 test results, but that is a reflection of several top performing nations had lower scores, not that the U.S. improved.
Performance Across Different Student Subgroups
One of the biggest takeaway’s from this report is the growing inequity in performance between the high student performers and low. The following chart looks at the average scores of a gap between student scores in the highest percent of academic achievement (90%) and scores at the lowest (10%).
Reading scores of the highest performing students have increased over the lasts two tests from 614 to 643, while the scores of the lowest 10% have decreased from 378 to 361. The result is a widening gap between the top and lowest performing students. While improving the scores of the best performing students is a laudable achievement, an education system must serve all of its students in the interest of equity.
Further analysis of the data shows a correlation between student performance differences and their socioeconomic status (SES). One of the metrics used to determine SES is whether or not students qualify for the National School Lunch Program.
This data shows a direct correlation between a school’s reading scores and the SES of its student body. The more low SES students, the lower the PISA reading, math, and science scores.
It is almost impossible to document cause and effect with data at this level of analysis and control. Still, when making policy and program decisions, we must use the best available evidence. In this case, the best available evidence portrays an education system that, despite significant school improvement efforts, has shown little or no improvement over time, performs worse than a significant number of other nations’ education systems and continues to have inequitable results.
Citation. OECD (2019), PISA 2018 Results (Volume I): What Students Know and Can Do, PISA, OECD Publishing, Paris, https://doi.org/10.1787/5f07c754-en.
Web Address. https://www.oecd.org/pisa/publications/pisa-2018-results-volume-i-5f07c754-en.htm
December 16, 2019
Examining racial/ethnic disparities in school discipline in the context of student-reported behavior infractions. Research strongly supports the existence of bias in human beings. Discrepancies between how teachers handle behavior management incidents for students of color and white students has been a concern of education researchers for well over a decade. This paper looks at the disproportionality of consequences for disciplinary infractions between these groups of students. The researchers were interested in determining whether students of color would show similar rates of suspensions, office referrals, personal warnings from a teacher, or warnings about their behavior sent home based on ethnicity, as is the case for white students. Wegman’s study finds that African American students are less likely to receive warnings for behavior infractions than white peers, resulting in escalating consequences for students of color. The unequal handling of disciplinary actions reflects a pressing need for schools to address issues of implicit and explicit bias as a means to address this central issue in education.
Citation: Wegmann, K. M., & Smith, B. (2019). Examining racial/ethnic disparities in school discipline in the context of student-reported behavior infractions. Children and Youth Services Review, 103, 18-27.
Link: https://www.sciencedirect.com/science/article/pii/S0190740918311095
December 5, 2019
The Effect of Principal Behaviors on Student, Teacher and School Outcomes: A Systematic Review and Meta-Analysis of the Empirical Literature. This meta-analysis finds a positive relationship between school principals spending time on five commonly assigned roles and student achievement. These principal responsibilities are instructional management, internal relations, organizational management, administration, and external relations. The study finds that a principal cannot focus on a select few of the categories, but must carve out adequate time for each role. The need to be proficient across all leadership categories offers little comfort to U.S. principals who report average work-weeks of 58.6 hours (U.S. Department of Education National Center for Education Statistics, 2017). The paper recommends school principals be provided with additional resources if they are to adequately meet the needs of the students, teachers, and the community. Although these roles differ from the responsibilities researched by Viviane Robinson (goal expectations, strategic resourcing, teaching and curriculum, teacher development, and supportive environment), there does appear to be a significant overlap with those identified in the Liebowitz meta-analysis (Robinson, Lloyd, & Rowe, 2008).
Citation: Liebowitz, D. D., & Porter, L. (2019). The Effect of Principal Behaviors on Student, Teacher, and School Outcomes: A Systematic Review and Meta-Analysis of the Empirical Literature. Review of Educational Research, 89(5), 785-827.
Link: https://journals.sagepub.com/doi/abs/10.3102/0034654319866133
November 14, 2019
Using Resource and Cost Considerations to Support Educational Evaluation: Six Domains. Assessing cost, along with the effectiveness of an initiative is common in public policy decision-making, but is frequently missing in education decision-making. Understanding the cost-effectiveness of an intervention is essential if educators are to maximize the impact of an intervention given limited budgets. Education is full of examples of practices, such as class-size reduction and accountability through high-stakes testing, that produce minimal results while consuming significant resources. It is vital for those making critical decisions to understand which practice is best suited to meet the needs of the school and the schools’ students that can be implemented using the available resources. The best way to do this is through the use of a cost-effectiveness analysis (CEA).
A CEA requires an accurate estimation of all added resources needed to implement the new intervention. Costs commonly associated with education interventions include; added personnel, professional development, classroom space, technology, and expenses to monitor effectiveness. The second variable essential to a CEA is the selection of a practice supported by research. In the past twenty years, a significant increase in the quality and quantity of research supporting different education practices has occurred. A CEA compares the extra expenditures required to implement a new intervention to current practices against targeted education outcomes. Examples of educational outcomes are standardized test scores, graduation rates, or student grades.
The focus of this essay is on which economic methods can complement and enhance impact evaluations. The authors propose the use of six domains to link intervention effectiveness to the best technique needed to determine which practice is the most cost-effective choice. The six domains outlined in the paper are outcomes, treatment comparisons, treatment integrity, the role of mediators, test power, and meta-analysis. This paper provides examples of how analyzing the costs associated with these domains can complement and augment practices in evaluating research in the field of education.
Citation: Belfield, C. R., & Brooks Bowden, A. (2019). Using Resource and Cost Considerations to Support Educational Evaluation: Six Domains. Educational Researcher, 48(2), 120-127.
Link: https://edre.uark.edu/_resources/pdf/er2018.pdf
November 7, 2019
What Do Surveys of Program Completers Tell Us About Teacher Preparation Quality? Identifying which teacher preparation programs produce highly qualified teachers is understood to be a means to improve the effectiveness of teacher preparation programs (TTP). One proposed method for measuring TTP effectiveness is surveying recent graduate’s satisfaction with the training received. Research suggests that analysis of the surveys correlate satisfaction with teacher classroom performance, evaluation ratings, and retention data. If correct, this data offers schools a wealth of information to aid in deciding which pre-service programs to focus recruiting efforts. It also suggests that surveys can provide data for holding TTP accountable.
But much of the available research lacks sufficient rigor. This paper uses survey data from teachers in the state of North Carolina to gauge graduate’s satisfaction with TTP training to raise the validity and reliability of the study’s findings. The study concludes perceptions of preparation programs are modestly associated with the effectiveness and retention of first and second-year teachers. The researchers find, on average, those who feel better prepared to teach are more effective and more likely to remain in teaching. These results indicate that surveys of preparation program graduate satisfaction be monitored to assure validity and reliability of polling, given the interest accreditation bodies, state agencies, and teacher preparation programs show in using this data for high stakes decision making. The results also imply that surveys alone do not provide sufficient data to identify which programs offer the best teacher training.
Citation: Bastian, K. C., Sun, M., & Lynn, H. (2018). What do surveys of program completers tell us about teacher preparation quality? Journal of Teacher Education, November 2019.
Link: https://aefpweb.org/sites/default/files/webform/AEFP_NTPS_final.pdf
October 25, 2019
A Quantitative Synthesis of Research on Writing Approaches in Grades 2 to 12. This paper looks at randomized or well-matched control group research on outcomes of writing programs. The average effect size for those writing programs reviewed was a small effect size of 0.18. The writing programs fell into three categories; those that focus on the writing process, cooperative learning, or interactions between reading and writing.
The core characteristics of programs that produced the best writing outcomes include:
• Use of cooperative learning
• Structured approaches that give students step-by-step guides to writing
• Programs that teach students to assess their own and peer writing,
• Programs that balance writing with reading
• Programs that attempt to build students’ motivation to write and enjoy self-expression
• Programs that teach writing conventions (e.g., grammar, punctuation, usage) explicitly, but in the context of creative writing
• Programs that provide extensive professional development to teachers, in which teachers experience the writing strategies they will employ
Citation: Slavin, R. E., Lake, C., Inns, A., Baye, A., Dachet, D., & Haslam, J. (2019). A Quantitative Synthesis of Research on Writing Approaches in Grades 2 to 12. Best Evidence Encyclopedia.
Link: https://eric.ed.gov/?id=ED597167
