Categories for Decision Making
May 29, 2020
|The 2020 pandemic is unprecedented in living memory. This event necessitates schools adopting new technologies and teachers mastering new ways of delivering instruction. Education is engaged in a grand experiment, implementing new practices in fifty states with over 13,000 school districts. Change on this magnitude would be daunting even in normal times, and is particularly difficult in a decentralized system such as in the United States. What we know is there are bound to be many failures. Fortunately, the past 15 years have seen remarkable progress in the creation of a science of implementation to address such hurdles. This paper offers examples of failed practices in guiding schools to avoid making similar mistakes over the coming year. |
Citation: States, J., & Keyworth, R. (2020). Why Practices Fail. Oakland, CA: The Wing Institute. https://www.winginstitute.org/roadmap-overview
April 21, 2020
Fluency in education. Being fluent in something taught is essential if learning is readily accessible to the learner at a later date. How teachers measure student progress and define mastery rarely receives the attention it deserves. The distinguishing characteristic of mastery learning lies in both quick and accurate performance of a skill. The fluid combination of accuracy plus speed characterizes competent performance. To provide all students with retention, endurance, and application of instructional content, teachers must monitor performance with clear and universal measures and make decisions using standard data displays. The use of standard units of measurement and a standard graphical display are essential features of effective instruction. One such discovery, performance standards, has demonstrated that students can retain skills over significant amounts of time, perform at high rates with little performance decrement, and apply “element” skills to more sophisticated “compound” skills. It is essential teachers build fluency through providing students with adequate opportunities to practice lessons before moving on to the next topic. To sustain learning over time, instructors must monitor performance days, weeks, and even months after completion of a lesson. Unless continuous monitoring of past experiences occurs, prerequisite skills will be lost and unavailable to the student when needed in future lessons.
Citation: Kubina, R. M., & Morrison, R. S. (2000). Fluency in education. Behavior and Social Issues, 10(1), 83-99.
Additional Fluency Research: Datchuk, S. M., & Hier, B. O. (2019). Fluency Practice: Techniques for Building Automaticity in Foundational Knowledge and Skills. TEACHING Exceptional Children, 51(6), 424-435.
Reading Fluency : Rasinski, T. (2006). Reading fluency instruction: Moving beyond accuracy, automaticity, and prosody. The Reading Teacher, 59(7), 704-706.
Samuels, S. J. (2006). Toward a Model of Reading Fluency.
Rasinski, T. V., Blachowicz, C. L., & Lems, K. (Eds.). (2012). Fluency instruction: Research-based best practices. Guilford Press.
Mathematics Fluency: Burns, M. K., Codding, R. S., Boice, C. H., & Lukito, G. (2010). Meta-analysis of acquisition and fluency math interventions with instructional and frustration level skills: Evidence for a skill-by-treatment interaction. School Psychology Review, 39(1), 69-83.
Singer-Dudek, J., & Greer, R. D. (2005). A long-term analysis of the relationship between fluency and the training and maintenance of complex math skills. The Psychological Record, 55(3), 361-376
Language Fluency: Housen, A., & Kuiken, F. (2009). Complexity, accuracy, and fluency in second language acquisition. Applied linguistics, 30(4), 461-473.
Writing Fluency: Alvis, A. V. (2019). Predictors of Elementary-aged Students’ Writing Fluency Growth in Response to a Performance Feedback Writing Intervention
March 25, 2020
School Principal Retention Overview. Principals are critical to determining teaching quality, and in turn, student learning and achievement; retaining effective principals therefore is paramount, particularly in schools striving for rapid improvement. Principal turnover is higher in public charter than traditional public schools, in part because many charter schools are located in economically disadvantaged areas which have higher turnover rates generally. Less effective principals are more likely to leave their schools, which may imply the chance for improved school outcomes if they are replaced by more effective principals; however, research has yet to explore the extent to which this occurs. Working conditions found to influence principal turnover include negative disciplinary environments, lack of autonomy in decision-making regarding personnel and finances, and salary, whose impact is moderated by job benefits and other nonmonetary working conditions. District and policy characteristics such as tenure/union membership and policies intended to reduce teacher turnover also reduce the likelihood of principal turnover, as do high-quality professional development and support programs. Principal turnover incurs significant financial costs, and often leads to increased teacher turnover and decreased student achievement, unless a ready supply of more effective principals is available to replace low-performing ones. Evidence-based strategies to improve principal retention include coaching, mentoring and leadership supports tailored to a principal’s school context, and pipeline initiatives designed to increase the supply of high-quality candidates through recruitment, preparation, and ongoing development and support. Targeted financial incentives to work in high-needs schools coupled with improvements to principals’ working conditions can enhance retention, as can principal accountability systems that given principals increased autonomy but that also focus on ensuring they can build teacher capacity for the use of evidence-based instructional strategies.
Citation: Donley, J., Detrich, R., States, J., & Keyworth, (2020). Principal Retention Overview. Oakland, CA: The Wing Institute. https://www.winginstitute.org/quality-leadership-principal-retention.
March 23, 2020
The Adoption of Curricula in K-12 Schools: An Exploratory Qualitative Analysis. This exploratory qualitative study investigated how school districts engage in the process of adopting curricula for use in grades K-12 and what factors influence administrators when making adoption decisions. The author and a graduate student used a semi-structured interview protocol to interview 21 building- and district-level administrators employed by an economically and geographically diverse sample of school districts in the United States. After completing the interviews, the author and four researchers employed thematic analysis to analyze the data. Results suggest that the curriculum adoption process varies between school districts and, for some, from one curriculum adoption to the next. Most respondents reported engaging in at least one of the following activities during the adoption process: gathering information, initial screening, engaging committees, reviewing potential programs, piloting, and obtaining approval. The factors that influence administrators’ adoption decisions fall into four categories: alignment, need, evidence, and aspects of programs. Based on the data obtained in this study, the author proposes a sequence of activities to follow during a curriculum adoption.
Citation: Rolf, K. (2020). The Adoption of Curricula in K-12 Schools: An Exploratory Qualitative Analysis. Utah State University. https://drive.google.com/file/d/1O_rvmZKGE8rCf_nVTdOwgy4AVk-Gw6hH/view
March 12, 2020
What Works Clearinghouse: Procedures Handbook, Version 4.1. The WWC systematic review process offers educators and policy-makers a mechanism to assure consistent, objective, and transparent standards and procedures for assessing the impact of practices and interventions. The review procedures handbook includes the following changes: (1) Removal of the “substantively important” designation; (2) Addition of standard error calculations for all effect sizes; (3) Addition of single-case design (SCD) procedures for synthesizing SCD study findings using design-comparable effect sizes; (4) Addition of methods to estimate effects from regression discontinuity designs (RDDs); (5) Clarification of decision rules determining the use of difference-in-difference effect sizes; (6) Synthesis of studies within intervention reports using a fixed-effects model; (7) Modification of the intervention report effectiveness rating; and (8) Levels of evidence in practice guides.
Citation: What Works Clearinghouse: Procedures Handbook, Version 4.1. Princeton, NJ: What Works Clearinghouse https://files.eric.ed.gov/fulltext/ED602035.pdf
March 6, 2020
What do surveys of program completers tell us about teacher preparation quality? Over the past twenty years, educators, policymakers, and the public have increasingly expressed interest in finding out which teacher preparation programs (TPP) produce the best teachers. One of the tools offered to identify exemplary pre-service training is satisfaction surveys of graduates. A 2019 teacher survey finds, “only 30 percent of general education teachers feel ‘strongly’ that they can successfully teach students with learning disabilities, and only 50 percent believe those students can reach grade-level standards.” Surveys highlight a misalignment between the intended outcomes of teacher preparation and the actual worth of the training teacher candidates receive. Given the potential importance of teacher surveys, it is imperative that policymakers and teacher educators better understand the efficacy of polling for providing program accountability and information for improving TTP performance.
This study provides a large-scale examination of how new teacher’s perception of the quality TTP training is associated and predictive of quality instruction. The study finds that perceptions of TTP are modestly associated with the effectiveness and retention of first and second-year teachers. The authors find that new teachers who perceive training to be supportive in critical skills were more productive on the job, and were more likely to remain a teacher after the first year in the classroom. Supportive learning environments were associated with extensive training in establishing orderly and positive classroom learning environments, communicating high expectations for students, and forming supportive relationships with all students. Those teachers who received training in classroom management were more effectively develop strategies for addressing conduct issues that arise on the job. This evidence of supportive learning environments suggests that TPPs should consider ways, to enhance the quality of preparation opportunities to master classroom management, building relationships with students, and creating high expectations for student success.
Citation: Bastian, K. C., Sun, M., & Lynn, H. (2018). What Do Surveys of Program Completers Tell Us About Teacher Preparation Quality?. Journal of Teacher Education, 0022487119886294.
March 4, 2020
Does Peer Assessment Promote Student Learning? A Meta-Analysis. Peer assessment has become a popular education intervention. In a peer assessment, student’s work is evaluated by a peer as opposed to the teacher. Extensive research is available on the reliability and validity of peer assessment results. A review of the literature finds few studies on the impact of Peer Review on student outcomes. This meta-analysis examines the effect sizes found in 58 studies. The paper finds a positive relationship for peer assessment on student outcomes. The study went on to identify the specific practice elements that comprise the practice of Peer Assessment to identify those elements that have the most significant impact on student performance. The study identified five components rater training, rating format, rating criteria, and frequency of peer assessment. The most critical factor of those examined is rater training.
Citation: Li, H., Xiong, Y., Hunter, C. V., Guo, X., & Tywoniw, R. (2020). Does peer assessment promote student learning? A meta-analysis. Assessment & Evaluation in Higher Education, 45(2), 193-211.
February 21, 2020
The Current Controversy About Teaching Reading: Comments for Those Left with Questions After Reading the New York times Article. This Op-Ed commentary by Daniel Willingham discusses the current knowledge base on effective reading instruction in the context of a recent New York Times article on the topic. For over twenty years, the core components of effective reading (phonics, phonemic awareness, vocabulary, fluency, and comprehension) have been available to educators. Despite ample evidence, a large number of teacher preparation programs do not adequately train teachers on the best available evidence, many relying on an approach, “Balanced Literacy.” Balanced literacy was offered as a compromise to end the conflict between those advocating for phonics instruction and instructors promoting the immersion in relevant texts designed to motivate student’s learning. In practice, when Balanced Literacy is implemented, phonics instruction is frequently not included in the curriculum. Willingham concludes that decoding is the most thoroughly researched aspect of reading, decoding’s efficacy is well documented, and he suggests it is about time educators take advantage of this work.
Citation: Willingham, D. (2020). The Current Controversy About Teaching Reading: Comments for Those Left with Questions After Reading the New York times Article. University of Virginia: Daniel Willingham-Science & Education. http://www.danielwillingham.com/daniel-willingham-science-and-education-blog
New York Times Article: An Old and Contested Solution. https://www.nytimes.com/2020/02/15/us/reading-phonics.html
January 23, 2020
Why is this question important? Given the limited resources that are available for the education of children, it is important to select interventions that have the greatest impact we can afford. Using Stuart Yeh’s effectiveness cost ratio formula, a rough comparison can be drawn comparing class size reduction with other educational interventions.
Citation: Yeh, S. S. (2007). The Cost-Effectiveness of Five Policies for Improving Student Achievement, American Journal of Evaluation, 28(4), 416-436.
January 15, 2020
Overview of Value-Added Research in Education: Reliability, Validity, Efficacy, and Usefulness. Value-added modeling (VAM) is a statistical approach that provides quantitative performance measures for monitoring and evaluating schools and other aspects of the education system. VAM comprises a collection of complex statistical techniques that use standardized test scores to estimate the effects of individual schools or teachers on student performance. Although the VAM approach holds promise, serious technical issues have been raised regarding VAM as a high-stakes instrument in accountability initiatives. The key question remains: Can VAM scores of standardized test scores serve as a proxy for measuring teaching quality? To date, research on the efficacy of VAM is mixed. There is a body of research that supports VAM, but there is also a body of studies suggesting that model estimates are unstable over time and subject to bias and imprecision. A second issue with VAM is the sole use of standardized tests as a measure of student performance. Despite these valid concerns, VAM has been shown to be valuable in performance improvement efforts when used cautiously in combination with other measures of student performance such as end-of-course tests, final grades, and structured classroom observations.
Citation: Cleaver, S., Detrich, R. & States, J. (2020). Overview of Value-Added Research in Education: Reliability, Validity, Efficacy, and Usefulness. Oakland, CA: The Wing Institute. https://www.winginstitute.org/staff-value-added.