Categories for Monitoring

How effective is Peer Assessment?

March 4, 2020

Does Peer Assessment Promote Student Learning? A Meta-Analysis. Peer assessment has become a popular education intervention. In a peer assessment, student’s work is evaluated by a peer as opposed to the teacher. Extensive research is available on the reliability and validity of peer assessment results. A review of the literature finds few studies on the impact of Peer Review on student outcomes. This meta-analysis examines the effect sizes found in 58 studies. The paper finds a positive relationship for peer assessment on student outcomes. The study went on to identify the specific practice elements that comprise the practice of Peer Assessment to identify those elements that have the most significant impact on student performance. The study identified five components rater training, rating format, rating criteria, and frequency of peer assessment. The most critical factor of those examined is rater training.

Citation: Li, H., Xiong, Y., Hunter, C. V., Guo, X., & Tywoniw, R. (2020). Does peer assessment promote student learning? A meta-analysis. Assessment & Evaluation in Higher Education45(2), 193-211.

Linkhttps://www.researchgate.net/profile/Hongli_Li4/publication/333571244_Does_peer_assessment_promote_student_learning_A_meta-analysis/links/5d276f9d92851cf4407a70c2/Does-peer-assessment-promote-student-learning-A-meta-analysis

 


 

How effective are reading instruction programs?

February 25, 2020

Comparing Reading Research to Program Design: An Examination of Teachers College Units of Study. This report is the first of a new initiative by Student Achievement Partners to review different reading instructional programs that have been adopted and are widely used in schools.  The goal is to examine the programs in the context of best available research evidence regarding the critical components of teaching reading. Specifically, the components identified include:  phonics and fluency, text complexity, building knowledge and vocabulary, and English learner supports.  The first program reviewed was Units of Study from the Teachers College Reading & Writing Project.  The findings were detailed, but the overall conclusion was that, despite the curriculum’s many qualities, it would “be unlikely to lead to literacy success for all of the American public schoolchildren, given the research.”  The curriculum would support children who show up at school already reading or primed to read.  It would not meet the needs of children who need practice opportunities in a specific area of reading or language development.

Citation: Adams, M.J., Fillmore, L.W., Goldenberg, C., Oakhill, J., Paige, D.D., Rasinski, T., & Shanahan, T. (2020). Comparing Reading Research to Program Design: An Examination of Teachers College Units of Study. Student Achievement Partners.

Link: https://achievethecore.org/page/3240/comparing-reading-research-to-program-design-an-examination-of-teachers-college-units-of-study

 


 

Use of Four-Day School Week Schedules Increasing Despite Lack of Evidence on Outcomes

February 24, 2020

Shortened School Weeks in U.S. Public Schools”There is an increasing trend among schools and districts to reduce the school week from five days to four (longer) days.  Much of the impetus of this structural intervention comes from the perception that this schedule would generate significant cost savings.  Additionally, there is a belief that it positively impacts student achievement, teacher recruitment, and other quality indicators.  Unfortunately, the lack of experimental evidence makes it difficult to prove or disprove most of the claims.  

In the 2017-18 school year approximately 1.9% of public schools provided shortened school weeks.  Eight states had more than 10% of their schools on this schedule, with Wyoming having almost twenty percent of its schools on a four-day week.  In general, rural schools and those in the West were more likely to adopt this model.  (NCES, 2020).

The number of school districts operating on a four-day schedule grew by 466% over the last three years.   There were 120 districts in 21 states in 2016 and 560 districts in 25 states in 2019.  Over half of Colorado’s districts now operate on four-day weeks. (Walker, T., 2019)

Proponents claim that the model saves money, improves student performance, helps with teacher recruitment and retention, reduces student absenteeism, and improves the quality of life for all involved as they have an extra day away from school to take care of personal business.  Opponents challenge these claims and highlight the potential new costs for parents, loss of wages for support professionals, and reduced access to services for low income students.

Regarding potential savings, the analysis is straightforward.  The Education Commission of the States conducted a detailed analysis calculating that the maximum savings for a district was 5.43%, but that the more likely average is in the .4% to 2.5% range. Many of the largest costs such as salaries, facilities, administrative costs, etc. are not affected by fewer days (Griffith, M., 2011).

Unfortunately, large-scale experimental studies on the other stated pros and cons of this intervention are as of yet nonexistent.  The studies that have been completed are often non- or quasi-experimental and produce results that are inconsistent, inconclusive, or show negative impact (Heyward, G., 2018).  Additionally, it is difficult to evaluate the impact of this intervention because, as with most structural interventions, the 4 day school schedule does not represent any particular teaching or educational model.  It is just a work schedule.  Regardless, “the idea has proved contagious because adults like it”.  (Hill, 2017).

Citation(s):  NCES. (2020). Shortened School Weeks in U.S. Public Schools. NCES 2020-011. National Center for Education Statistics.

Griffith, M. (2011). What Savings Are Produced by Moving to a Four-Day School Week?. Education Commission of the States (NJ3).

Heyward, G. (2018). What Do We Actually Know About the Four-Day School Week? Center on Reinventing Public Education 

Hill, P.T. & Heyward, G.,  (2017).  A Troubling Contagion: The Rural 4-Day School Week.  Brookings Institute, Brown Center Chalkboard.

Long, C. (2019). Four-Day School Weeks More Popular, But Impact on Students and Educators Unclear. neaToday 

National Conference of State Legislatures (NCSL). (2019). Four-Day School Week Overview 

Walker, T. (2019). After Moving to a Four-Day School Week, There May Be No Going Back. neaToday

Link: https://www.ecs.org/wp-content/uploads/Instructional-Time-Trends_revised-1.pdf

 


 

How effective are teacher preparation programs teaching reading?

February 13, 2020

Teacher prep review: Program Performance in Early Reading Instruction.  Each school year, over one million public school 4th graders fail to achieve proficiency in reading. Evidence strongly suggests this is unnecessary as a roadmap with the potential to reduce the rate of reading failure from 3% to 1% of children is available. 

The National Council of Teacher Quality (NCTQ) review examines teacher preparation program progress in adopting the necessary components of evidence-based reading instruction. The report continues the effort of two previous reports offering educators a look at trends on preparation program progress on providing this essential training. NCTQ assesses Teacher preparation programs on the instruction of the five components of effective reading instruction: phonemic awareness, phonics, fluency, vocabulary, and comprehension. The NCTQ assessed preparation programs on three criteria; teaching all five components, the inclusion of high-quality textbooks offering detailed information on all five factors, and requiring teacher candidates to demonstrate mastery of the five skill sets.

The findings reveal that preparation programs are making persistent progress delivering instruction for all five reading components. The report concludes the field of teacher preparation is at a critical juncture in reading training as the data suggests a continued momentum in favor of science-based reading instruction. This year’s report finds undergraduate programs consistently improved scientifically-based reading instruction since NCTQ first began these studies. This report finds 57 percent now earning an A or B. This growth represents a 10-point improvement when compared to 2016 and an 18-point increase over the 2013 Teacher Prep Review. 

Figure 1. Drake, G., et al. (2020)

Although this progress is to lauded, it is important to note only half of the programs provide instruction in phonemic awareness, the first step in mastering reading. The study also finds teachers are not any more likely to learn the importance of fluency, with only 53 percent of programs providing adequate coverage of this component. 

Citation: Drake, G., et al. (2020). Teacher Prep Review: Program Performance in Early Reading Instruction. National Council on Teacher Quality.https://www.nctq.org/dmsView/NCTQ_2020_Teacher_Prep_Review_Program_Performance_in_Early_Reading_Instruction

Link: https://www.nctq.org/dmsView/NCTQ_2020_Teacher_Prep_Review_Program_Performance_in_Early_Reading_Instruction

 


 

Is Dyslexia a Genuine Phenomenon?

December 16, 2019

On the Reality of Dyslexia. This paper assesses research on the topic of dyslexia. Willingham’s piece is in response to comments made by literacy researcher, Dick Allington, in which he questions the legitimacy of the label, dyslexia. Answering this question is more than an academic exercise as having a clearer understanding of dyslexia is crucial if educators are to understand why 10% of students struggle to master reading, the skill essential to success in academic learning.  Willingham highlights the etiology of the disorder, and he concludes that the ability to read is the product of the home environment, instruction at school, and genetics within the child. Dyslexia is a problem in the child’s ability to successfully master the skills of reading and is closely related to fluency in language. Dyslexia is not like measles in which you are ill, or you aren’t. Dyslexia is more like high blood pressure where individuals fall on a bell curve. Falling somewhere on the bell curve is supported by the hypothesis that the disorder is the complex interaction between multiple causes. Although it does not have a single source, dyslexia is successfully remediated through evidence-based language and reading instruction.

Citation: Willingham, D. (2019). On the Reality of Dyslexia. Charlottesville, VA.http://www.danielwillingham.com/daniel-willingham-science-and-education-blog/on-the-reality-of-dyslexia?utm_source=feedburner&utm_medium=email&utm_campaign=Feed%3A+nbspDanielWillingham-DanielWillinghamScienceAndEducationBlog+%28Daniel+Willingham%27s+Science+and+Education+Blog%29.

LinkOn the Reality of Dyslexia

 


 

Most Recent Test Results from Pisa On Reading, Math and Science

December 16, 2019

PISA 2018 Results (Volume I): What Students Know and Can Do. Benchmark Indicators are critical tools to help education stakeholders track their education system’s performance over time, in comparison to other similar level education systems (state, national, international), and by student groups (ethnicity, disabilities, socioeconomic status, etc.).  One of the most respected tools for benchmarking system performance is the Program for International Student Assessment (PISA), which tests 15-year-old-students across nearly 80 countries and educational systems in reading, mathematics, and science.  The results from the most recent testing (2018) were just released.  The report itself has an enormous amount of data.  A summary of key findings follows. 

Performance Over Time

U.S. test performance, despite small fluctuations, has been virtually flat over the past twelve to eighteen years (depending on the subject area).  The reading performance score was 504 in 2000 and 505 in 2018.  Math performance got worse, dropping from 483 in 2003 to 478 in 2018.  And science performance has remained the same over the last four testing periods, remaining at 502 between 2009 and 2018.  Consistency is not inherently a bad thing depending on how well a system is performing.  However, PISA data suggests that the U.S. system is significantly underperforming compared to other international systems (see following).  Consistency is also a problem when one considers that unprecedented investments in school reform efforts during this time period (e.g. No Child Left Behind, School Improvement Grants, Race to the Top, and Every Student Succeeds) have failed to move the needle in any significant way.

Performance Compared to Other Countries

U.S performance in reading ranked thirteenth among participating nations in reading, thirty-eighth in math, and twelfth in science.  These represent a slight improvement in rankings from the 2015 test results, but that is a reflection of several top performing nations had lower scores, not that the U.S. improved.

Performance Across Different Student Subgroups

One of the biggest takeaway’s from this report is the growing inequity in performance between the high student performers and low.  The following chart looks at the average scores of a gap between student scores in the highest percent of academic achievement (90%) and scores at the lowest (10%).

Reading scores of the highest performing students have increased over the lasts two tests from 614 to 643, while the scores of the lowest 10% have decreased from 378 to 361. The result is a widening gap between the top and lowest performing students.  While improving the scores of the best performing students is a laudable achievement, an education system must serve all of its students in the interest of equity.

Further analysis of the data shows a correlation between student performance differences and their socioeconomic status (SES).  One of the metrics used to determine SES is whether or not students qualify for the National School Lunch Program.  

This data shows a direct correlation between a school’s reading scores and the SES of its student body.  The more low SES students, the lower the PISA reading, math, and science scores.

It is almost impossible to document cause and effect with data at this level of analysis and control.  Still, when making policy and program decisions, we must use the best available evidence.  In this case, the best available evidence portrays an education system that, despite significant school improvement efforts, has shown little or no improvement over time, performs worse than a significant number of other nations’ education systems and continues to have inequitable results.

Citation. OECD (2019), PISA 2018 Results (Volume I): What Students Know and Can Do, PISA, OECD Publishing, Paris, https://doi.org/10.1787/5f07c754-en.

Web Address. https://www.oecd.org/pisa/publications/pisa-2018-results-volume-i-5f07c754-en.htm

 


 

How effective is the use of video in professional development?

November 12, 2019

A systematic review of single-case research on video analysis as professional development for special educators. Professional development is viewed as essential to providing teachers with the skills needed to be successful in the classroom. Research strongly supports the need to go beyond the typical in-service training that is commonly provided teachers. Coaching and feedback have been found to be very effective in increasing the likelihood that training will be implemented in classrooms. The use of video has been offered as a cost-effective way to trainers to provide feedback to teachers in training based on actual performance in classroom use of the new skill(s).

Citation: Morin, K. L., Ganz, J. B., Vannest, K. J., Haas, A. N., Nagro, S. A., Peltier, C. J., … & Ura, S. K. (2019). A systematic review of single-case research on video analysis as professional development for special educators. The Journal of Special Education53(1), 3-14.

Linkhttps://www.researchgate.net/publication/328538866_A_Systematic_Review_of_Single-Case_Research_on_Video_Analysis_as_Professional_Development_for_Special_Educators

 


 

What tools do state departments of education provide for districts to evaluate curriculum?

October 24, 2019

State Department of Education Support for Implementation Issues Faced by School Districts during the Curriculum Adoption Process. The results of this systematic review of the websites of all 50 of the departments of education in the United States show that relatively few states provide state-created curriculum evaluation tools in the areas of English/language arts and mathematics, and only one state provides a curriculum evaluation tool that thoroughly addresses issues of implementation. In the area of English/language arts, the implementation issue most commonly addressed is fit of an instructional program with the district. Evidence demonstrating the effectiveness of a curriculum and the district’s capacity to effectively implement a curriculum are the next two most frequently addressed implementation-related issues. In the area of mathematics, fit with the district is also the most commonly addressed implementation-related issue. The next two most frequently addressed implementation-related issues are supports for the personnel implementing the curriculum and the capacity of the district to successfully implement. Only one state provided a state-created evaluation tool that thoroughly addressed all aspects of implementation as defined by The Hexagon Tool. Interestingly, this tool was generic. It was not designed to be used with English/language arts or mathematics curricula specifically, but with a variety of innovations that districts may consider adopting.

Citation: Rolf, R., R. (2019). State Department of Education Support for Implementation Issues Faced by School Districts during the Curriculum Adoption Process. Oakland, CA: The Wing Institute. https://www.winginstitute.org/student-research-2019.

Link: https://drive.google.com/file/d/1x_RZANxBKX4EnpP339ja8dxR_e5YmHav/view


 


 

What is the impact of cash incentives for grades on student cheating?

September 30, 2019

Do Pay-for-Grades Programs Encourage Student Cheating? Evidence from a randomized experiment.  Pay-for-grades programs are designed to increase student academic performance. One of the claims of those opposing such incentive systems is monetary incentives may lead to academic cheating. This randomized controlled study of 11 Chinese primary schools examines the effects of pay-for-grades programs on academic fraud. The study found widespread cheating behavior for students regardless of being in the control or experimental group, but no overall increase in the level of cheating for students in the pay-for-grades program. The authors conclude that educators need to be on the lookout for academic dishonesty, especially on standardized tests, while using moderate incentives to encourage student learning did not lead to increased levels of gaming the system.

Citation:  Li, T., & Zhou, Y. (2019). Do Pay-for-Grades Programs Encourage Student Academic Cheating? Evidence from a Randomized Experiment. Frontiers of Education in China14(1), 117-137.

Linkhttps://link.springer.com/article/10.1007/s11516-019-0005-9

 


 

Informal Teacher Evaluation (Original Wing Institute Paper)

September 3, 2019

Research on informal teacher evaluation reveals the predominant evaluation method is the walk-though, which ranges from a brief 2- to 3-minute snapshot to longer observation. Studies support the important role principals play in instructional leadership but also suggest that principals are not good at identifying which teachers are the best instructors. Research finds that principals overwhelmingly understand the need to sample teacher performance but that they are rarely trained in how to accomplish this. 

Citation: Cleaver, S., Detrich, R., & States, J. (2019). Informal Teacher Evaluation. Oakland, CA: The Wing Institute. Retrieved from https://www.winginstitute.org/staff-informal.

Link: https://www.winginstitute.org/staff-informal-walk-throughs