Categories for Societal Outcomes
November 28, 2018
Evaluating the Relationships Between Poverty and School Performance
One of the most critical issues facing K-12 education is the impact that poverty has on school performance. This study first examines school performance using traditional metrics for school poverty levels (percent of student body that qualify for free and reduced lunch: FRL) and school performance (school achievement based on the aggregate test scores of its student body). The results support prior research documenting the negative relationship between the level of poverty in a school and student achievement (the higher the poverty the lower the achievement). However, when replacing the student achievement metric with a student growth metric, the relationship is significantly different.
This NWEA study argues that, while it is important to measure and report a school’s student achievement, it is often a function of the demographics of a school’s population rather than a school’s effectiveness at teaching. Student growth tracks the learning of students regardless of their poverty level and is a more useful tool for comparing individual school performance. Sixty percent of schools with high poverty student populations had above average student growth. And a larger percentage of high poverty schools demonstrated substantial growth than schools from wealthy communities. The dramatic negative relationship between poverty and student achievement was much less evident when looking at student growth, and much more nuanced.
The implications of the study are profound. First, it highlights the need for school performance measures to include student growth in addition to school achievement. Both are critical measures. More importantly, it raises the question: If students make comparable progress during the school year regardless of their poverty level, what accounts for the significant differences in test scores. One study tracked the performance of students by poverty level and their performance on tests administered at the beginning and end of each school year. The data showed a clear pattern. Students of all socio economic groups made comparable progress during the school year. The biggest, and compounding gaps, occurred during the summer months suggesting that poverty’s largest impact occurs outside of school.
Citation:Alexander, K. L., Entwisle, D. R., & Olson, L. S. (2001). Schools, achievement, and inequality: A seasonal perspective. Educational Evaluation and Policy Analysis, 23, 171–191.
October 29, 2018
11 Million Days Lost: Race, Discipline, and Safety at U.S. Public Schools: Part I
Research tells us that student engagement is one of the most important components of a classroom strategy to facilitate student learning, as is effective teaching, a systematic instruction pedagogy, and evidence-based curriculum. Yet none of these interventions matter if a student is not in school. There are an increasing number of studies examining student absenteeism and its negative impact on student achievement. This descriptive summary is one of the first reviews to examine the number of days of “lost instruction” resulting from student suspensions. The study examines the total number of days lost nationwide, disparities among different student subgroups, and differences across individual states. Read more Overall, students lost a total of 11,360,004 days of instruction as a result of school suspensions. In order to facilitate comparisons of student subgroups with different enrollment numbers, the study uses a metric of “days of lost instruction per 100 students enrolled”. The resulting analysis documents significant disparity between subgroups. Black students lost 66 days of instruction (for every 100 students) compared to 14 for White students (the national average was 23 lost days). Students with disabilities lost 44 days of instruction, twice the 20 lost days of those without disabilities. Individual states had radically different performance in this area, with North Carolina averaging 51 days lost instruction per 100 enrollment annually and Utah 5 days. When examined by subgroup data, some states have extremely large instructional day loss. Black students lost over 100 days of lost instruction per 100 enrolled in Ohio, Michigan, Mississippi, Tennessee, and Virginia. The impact of loss of instruction due to suspensions has a lifelong impact on students, including: lower graduation rates (Rumberger and Losen, 2017), increased involvement in the juvenile justice system (Mowicki, 2018), and arrests as adults Rosenbaum (2018)
Citations: Russell W. Rumberger and Daniel J.Losen, The Hidden Cost of California’s Harsh School Discipline, The Civil Rights Project at UCLA, (2017) Retrieved from http://www.schooldisciplinedata.org/ccrr/docs/CostofSuspensionReportFinal.pdf
Janet Rosenbaum (2018). Educational and Criminal Justice Outcomes 12 Years After School Suspension. Youth & Society.
Jacqueline M. Mowicki, Discipline Disparities for Black Students, Boys and Students with Disabilities, GAO (March 2018). http://www.gao.gov/assets/700/690828.pdf
October 22, 2018
Teacher Soft Skills Overview
A teacher’s success is predicated on effective mastery of two requisite skill categories: technical competencies and personal competencies (soft skills). Technical skills are the specific skills and factual knowledge intrinsic to a specific job. Technical competencies elemental to teaching include instruction, assessment, and classroom management. Personal competencies, on the other hand, are skills broadly applicable to almost all professions; they create the foundation that enables a person to effectively use technical skills. Personal competencies basic to teaching include high expectations, love of learning, active listening, ability to adapt to novel situations, empathy, cultural sensitivity, positive regard for students, and good time management. Personal competency research shows large effect sizes, ranging from 0.72 to 0.87, for effective teacher-student relations that increase student academic performance and improve classroom climate. Unfortunately, teacher preparation and on-the-job staff development neglect this important training. To remedy the situation, more research is required to better define the field of personal competencies, and expanded training, including coaching, must be adopted during pre-service and induction.
Citation: States, J., Detrich, R. & Keyworth, R. (2018). Overview of Teacher Soft Skills.Oakland, CA: The Wing Institute. https://www.winginstitute.org/teacher-compentencies-soft-skills.
October 15, 2018
Restorative justice in Oakland schools. Implementation and impact: An effective strategy to reduce racially disproportionate discipline, suspensions, and improve academic outcomes.
Schools around the United States continue to use zero-tolerance disciplinary policies and practices to suspend or expel students for minor behavioral infractions, such as verbal disrespect, fighting, or truancy Compelling evidence suggests that zero tolerance disciplinary policies and practices used for decades have proven ineffective. This study examines the impact of The Whole School Restorative Justice Program (WSRJ). WSRJ utilizes a multi-tiered strategy. Tier 1 is regular classroom circles, Tier 2 is repair harm/conflict circles, and Tier 3 includes mediation, family group conferencing, and welcome/re-entry circles to initiate successful re-integration of students being released from juvenile detention centers.The key findings of this report show decreased problem behavior, improved school climate, and improved student achievement. In WSRJ schools, suspensions were cut in half (34% to 14%). Chronic absences diminished in WSRJ middle and high schools. Reading levels for ninth graders increased more in WSRJ schools than in non-WSRJ schools and four-year graduation rates increased over control schools.
Citation: Jain, S., Bassey, H., Brown, M. A., & Kalra, P. (2014). Restorative justice in Oakland schools. Implementation and impact: An effective strategy to reduce racially disproportionate discipline, suspensions, and improve academic outcomes. http://www.rjtica.org/wp-content/uploads/2015/04/OUSD-RJ-Report-full.pdf
October 1, 2018
Excessive student absenteeism (chronic absences) can have a devastating impact on student achievement. It’s significance was recognized in the recently passed Every Student Succeeds Act (ESSA) which now requires all states to include in their school report cards how many students are chronically absent. The Data Matters report analyzes the data in an attempt better understand the relationship between chronic absences and a wide range of variables: chronic absences by year, state, schools, poverty, ethnicity, and other school characteristics. Key findings include: (1) Nearly eight million students in the nation (15%) were chronically absent in the 2015-16 school year, (2) While chronic absenteeism affects all states, there is great variability between individual states ranging from 12% to 31% students with chronic absences, (3) over half (52%) of chronic absenteeism comes from students in schools with high or extreme levels of chronic absenteeism, and (4) schools with higher concentrations of low income students are more likely to experience chronic absenteeism. The report provides recommendations and strategies for managing chronic absenteeism at all levels of education leadership, from state agencies through individual schools. It also has an interactive web site where the reader can drill down on specific data at all levels of the education system. www.attendanceworks.org
Citation: Chang, Hedy N., Bauer, Lauren and Vaughan Byrnes, Data Matters: Using Chronic Absence to Accelerate Action for Student Success, Attendance Works and Everyone Graduates Center, September 2018.
October 1, 2018
Standardized tests play a critical role in tracking and comparing K-12 student progress across time, student demographics, and governing bodies (states, cities, districts). Each individual state establishes its own curricula, standardized tests, and achievement (proficiency) standards. As these decisions vary significantly across states and time, it becomes difficult to establish common metrics that would assess the true picture of a state’s performance. One methodology is to benchmark the each state’s proficiency standards against those of the National Assessment of Educational Progress (NAEP) test. This study does just that. Using NAEP as a common yardstick allows a comparison of different state assessments. The results confirm the wide variation in proficiency standards across states. It also documents that the significant majority of states have standards are much lower than those established by the NAEP.
At the national level, the National Assessment of Educational Progress (NAEP) test is the gold standard. It has been administered annually since 1970 and establishes student achievement standards using rigorous and independent evaluation methodology to ensure they are reasonable, valid, and informative. The achievement levels (Basic, Proficient, and Advanced) identify what students should know and be able to do at each grade level in a specific subject area. Students performing at or above the Proficient level on NAEP assessments demonstrate solid academic performance and competency over challenging subject matter. Proficiency is the target academic performance benchmark for students.
The study provides two sets of data regarding this issue: (1) how the individual state standards themselves compare to NAEP standards and (2) where students would have been placed for both the state and national tests. In 2015, only two states had proficiency standards for reading at the same range as those of NAEP. Forty-one states had proficiency standards that corresponded with the Basic Level in NAEP. One state had standards that corresponded with below Basic. In other words, the achievement levels in most states for being identified as proficient are much lower than those identified by NAEP.
The second analysis examined the percent of students identified as proficient in each state and compares it to the percent of the same student population that would be considered proficient by NAEP standards. For example, Louisiana standards identified 66.8% of its students as being proficient across reading and math, only 24.8 % of the students would have been classified as proficient by NAEP. In Nebraska, 76.2% of students were classified as proficient as opposed to only 40.2% meeting the proficiency standard for NAEP.
Citation: Bandeira de Mello, V., Rahman, T., and Park, B.J. (2018). Mapping State Proficiency Standards Onto NAEP Scales: Results From the 2015 NAEP Reading and Mathematics Assessments (NCES 2018-159). U.S. Department of Education, Washington, DC: Institute of Education Sciences, National Center for Education Statistics.
September 10, 2018
Effectiveness of the Practice Style and Reciprocal Style of Teaching: A Meta-Analysis
This meta-analysis looks at the effectiveness of two strategies in teaching motor skills to students: practice and reciprocal. The research examined two of the 11 teaching strategies identified in Mosston’s Spectrum of Teaching Styles designed for teachers in physical education. Six studies met the criteria for inclusion in this paper. The practice strategy involves the student in the decision-making process. The reciprocal strategy assigns each learner to a specific role: One learner performs the task and the other is the observer who offers immediate and ongoing feedback using a criteria sheet designed by the teacher. At the end of the practice, the students switch roles.
The study showed a very large effect size of 1.16 for the practice strategy, and a large effect size of 0.94 for the reciprocal strategy. It would not be surprising to see these particularly large effect sizes moderated in subsequent replication studies (Makel & Plucker, 2014; van Aert & van Assen, 2018). The study confirms previous research on reciprocal teaching as an effective instructional strategy. Reciprocal teaching has been found to be a powerful strategy for teaching reading and other academic subjects. John Hattie (1995) reported an effect size of 0.74 for reciprocal teaching. The takeaway from this meta-analysis is that practice and reciprocal styles have positive effects on motor skill acquisition.
Chatoupis, C., & Vagenas, G. (2018). Effectiveness of the practice style and reciprocal style of teaching: A meta-analysis. Physical Educator, 75(2), 175–194.
Makel, M. C., & Plucker, J. A. (2014). Facts are more important than novelty: Replication in the education sciences. Educational Researcher, 43(6), 304–316.
van Aert, R. C. M., & van Assen, M. A. L. M. (2018). Examining reproducibility in psychology: A hybrid method for combining a statistically significant original study and a replication. Behavior Research Methods, 50(4),1515–1539.
Hattie, J. (2008). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. New York, NY: Routledge.
August 23, 2018
Promoting Open Science to Increase the Trustworthiness of Evidence in Special Education
The past two decades has seen an explosion of research to guide special educators improve the lives for individuals with disabilities. At the same time society is wrestling with the challenges posed by a post-truth age in which the public is having difficulty discerning what to believe and what to consider as untrustworthy. In this environment it becomes ever more important that researchers find ways to increase special educator’s confidence in the available knowledge base of practices that will reliably produce positive outcomes. This paper offers methods to increase confidence through transparency, openness, and reproducibility of the research made available to special educators. To accomplish this the authors propose that researchers in special education adopt emerging open science reforms such as preprints, data and materials sharing, preregistration of studies and analysis plans, and Registered Reports.
Citation:Cook, B. G., Lloyd, J. W., Mellor, D., Nosek, B. A., & Therrien, W. (2018). Promoting Open Science to Increase the Trustworthiness of Evidence in Special Education.
July 10, 2018
The Effect of Voucher Programs on Student Achievement: A Meta-Analysis
News Summary: This meta-analysis examines the effectiveness of school voucher programs for improving student achievement. The research is of special interest for three important reasons; (1) it examines an important policy issue relevant to educators and the public, (2) it is an example of a study designed to replicate previous research on an important topic, and (3) it highlights the importance of examining the cost effectiveness associated with implementing practices in real world settings.
The use of school vouchers as a means to improve the quality of education has been an attractive, although controversial. It has been touted as a way to use public funding to overcome academic deficits in school systems. Enabling parents to exercise choice as to where to send a child to school is enticing. It appeals to the belief that exercising “control” over where your child can go to school will have an impaction the quality of education. The important question is, does this type of structural intervention produce both statistically significant, but more importantly do school vouchers produce socially significant academic gains?
This study tries to answer this question by replicating a previous meta-analysis of school voucher programs by Shakeel, Anderson, and Wolf (2016). The studies included in both meta-analyses required that to be included in the research individual studies must use randomized control trials of school voucher programs reporting quantitative measures of reading and/or math performance. The Wing Institute chose to include this new item to feature the need for replicating research as means to increase confidence in the results and broaden our knowledge base on a topic. The results indicated that, compared to the original study, this meta-analysis obtained smaller effect sizes with larger standard errors. This is not surprising as most replication of original research report smaller effects. The results of this research although somewhat smaller are consistent with the Shakeel, Anderson, and Wolf (2016) study. More importantly, both studies found that although the effects were for the most part positive they were very small ranging from 0.080 effect size on reading to 0.135 effect size on math.
It is important to note these results are also similar to those found in the Yeh, S. S. (2007) research on this topic. Yeh goes beyond reporting on effect size to examine the importance of putting effect size in the context of cost-effectiveness. It becomes quite clear when one asks the question, are school vouchers a cost effective intervention designed to deliver significant change, this intervention is found wanting. Yeh in his research concludes that educators and the public will be better served by adopting a practice such as formative assessment, that has a greater effective size (0.90) and can be implemented at a much smaller cost.
(1) Bennett, M., Banerjee, H. L., Doan, L. N., Geib, T., and Burley, A. (2018). The Effect of Voucher Programs on Student Achievement: A Meta-Analysis. AERA Conference New York, NY. 10.302/1302823.
(2) Shakeel, M., Anderson, K., & Wolf, P. (2016). The participant effects of private school vouchers across the globe: A meta-analytic and systematic review.
(3) Yeh, S. S. (2007). The cost-effectiveness of five policies for improving student achievement. American Journal of Evaluation, 28(4), 416-436.
July 2, 2018
Postsecondary Institutions and Cost of Attendance in 2017-18; Degrees and Other Awards Conferred, 2016-17; and 12-Month Enrollment, 2016-17
News Summary: The purpose of this preliminary report is examine the most recent data on the cost of sending students to college in the Unities States. During the 2017–18 academic year, there were 6,642 Title IV institutions of this total, 2,902 were classified as 4-year institutions, 1,932 were 2-year institutions, and the remaining 1,808 were less-than-2-year institutions. Average tuition and required fees for full-time, first-time degree/certificate-seeking undergraduates at 4-year institutions increased across all institutional controls except private for-profit institutions from 2015-16 to 2017–18. Public institutions reported a roughly 2 percent increase for in-state students (to approximately $8,300) and for out-of-state students (to approximately $18,700). Private nonprofit institutions reported an increase of approximately 3 percent (to about $28,000). Private for-profit institutions reported average tuition and required fees of approximately $16,200 for 2017–18. This represents a decrease of over 1 percent when compared with the inflation-adjusted figure from 2015–16. Approximately 3.3 million students received degrees or certificates at 4-year degree-granting institutions with more than 58 percent obtained a bachelor’s degree.
Citation:Ginder, S. A., Kelly-Reid, J. E., & Mann, F. B. (2018). Postsecondary Institutions and Cost of Attendance in 2016-17; Degrees and Other Awards Conferred, 2015-16; and 12-Month Enrollment, 2015-16: First Look (Provisional Data). NCES 2017-075rev. National Center for Education Statistics.