Education Drivers

Best Available Evidence Overview

Best available evidence is one of the cornerstones of evidence-based decision making. The implication is that evidence falls along a continuum from very strong evidence at one end to very weak evidence at the other end. In a perfect world, decisions would always be based on very strong evidence, but, in reality, many situations occur in which the evidence is less than very rigorous. There are three considerations in making judgments about the best available evidence: (1) the scientific rigor of the studies being evaluated, (2) the amount of evidence relevant to the specific problem, and (3) the relevance of the evidence to the specific situation under consideration. A study is deemed to be more rigorous if the experimenters have controlled for factors that might provide an alternative explanation for the results. Generally speaking, the greater the amount of evidence supporting an outcome, the more confidence decision makers have in their decisions. The relevance of a study to a particular practice question can vary depending on such factors as participants, interventions, settings, implementers, and outcomes.

Publications

TITLE
SYNOPSIS
CITATION
Evidence-Based Practice in the Broader Context: How Can We Really Use Evidence to Inform Decisions?

This paper provides an overview of the considerations when introducing evidence-based services into established mental health systems.

Chorpita, B. F., & Starace, N. K. (2010). Evidence-Based Practice in the Broader Context: How Can We Really Use Evidence to Inform Decisions? Journal of Evidence-Based Practices for Schools, 11(1), 4-29.

Evidence-Based, Empirically Supported, OR Best Practice?

Evidence-based, empirically-supported, and best practice are often used interchangeably. A case is made that for clarity each term should have a separate and distinct meaning.

Detrich, R. (2008). Evidence-Based, Empirically Supported, OR Best Practice?. Effective practices for children with autism, 1.

A roadmap to evidence-based education: Building an evidence-based culture

Increasing education’s reliance on evidence to guide decisions requires a significant change in the culture of districts and schools. This paper reviews the implications of moving toward evidence-based education.

Detrich, R., Keyworth, R., & States, J. (2007). A Roadmap to Evidence-based Education: Building an Evidence-based Culture. Journal of Evidence-based Practices for Schools, 8(1), 26-44.

Evidence-Based Education and Best Available Evidence: Decision-Making Under Conditions of Uncertainty

Evidence-based practice is a framework for decision making.  Even with high quality evidence there are likely sources of uncertainty that practitioners must confront.

Detrich, R., Slocum, T. A., & Spencer, T. D. (2013). Evidence-based education and best available evidence: decision-making under conditions of uncertainty. Evidence-Based Practices, 26, 21.

 

Presentations

TITLE
SYNOPSIS
CITATION
Identifying Research-based Practices for RtI: Scientifically Based Reading

This paper examines the types of research to consider when evaluating programs, how to know what “evidence’ to use, and continuums of evidence (quantity of the evidence, quality of the evidence, and program development).

Twyman, J. (2007). Identifying Research-based Practices for RtI: Scientifically Based Reading [Powerpoint Slides]. Retrieved from 2007-wing-presentation-janet-twyman.

Evolution of the Revolution: How Can Evidence-based Practice Work in the Real World?
This paper provides an overview of the considerations when introducing evidence-based services into established mental health systems.
Chorpita, B. (2008). Evolution of the Revolution: How Can Evidence-based Practice Work in the Real World? [Powerpoint Slides]. Retrieved from 2008-wing-presentation-bruce-chorpita.
If We Want More Evidence-based Practice, We Need More Practice-based Evidence
This paper discusses the importance, strengths, and weaknesses of using practice-based evidence in conjunction with evidence-based practice.
Cook, B. (2015). If We Want More Evidence-based Practice, We Need More Practice-based Evidence [Powerpoint Slides]. Retrieved from 2015-wing-presentation-bryan-cook.
From Evidence-based Practice to Practice-based Evidence: Behavior Analysis in Special Education
Evidence-based practice is a decision making framework. This talk reviews the types of evidence that can be used in decision-making and when each source of evidence is best used.
Detrich, R. (2006). From Evidence-based Practice to Practice-based Evidence: Behavior Analysis in Special Education [Powerpoint Slides]. Retrieved from 2006-calstatefresnoaba-presentation-ronnie-detrich.
Evidence, Ethics, and the Law
This paper reviews the legal and ethical basis for relying on scientifically supported interventions to improve outcomes for students.
Detrich, R. (2007). Evidence, Ethics, and the Law [Powerpoint Slides]. Retrieved from 2007-apbs-presentation-ronnie-detrich.
Evidence-based Education: It Isn't as Simple as You Might Think
On the face of it, the mandate to utilize scientifically supported interventions to improve outcomes seems obvious and straighforward. The paper reviews the challenges involved in doing so.
Detrich, R. (2007). Evidence-based Education: It Isn't as Simple as You Might Think [Powerpoint Slides]. Retrieved from 2007-calaba-ebe-presentation-ronnie-detrich.
An Expanded Model of Evidence-based Practice in Special Education
This paper reviews the types of evidence that can used to guide decision-making in special education as well as the necessity for high quality implementation, and monitoring the effects of intervention.
Detrich, R. (2006). An Expanded Model of Evidence-based Practice in Special Education [Powerpoint Slides]. Retrieved from 2006-campbell-presentation-ronnie-detrich.
Single Subject Research and Evidence-based Interventions: Are SSDs Really the Ugly Stepchild?
In most discussions about high quality research, single participant designs have been relegated to a lower status. This paper reviews the characteristics of SSDs and the contributions they can make to the evidence-base.
Detrich, R. (2007). Single Subject Research and Evidence-based Interventions: Are SSDs Really the Ugly Stepchild? [Powerpoint Slides]. Retrieved from 2007-aba-presentation-ronnie-detrich.
Evidence-based Education: Can We Get There From Here?
This paper reviews the steps that will be necessary to make evidence-based education a reality.
Detrich, R. (2008). Evidence-based Education: Can We Get There From Here? [Powerpoint Slides]. Retrieved from 2007-calaba-ebe-presentation-ronnie-detrich.
IDEIA and Evidence-based Interventions: Implications for Practitioners
The reauthorization of special education law (IDEIA) emphasizes using scientifically supported programs. This talk reviews the implications for special education practitioners.
Detrich, R. (2008). IDEIA and Evidence-based Interventions: Implications for Practitioners [Powerpoint Slides]. Retrieved from 2008-apbs-txint-presentation-ronnie-detrich.
The Ethical and Legal Basis for Evidence-based Education: Implications for the Profession
No Child Left Behind emphasized the importantance of utilizing practices that are scientifically supported. This paper reviews the implications of evidence-based education for the profession.
Detrich, R. (2008). The Ethical and Legal Basis for Evidence-based Education: Implications for the Profession [Powerpoint Slides]. Retrieved from 2008-teacher-presentation-ronnie-detrich.
The Four Assumptions of the Apocalypse
This paper examines the four basic assumptions for effective data-based decision making in education and offers strategies for addressing problem areas.
Detrich, R. (2009). The Four Assumptions of the Apocalypse [Powerpoint Slides]. Retrieved from 2009-wing-presentation-ronnie-detrich.
Evidence-based Practice for Applied Behavior Analysts: Necessary or Redundant
Evidence-based practice has been described as a decision making framework. This presentation describes the features and challenges of this perspecive.
Detrich, R. (2015). Evidence-based Practice for Applied Behavior Analysts: Necessary or Redundant [Powerpoint Slides]. Retrieved from 2013-aba-presentation-ronnie-detrich-tim-slocum-teri-lewis-trina.
Workshop: Evidence-based Practice of Applied Behavior Analysis.
Evidence-based practice is a decision-making framework that integrates best available evidence, professional judgement, and client values and context. This workshop described the relationship across these three dimensions of decision-making.
Detrich, R. (2015). Workshop: Evidence-based Practice of Applied Behavior Analysis. [Powerpoint Slides]. Retrieved from 2015-missouriaba-workshop-presentation-ronnie-detrich.
Data-Based Decision Making for Students Social Behavioral Difficulties
This paper discusses methods for making valid data-based decisions for student social behavior.
Gresham, F. (2009). Data-Based Decision Making for Students Social Behavioral Difficulties [Powerpoint Slides]. Retrieved from 2009-wing-presentation-frank-gresham.
Professional Judgment: Fallibility, Inevitability,and Manageability
This paper examines the many obstacles to effective professional judgment, the role it plays, and strategies for improving this critical function.
Keyworth, R. (2007). Professional Judgment: Fallibility, Inevitability,and Manageability [Powerpoint Slides]. Retrieved from 2007-aba-presentation-rk.
TITLE
SYNOPSIS
CITATION
Chronic Student Absenteeism: A Significant and Overlooked Obstacle to Student Achievement

This overview examines the best available evidence from a wide range of descriptive and correlational analyses executed by various state and city education departments, research groups, and academic researchers. Fortunately, the data paint an unequivocal picture. The results are overwhelmingly consistent across levels of analysis (school, students), units of measurement (achievement tests, graduation rates, dropout rates), areas of focus (reading, math, social indicators), units of education (grades, schools), and students of all demographics. Additionally, each analysis shows a linear relationship between absences and performance; the greater the number of absences, the worse the performance.

Alexander, K. L., Entwisle, D. R., & Olson, L. S. (2001). Schools, achievement, and inequality: A seasonal perspective. Educational Evaluation and Policy Analysis, 23(2), 171–191. 

Allison, M. A., Attisha, E., & AAP Council on School Health. (2019). The link between school attendance and good health. Pediatrics, 143(2). e20183648

Anguiano, M., Eastin D., Fine, M, Lockyer, B., Robles, D., Santana, M.,…Wong, K. (2015). Los Angeles Unified School District Report of the Independent Financial Review Panel. Los Angeles, CA: Los Angeles Unified School District.

Applied Survey Research. (2011). Attendance in early elementary grades: Associations with student characteristics, school readiness, and third grade outcomes. San Francisco, CA: Attendance Works.

Ashby, C. M. (2010). K–12 education: Many challenges arise in educating students who change schools frequently.Report to Congressional Requesters (GAO-11-40). Washington, DC: Government Accountability Office.

Attendance Works. (2018).3 tiers of intervention.Retrieved from https://www.attendanceworks.org/chronic-absence/addressing-chronic-absence/3-tiers-of-intervention/

Attendance Works and Everyone Graduates Center. (2017). Portraits of change: Aligning school and community resources to reduce chronic absence.Retrieved from https://www.attendanceworks.org/portraits-of-change/

Balfanz, R., & Byrnes, V. (2012). The importance of being in school: A report on absenteeism in the nation’s public schools. Baltimore, MD: Johns Hopkins University Center for Social Organization of Schools.

Balfanz, R., & Byrnes, V. (2013). Meeting the challenge of combating chronic absenteeism: Impact of the NYC mayor’s interagency task force on chronic absenteeism and school attendance and its implications for other cities. Baltimore, MD: Johns Hopkins School of Education.

Balfanz, R., Durham, R., & Plank, S. (2008). Lost days: Patterns and levels of chronic absenteeism among Baltimore City public school students 1999-00 to 2005-06. Baltimore, MD: Baltimore Education Research Consortium.

Balfanz, R., Herzog, L., & Mac Iver, D. J. (2007). Preventing student disengagement and keeping students on the graduation path in urban middle-grades schools: Early identification and effective interventions. Educational Psychologist42(4), 223–235. 

Baltimore Education Research Consortium. (2011). Destination graduation: Sixth grade early warning indicators for Baltimore city schools. Their prevalence and impact. Baltimore, MD: Author.

Bauer, L., Liu, P., Whitmore Schanzenbach, D., & Shambaugh, J. (2018). Reducing chronic absenteeism under the every student succeeds act. The Hamilton Project. Washington, DC: Brookings Institute. Retrieved from http://www.hamiltonproject.org/assets/files/reducing_chronic_absenteeism_under_the_every_student_succeeds_act.pdf

Buehler, M. H., Tapogna, J., Chang, H. N., & ECO Northwest, Ltd. (2012). Why being in school matters: Chronic absenteeism in Oregon Public Schools. Attendance Works. 

Burkam, D. T., Ready, D. D., Lee, V. E., & LoGerfo, L. (2004). Social-class differences in summer learning between kindergarten and first grade: Model specification and estimation. Sociology of Education, 77(1), 1­–31.

Chang, H. N., Bauer, L., & Byrnes, V. (2018). Data matters: Using chronic absence to accelerate action for student success.Attendance Worksand Everyone Graduates Center. 

Chang, H. N., & Romero, M. (2008). Present, engaged, and accounted for: The critical importance of addressing chronic absence in the early grades.New York, NY: National Center for Children in Poverty. 

Chen, C., & Stevenson, H. W. (1995). Motivation and mathematics achievement: A comparative study of Asian‐American, Caucasian‐American, and East Asian high school students. Child Development66(4), 1215–1234. 

Chingos, M., & Blagg, K. (2017). Making sense of state school funding policy.Washington, DC: Urban Institute. 

Coelho, R., Fischer, S., McKnight, F., Matteson, S., & Schwartz, T. (2015). The effects of early chronic absenteeism on third-grade academic achievement measures.Madison, WI: Robert M. La Follette School of Public Affairs, University of Wisconsin.

Connell, J. P., Spencer, M. B., & Aber, J. L. (1994). Educational risk and resilience in African‐American youth: Context, self, action, and outcomes in school. Child Development65(2), 493­–506.

da Costa Nunez, R., Erb-Downward, J., & Shaw-Amoah, A. (2015). Empty seats: The epidemic of absenteeism among homeless elementary students. New York, NY: Institute for Children, Poverty & Homelessness. Retrieved from https://www.attendanceworks.org/wp-content/uploads/2017/12/ICPH-Policy-Report_Empty-Seats_Chronic-Absenteeism.pdf

Digest of Education Statistics. (2017). Homeless students enrolled in public elementary and secondary schools, by grade, primary nighttime residence, and selected student characteristics: 2009-10 through 2015–16.Table 204.75a.Washington, DC: U.S. Department of Education, National Center for Education Statistics.

Downey, D. B., von Hippel, P. T., & Broh, B. A. (2004). Are schools the great equalizer? Cognitive inequality during the summer months and the school year. American Sociological Review, 69(5), 613–635.

Durán-Narucki, V. (2008). School building condition, school attendance, and academic achievement in New York City public schools: A mediation modelJournal of environmental psychology28(3), 278–286.

Epstein, J. L. & Sheldon, S. B. 2002. Present and accounted for: Improving student attendance through family and community involvementJournal of Educational Research 95(5): 308–318.

Fantuzzo, J. W., LeBoeuf, W. A., Chen, C. C., Rouse, H. L., & Culhane, D. P. (2012). The unique and combined effects of homelessness and school mobility on the educational outcomes of young children. Educational Researcher, 41(9), 393–402.

Fiester, L. (2010). Early warning! Why reading by the end of third grade matters.Kids Count special report. Baltimore, MD: Annie E. Casey Foundation.

GAO, (1994) Elementary school children: Many change schools frequently, harming their education,GAO/HEHS-94-45 (Washington, D.C.: Feb. 4, 1994).

Ginsburg, A., Jordan, P., & Chang, H. (2014). Absences Add Up: How School Attendance Influences Student Success. Attendance Works.

Gottfried, M. A. (2010). Evaluating the relationship between student attendance and achievement in urban elementary and middle schools: An instrumental variables approach. American Educational Research Journal47(2), 434–465.

Gottfried, M. A. (2014). Chronic absenteeism and its effects on students’ academic and socio-emotional outcomes. Journal of Education for Students Placed at Risk (JESPAR)19(2), 53–75.

Gottfried, M. A. (2015). Chronic absenteeism in the classroom context: Effects on achievement. Urban Education,54(1), 3–34.

Gottfried, M. A. & Hutt, E.L. (Eds.). (2019). Absent from school: Understanding and addressing student absenteeism.Cambridge, MA: Harvard Education Press.

Harris, Kamala. (2016). In School + On Track 2016.Sacramento, CA: Office of the Attorney General, State of California Department of Justice.

Henderson, T., Hill, C., & Norton, K. (2014). The connection between missing school and health: A review of chronic absenteeism and student health in Oregon.Portland, OR: Upstream Public Health.

Hernandez, D. (2011). Double jeopardy: How third-grade reading skills and poverty influence high school graduation.Baltimore, MD: Annie E. Casey Foundation.

Hess, F. M., & McShane, M. Q. (2018), Bush-Obama school reform: Lessons learned.Cambridge, MA: Harvard Education Press.

Johnson, G. M. (2005). Student alienation, academic achievement, and WebCT use. Journal of Educational Technology & Society8(2), 179–189.

Kearney, C. A. (2016). Managing school absenteeism at multiple tiers: An evidence-based and practical guide for professionals. New York City, NY: Oxford University Press.

Kearney, C. A., & Graczyk, P. (2014). A response to intervention model to promote school attendance and decrease school absenteeism. Child & Youth Care Forum,43(1), 1–25.

Lawrence, E. M., Rogers, R. G., & Zajacova, A. (2016). Educational attainment and mortality in the United States: Effects of degrees, years of schooling, and certification. Population Research and Policy Review35(4), 501–525.

 London, R, A., Sanchez, M., & Castrechini, S. (2016). The dynamics of chronic absence and student achievement. Education Policy Analysis Archives, 24(112), 1–27.

Losen, D. J., & Whitaker, A. (2018). 11 million days lost: Race, discipline, and safety at U.S. public schools. A joint report by the Center for Civil Rights Remedies of UCLA’s Civil Rights Project and the American Civil Liberties Union of Southern California.  

Mac Iver, M. A., & Messel, M. (2012). Predicting high school outcomes in the Baltimore city public schools.The Senior Urban Education Research Fellowship Series. Volume VII. Washington, DC: Council of the Great City Schools.

NAEP Data Explorer, 2015 and 2017 mathematics and reading assessments. Washington, DC: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress (NAEP). Retrieved from https://nces.ed.gov/nationsreportcard/naepdata/

NAEP Data Explorer, 2015 and 2017 reading and mathematics scale scores of 4th, 8th, and 12th graders and percentage absent from school, by selected characteristics and number of days absent in the last month. Table 227.50. Washington, DC: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress (NAEP). Retrieved from https://www2.ed.gov/datastory/chronicabsenteeism.html#one

National Forum on Education Statistics. (2009). Every school day counts: The forum guide to collecting and using attendance data (NFES 2009–804). Washington, DC: U.S. Department of Education, National Center for Education Statistics.

Nowicki,J. M. (2018). K–12 Education: Discipline disparities for Black students,boys, and students with disabilities. Report to Congressional Requesters. GAO-18-258. Washington, DC: Government Accountability Office.

Olsen, L. S. (2014). Why September matters: Improving student attendance.Policy brief. Baltimore, MD: Baltimore Education Research Consortium.

Office for Civil Rights. (2016). 2013–2014 civil rights data collection: A first look.

Railsback J. (2004). Increasing student attendance: Strategies from research and practice.Northwest Regional Educational Laboratory. Portland, OR.

Ready, D. D. (2010). Socioeconomic disadvantage, school attendance, and early cognitive development: The differential effects of school exposure. Sociology of Education 83(4): 271–286.

Robertson, A. A., & Walker, C. S. (2018). Predictors of justice system involvement: Maltreatment and education. Child Abuse & Neglect76, 408–415.

Robinson, C. D., Lee, M. G., Dearing, E., & Rogers, T. (2018). Reducing student absenteeism in the early grades by targeting parental beliefs. American Educational Research Journal55(6), 1163–1192.

Rogers, T., & Feller, A. (2018). Reducing student absences at scale by targeting parents’ misbeliefsNature Human Behaviour2(5), 335.

Romero, M., & Lee, Y. (2007). A national portrait of chronic absenteeism in the early grades.New York, NY: National Center for Children in Poverty, the Mailman School of Public Health at Columbia University.

Rumberger, R. W. (2015). Student mobility: Causes, consequences, and solutions. Boulder, CO: National Education Policy Center, University of Colorado.

Snell L., Smith, G. A., Koteskey, T., Joffe, M., & Bui, T. (2018). A 2018 evaluation of LAUSD’s fiscal outlook: Revisiting the findings of the 2015 Independent Financial Review Panel.Los Angeles, CA: Los Angeles Unified School District.

Telfair, J., & Shelton, T. L. (2012). Educational attainment as a social determinant of health. North Carolina Medical Journal, 73(5), 358–365.

Tourangeau, K., Nord, C., Lê, T., Sorongon, A. G., & Najarian, M. (2009). Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K): Combined User's Manual for the ECLS-K Eighth-Grade and K-8 Full Sample Data Files and Electronic Codebooks. NCES 2009-004. National Center for Education Statistics.

U.S. Department of Education. (2008). A uniform, comparable graduation rate: How the final regulations for Title I hold schools, districts, and states accountable for improving graduation rates. Retrieved from https://www2.ed.gov/policy/elsec/reg/proposal/uniform-grad-rate.html

U.S. Department of Education, Office for Civil Rights, 2013–2014 Civil Rights Data Collection. A First Look; Key Data Highlights on Equity and Opportunity Gaps in Our Nation’s Public Schools.Retrieved from https://www2.ed.gov/about/offices/list/ocr/docs/2013-14-first-look.pdf

U.S. Department of Education, Office for Civil Rights, 2015–2016 Civil Rights Data Collection. Chronic absenteeism in the nation’s schools: A hidden educational crisis. Retrieved from https://www2.ed.gov/datastory/chronicabsenteeism.html#one

Utah Education Policy Center [UEPC]. (2012). Research brief: Chronic absenteeism.Retrieved from https://www.schools.utah.gov/file/31291767-087c-4edb-8042-87f272507c1d

Beall’s List of Predatory Journals and Publishers

This news item offers a list of questionable, scholarly open-access publishers. In an era in which we are bombarded with volumes of research, it becomes ever more challenging to decide which journals and publishers are reputable. This web site reviews, assesses, and provides guidelines on how to decide which are trustworthy, whether you want to submit articles, serve as an editor, or serve on an editorial board. The web site provides a list that mostly consists of open access journals, although, a few non-open access publishers whose practices match those of predatory publishers have been added to the list.

 

Beall, J. (2012). Predatory publishers are corrupting open access. Nature, 489(7415), 179.

Professional development and teacher learning: Mapping the terrain

Teacher professional development is essential to efforts to improve our schools. This article maps the terrain of research on this important topic. It first provides an overview of what we have learned as a field, about effective professional development programs and their impact on teacher learning. It then suggests some important directions and strategies for extending our knowledge into new territory of questions not yet explored.

Borko, H. (2004). Professional development and teacher learning: Mapping the terrain. Educational Researcher30(8), 3–15.

Why Education Experts Resist Effective Practices (And What It Would Take To Make Education More Like Medicine

The first section of this essay provides examples from reading and mathematics curricula that show experts dispensing unproven methods and flitting from one fad to another. The middle section describes how experts, for ideological reasons, have shunned some solutions that do display robust evidence of efficacy. The following sections show how public impatience has forced other professions to "grow up" and accept accountability and scientific evidence. The paper concludes with a plea to develop education into a mature profession.

Carnine, D. (2000). Why Education Experts Resist Effective Practices (And What It Would Take To Make Education More Like Medicine).

 

How Methodological Features Affect Effect Sizes in Education

The purpose of this article is to examine how methodological features such as types of publication, sample sizes, and research designs affect effect sizes in experiments.

Cheung, A., & Slavin, R. E. (2015). How methodological features affect effect sizes in education. Best Evidence Encyclopedia, Johns Hopkins University, Baltimore, MD.

 

The Frontier of Evidence-Based Practice

These guidelines emphasized the dimensions of 1) efficacy and 2) effectiveness. A model is provided that proposes how evidence--however defined--will ultimately connect with practice. 

Chorpita, B. F. (2003). The frontier of evidence-based practice.

Teaching Secondary Students to Write Effectively

This practice guide released by What Works Clearinghouse presents three recommendations for helping students in grades 6 to 12 develop effective writing skills along with the strength of evidence to support the recommendations.

  • Explicitly teach appropriate writing strategies using a model-practice-reflect instructional cycle. Strong Evidence
  • Integrate writing and reading to emphasize key writing features. Moderate Evidence
  • Use assessments of student writing to inform instruction and feedback. Minimal evidence

Each recommendation includes specific actionable guidance for educators on implementing these practices in the classroom. It is geared toward administrators and teachers in all disciplines who want to help improve their students’ writing.

CLEARINGHOUSE, W.W. Teaching Secondary Students to Write Effectively.

Promoting Open Science to Increase the Trustworthiness of Evidence in Special Education.

The past two decades has seen an explosion of research to guide special educators improve the lives for individuals with disabilities. At the same time society is wrestling with the challenges posed by a post-truth age in which the public is having difficulty discerning what to believe and what to consider as untrustworthy. In this environment it becomes ever more important that researchers find ways to increase special educator’s confidence in the available knowledge base of practices that will reliably produce positive outcomes. This paper offers methods to increase confidence through transparency, openness, and reproducibility of the research made available to special educators. To accomplish this the authors propose that researchers in special education adopt emerging open science reforms such as preprints, data and materials sharing, preregistration of studies and analysis plans, and Registered Reports.

Cook, B. G., Lloyd, J. W., Mellor, D., Nosek, B. A., & Therrien, W. (2018). Promoting Open Science to Increase the Trustworthiness of Evidence in Special Education.

Visible learning: A synthesis of over 800 meta-analyses relating to achievement

Hattie’s book is designed as a meta-meta-study that collects, compares and analyses the findings of many previous studies in education. Hattie focuses on schools in the English-speaking world but most aspects of the underlying story should be transferable to other countries and school systems as well. Visible Learning is nothing less than a synthesis of more than 50.000 studies covering more than 80 million pupils. Hattie uses the statistical measure effect size to compare the impact of many influences on students’ achievement, e.g. class size, holidays, feedback, and learning strategies.

Hattie, J. (2008). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. New York, NY: Routledge.

 

Pitfalls of Data Analysis (or How to Avoid Lies and Damned Lies)

This paper examines things that people often overlook in their data analysis, and ways people sometimes "bend the rules" of statistics to support their viewpoint. It discusses ways you can make sure your own statistics are clear and accurate.

Helberg, C., (1995). Pitfalls of Data Analysis (or How to Avoid Lies and Damned Lies). Third International Applied Statistics in Industry Conference in Dallas, TX, June 5-7, 1995.

Some recommendations for the reporting of quantitative studies

This editorial offers recommendations aimed at providing examples of a series of elements that may significantly contribute towards demonstrating the robustness of quantitative results. It is therefore not a methodological guide but instead a guide that acts as a reminder of some basic principles when reporting quantitative research.

López, X., Valenzuela, J., Nussbaum, M., & Tsai, C. C. (2015). Some recommendations for the reporting of quantitative studies. Computers & Education, 91(C), 106-110.

Facts are more important than novelty: Replication in the education sciences

Despite increased attention to methodological rigor in education research, the field has focused heavily on experimental design and not on the merit of replicating important results. The present study analyzed the complete publication history of the current top 100 education journals ranked by 5-year impact factor and found that only 0.13% of education articles were replications. Contrary to previous findings in medicine, but similar to psychology, the majority of education replications successfully replicated the original studies. However, replications were significantly less likely to be successful when there was no overlap in authorship between the original and replicating articles. The results emphasize the importance of third-party, direct replications in helping education research improve its ability to shape education policy and practice.

Makel, M. C., & Plucker, J. A. (2014). Facts are more important than novelty: Replication in the education sciences. Educational Researcher, 43(6), 304–316.

Effective programs in elementary mathematics: A best-evidence synthesis

This research synthesis examines randomized controlled trials and quasi-experimental research on the mathematics achievement outcomes for elementary school programs. The best outcomes were found for tutoring programs. The findings suggest that programs emphasizing personalization, engagement, and motivation are most impactful in elementary mathematics instruction.

Pellegrini, M., Lake, C., Inns, A, & , Slavin, R. (2018). Effective programs in elementary mathematics: A best-evidence synthesis. Best Evidence Encyclopedia. Retrieved from http://www.bestevidence.org/word/elem_math_Oct_8_2018.pdf

Friends don’t let friends misuse NAEP data

This article shows some common type of misused or unhelpful NAEP analyses to look out for and avoid. This article also give some warning to avoid misuse of the NAEP data.

Polikoff, M.S. (2015). Friends don’t let friends misuse NAEP data. Retrieved from https://morganpolikoff.com/2015/10/6/friends-dont-let-friends-misuse-naep-data/

Stanfrod Education Data Archive

The Stanford Education Data Archive (SEDA) is an initiative aimed at harnessing data to help scholars, policymakers, educators, and parents learn how to improve educational opportunity for all children. The data are publicly available here, so that anyone can obtain detailed information about American schools, communities, and student success.

Stanfrod Education Data Archive. Standford Center for Education Policy Analysis. Retrieved from https://cepa.stanford.edu/seda/overview

The Number of Low-Performing Schools by State in Three Categories (CSI, TSI, and ATSI), School Year 2018-19.

This report, completed by the Center on Education Policy, attempts to provide an initial snapshot of the number and percentages of schools each states has identified low performing. It provides an early look at a very diverse set of guidelines.  The data show a wide range of results in terms of the percentage of schools identified as low performing. The overall range is 3% to 99%, with individual states spread out fairly evenly in between. Eight states identified over 40% of their public schools as low performing, eleven states 20%–40%, fifteen states 11%–19%, and thirteen states 3%–10%. Even with the limitations of the data listed above, this data suggests inconsistent standards across states.

Stark Renter, D., Tanner, K., Braun, M. (2019). The Number of Low-Performing Schools by State in Three Categories (CSI, TSI, and ATSI), School Year 2018-19. A Report of the Center on Education Policy

A Meta-Analysis of Direct Instruction

A soon to be published meta-analysis of Direct Instruction (DI) curricula that reviews research on DI curricula between 1966-2016 reports that DI curricula produced moderate to large effect sizes across the curriculum areas reading, math, language, and spelling.  The review is notable because it reviews a much larger body of DI research than has occurred in the past and covers a wide range of experimental designs (from single subject to randomized trials).  328 studies were reviewed and almost 4,000 effects were considered.  Given the variability in research designs and the breadth of the effects considered, it suggests that DI curricula produce robust results.  There was very little decline during maintenance phases of the study and greater exposure to the curricula resulted in greater effects.

Stockard, J., Wood, T. W., Coughlin, C. & Khoury, C. R. (in press), Review of Educational Research.  DOI: 10.3102/0034654317751919

 

Examining reproducibility in psychology: A hybrid method for combining a statistically significant original study and a replication

The unrealistically high rate of positive results within psychology has increased the attention to replication research. However, researchers who conduct a replication and want to statistically combine the results of their replication with a statistically significant original study encounter problems when using traditional meta-analysis techniques. The original study’s effect size is most probably overestimated because it is statistically significant, and this bias is not taken into consideration in traditional meta-analysis. We have developed a hybrid method that does take the statistical significance of an original study into account and enables (a) accurate effect size estimation, (b) estimation of a confidence interval, and (c) testing of the null hypothesis of no effect. We analytically approximate the performance of the hybrid method and describe its statistical properties. By applying the hybrid method to data from the Reproducibility Project: Psychology (Open Science Collaboration, 2015), we demonstrate that the conclusions based on the hybrid method are often in line with those of the replication, suggesting that many published psychological studies have smaller effect sizes than those reported in the original study, and that some effects may even be absent. We offer hands-on guidelines for how to statistically combine an original study and replication, and have developed a Web-based application (https://rvanaert.shinyapps.io/hybrid) for applying the hybrid method.

van Aert, R. C. M., & van Assen, M. A. L. M. (2018). Examining reproducibility in psychology: A hybrid method for combining a statistically significant original study and a replication. Behavior Research Methods, 50(4),1515–1539.

Randomized Trials and Quasi-Experiments in Education Research
This paper examines the benefits and challenges inherent in using randomized clinical trials and quasi-experimental designs in the field of education research.
Angrist, J. D. (2003). Randomized trials and quasi-experiments in education research. NBER Reporter Online, (Summer 2003), 11-14.
The Right To Effective Education
The purpose of this Web Site is to disseminate information about behavioral fluency; and to connect people interested in building fluent behavior of all kinds and for all types of people.
Barrett, B. H., Beck, R., Binder, C., Cook, D. A., Engelmann, S., Greer, R. D., ... & Watkins, C. L. (1991). The right to effective education. The Behavior Analyst, 14(1), 79.
Assessing The Value-Added Effects Of Literacy Collaborative Professional Development On Student Learning
This is a 4-year longitudinal study of the effects of Literacy Collaborative (LC), a school-wide reform model that relies on coaching of teachers for improving student literacy learning.
Biancarosa, G., Bryk, A. S., & Dexter, E. R. (2010). Assessing the value-added effects of literacy collaborative professional development on student learning. The elementary school journal, 111(1), 7-34.
Distinguishing Science and Pseudoscience
This paper works as a primer to help discern real science from the significant numbers of reports of false science and pseudoscience that we are daily bombarded with from the media as well as published works that purport to be scientific.
Coker, R. (2001). Distinguishing science and pseudoscience. Retrieved September, 10, 2009.
Fallacy Files
This is a collection and examination of logical fallacies.
Curtis, G. N. (2012). Fallacy files. URL http://www. fallacyfiles. org.
Stephen's guide to the logical fallacies
This paper examines common logical fallacies
Downes, S. (1995). Stephen’s guide to the logical fallacies. Electronic document.
Can Randomized Trials Answer the Question of What Works?
This article discusses the use of randomized controlled trials as required by the Department of Education in evaluating the effectiveness of educational practices.
EDUC, A. R. O. (2005). Can randomized trials answer the question of what works?.
Scientific Research and Evidence-Based Practice, 2003
This paper examines the issue of evidence-based practices for use in education. Evidence-based education (EBE) is examined in the context of evidence-based practice in the field of medicine on which EBE is based. The medical model is compared with EBE with an emphasis on how to develop EBE products and services.
Hood, P. D. (2003). Scientific research and evidence-based practice. San Francisco: WestEd.
Why Most Published Research Findings Are False
This essay discusses issues and concerns that too many research findings may be false. The paper examines reasons a study may prove inaccurate including: the study power and bias, the number of other studies on the same question, and the ratio of true to no relationships. Finally, it considers the implications these problems create for conducting and interpreting research.
Ioannidis, J. P. (2005). Why most published research findings are false. PLoS medicine, 2(8), e124.
The National Center for Special Education Research (NCSER) in IES
This powerpoint presentation provides an overview of the National Center for Education Research (NCSER)
Kame’enui, E. and Gonzalez, P. (2006)
Single-Case Designs for Educational Research
This paper examines the benefits and challenges inherent in using of randomized clinical trials and quasi-experimental designs in the field of education research.
Kazdin, A. E. (2011). Single-case research designs: Methods for clinical and applied settings . Oxford University Press.
A Policymaker's Primer on Education Research
The goal of this Policymaker’s Primer on Education Research is to help policymakers and other interested individuals answer three big questions: (1) What does the research say? (2) Is the research trustworthy? (3) How can the research be used to guide policy?
Lauer, P. A. (2004, February). A Policymaker's Primer on Education Research: How to Understand, Evaluate and Use it. ECS.
What Is a Culture of Evidence? How Do You Get One? And... Should You Want One?
This paper uses a framework derived from Cultural Historical Activity Theory to describe changes in organizational practice in two teacher education programs as they began to use new sources of outcome data to make decisions about program design, curriculum and instruction.
Peck, C. A., & McDonald, M. A. What Is a Culture of Evidence? How Do You Get One? And... Should You Want One?. Teachers College Record. Date accessed: 3/21/14 http://www.tcrecord.org/Content.asp?contentid=17359
Experimental and Quasi-Experimental Designs for Generalized Causal Inference
This book is a valuable resource for graduate students and applied researchers who are interested designing experimental studies as well as for those needing to interpret both experimental and quasi-experimental research in the social and behavioral sciences.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Wadsworth Cengage Learning.
Baloney Detection Kit
“The Demon-Haunted World”, Carl Sagan provides tools for skeptical thinking. This excellent list is a strong tool to weed out the bad seeds in science.
Shermer, M., & Linse, P. (2001). The Baloney Detection Kit. Skeptic Society.
Pasteur’s Quadrant as the Bridge Linking Rigor with Relevance
The authors propose educational design research and communities of practice as frameworks through which to realize the promise of Pasteur's quadrant.
Smith, G. J., Schmidt, M. M., Edelen-Smith, P. J., & Cook, B. G. (2013). Pasteur's Quadrant as the Bridge Linking Rigor With Relevance. Exceptional Children, 79(2), 147-161.
Can Traditional Public Schools Replicate Successful Charter Models? A Different Take
This Op Ed piece from Daniel Willingham examines the study, Injecting Charter School Best Practices into Traditional Public Schools: Evidence from Field Experiments. Willingham makes a number of interesting points including: the need to disseminate the results of studies that failed to produce significant effects and the importance of understanding what and how it the study failed.
Willingham, D. (2014). Can Traditional Public Schools Replicate Successful Charter Models? A Different Take. Real Clear Education.
Translating Evidence into Efficacy: Evaluating Strengths and Weaknesses of Different Study Designs
This is a critical examination of the strength and weaknesses of research designs.
Wong, N. (2006). Translating Evidence into Efficacy: Evaluating Strengths and Weaknesses of Different Study Designs. Retrieved July 17, 2014.
TITLE
SYNOPSIS
Australian Society for Evidence Based Teaching

This web site provides evidence-based resources for free to teachers, principals, and parents.

Best Evidence Encyclopedia

The Best Evidence Encyclopedia is a free web site created by the Johns Hopkins University School of Education's Center for Data-Driven Reform in Education (CDDRE) under funding from the Institute of Education Sciences, U.S. Department of Education. It is intended to give educators and researchers fair and useful information about the strength of the evidence supporting a variety of programs available for students in grades K-12.

Cambridge Center for Behavioral Studies
The mission of the organization is to advance the scientific study of behavior and its humane application to the solution of practical problems in the home, school, community, and the workplace
Campbell Collaboration (C2)

The organization promotes well-informed decision making by preparing, maintaining and disseminating systematic reviews in education, crime and justice, social welfare and international development.

Center for Research and Reform in Education (CRRE)
CRRE is a research center who’s major goal is to improve the quality of education through high-quality research and evaluation studies and the dissemination of evidence-based research.
Cochrane Collaboration
Cochrane is an independent network of health practitioners, researchers, patient advocates and others, responding to the challenge of making the vast amounts of evidence generated through research useful for informing decisions about health.
Current Controlled Trials - Medicine
This is an example from medicine of dissemination of evidence-based practices.
Daniel Willingham - Web Site
Daniel Willingham is a resource to help those interested in issues of education to find practical, helpful information on what works and what doesn’t. His videos are of special interest.
Data Quality Campaign
This nonprofit organization promotes the systematic and outcome driven use of data at all levels of education
Institute of Education Sciences
IES is the statistics, research, and evaluation arm of the U.S. Department of Education. Its mission is to provide scientific evidence on which to ground education practice and policy and to share this information in formats that are useful and accessible to educators, parents, policymakers, researchers, and the public.
Journal of Contemporary Clinical Trials
Contemporary Clinical Trials is an international journal that publishes manuscripts pertaining to the design, methods and operational aspects of clinical trials.
Logical Positivism
An overview of Logical Positivism and it’s impact on science and the issue of verifiability.
National Education Policy Center
The mission of the National Education Policy Center is to produce and disseminate high-quality, peer-reviewed research to inform education policy discussions.
National Institute of child Health and Human Development (NICHD)
The NICHD conducts and supports research, that explore health processes; examines the impact of disabilities, diseases, and variations on the lives of individuals.
National Science Foundation
NSF is a federal agency created to promote the progress of science; to advance the national health, prosperity, and welfare; to secure the national defense.
New Evidence-based Web Site for ESSA

The Center for Research and Reform in Education at Johns Hopkins University has announced a February release for a website that reviews every math and reading program for grades K to 12 to determine which meet the strong, moderate, or promising levels of evidence defined in the Every Student Succeeds Act (ESSA). This web site is designed to provide education decision-makers at the state, district and school levels, teachers, parents, and the public with the information to ascertain which programs meet the ESSA evidence standards.

 

Positive Behavioral Interventions and Supports (PBIS)

The Technical Assistance Center on PBIS provides support states, districts and schools to establish, scale-up and sustain the PBIS framework.

Society for Prevention Research
This organization is dedicated to advancing scientific investigation on the etiology and prevention of social, physical and mental health, and academic problems and the translation of that information to promote health and the well being of the public.
Spurious Correlations
An important rule of research is; correlation does not equal causation. Just because two events track each other over time does not mean that one caused the other. This web site mines data and uses to humor to make the point that for such correlations are often “Spurious Correlations”.
What Works Clearinghouse (WWC)

The goal of the WWC is a resource for informed education decision-making. The WWC identifies evidence-based practice, program, or policy, and disseminates summary information on the WWC website.

Back to Top