Definition

Implementation

is that point where an intervention or practice moves from the domain of the researcher to that of the practitioner.

Executive Summary

The primary goal of educational research is the identification of practices for schools to use to teach skills and knowledge that enable students to live productive and successful lives. Implementation is the point where the practice moves from the domain of the researcher to that of the practitioner. Successful implementation is hard but not impossible. Exemplars on how to overcome the challenges exist and are valuable in building our knowledge base in this critical component of our model. This article describes the fundamental elements that comprise implementation.

Phase One-Adoption
Getting a practice approved by decision-makers for use by practitioners

Phase Two-Sustainability
Maintaining a practice beyond the implementation phase

Implementation Paradox
Why do bad things happen to good interventions

1. Controlling the effects of increased numbers of complicating variables that arise in the natural environment
2. Influencing multiple levels of the institution of education
3. Managing unforeseen consequences resulting from the introduction of the intervention
4. Insufficient availability of rigorous evidence based research
more

1. Outcomes
2. Motivation
3. Obstacles to Adoption
4. Tools for Adoption
   more

1. Outcomes
2. Performance Management Strategies
3. Obstacles to Sustainability
4. Tools for Sustainability
   more

The primary goal of educational research is the identification of practices for schools to use to teach skills and knowledge that enable students to live productive and successful lives. Implementation is the point where the practice moves from the domain of the researcher to that of the practitioner.

Unfortunately, practices that have been identified as both efficacious and effective frequently fail to survive this transition and consequently never become institutionalized. The barriers and obstacles that must be surmounted are formidable. It is a challenge to create lasting change within one school, and it becomes increasingly difficult to affect changes across schools (districts, regions, and states).

Successful implementation is hard but not impossible. Exemplars on how to overcome the challenges exist and are valuable in building our knowledge base in this critical component of our model. This article describes the fundamental elements that comprise implementation.

The Two Phases of Implementation

Phase One-Adoption
Getting a practice approved by decision-makers for use by practitioners

The key to successful implementation is the formal authorization of decision-makers. Achieving this requires support from key stakeholders: practitioners and consumers (parents and students). Knowing what motivates each constituency is critical in understanding how to gain this support. Effective use of social influence techniques is essential in persuading stakeholders to commit to putting into practice the new intervention (i.e., practice).

Phase Two-Sustainability
Maintaining a practice beyond the implementation phase

To make an impact an intervention must meet the test of time. Administrators, teaching staff, and parents all play critical roles in determining whether an intervention will survive. Educators are all too familiar with initiatives coming and going. It is important to note that this is true regardless of the efficacy or effectiveness of the intervention. To ensure durability it is imperative that there are built-in motivations that support the practitioner s implementation of the new intervention. To accomplish this it is vital to start with a receptive audience. Achieving this requires matching the practice to the culture that will implement the intervention. Interventions often fail with a group that is already hostile to it. Finally, sound performance management techniques must be employed to sustain the intervention once it is put into service.

Implementation failures take the form of two paradoxical situations:

1. Practices established as efficacious and effective frequently fail to be adopted or fail to win wide support within the educational community.
2. Practices that are ineffective and even harmful are often adopted and retained for years despite failing to produce desired results.

(Research -> Practice)

Why do implementations fail? What impedes efficacious, effective interventions from being adopted and sustained?

• Controlling the effects of increased numbers of complicating variables that arise in the natural environment
  Research settings are specifically designed to control for variables that may compromise the integrity of the results. In the laboratory, the researcher can increase confidence that the intervention is the cause of the change to the dependent variable by minimizing the threats to validity of the results. In classroom settings, educators are confronted with multiple unplanned factors that distract both student and instructor. Unfortunately, these factors frequently reduce the effectiveness of the practice. An apt comparison would be trying to thread a needle while riding a roller coaster.
  
  Common complicating factors include:
  
  • Turnover in service practitioners
  • Low acceptance of the intervention by the practitioners
  • Turnover in students
  • Changes in resources or lack of required resources
  • Changes in monitoring standards
  • Consumer dissatisfaction
• Influencing multiple levels of the institution of education
  The introduction of a new intervention requires successful navigation of the complex political world of policy-makers, decision-makers, administrators, service providers, and consumers. Each stakeholder level has its own unique set of contingencies that must be addressed.
  
• Managing unforeseen consequences resulting from the introduction of the intervention
  Every attempt to change a complex system has unplanned results. Good planning is the best tool for predicting possible consequences but does not eliminate unplanned consequences. Planning is also critical in minimizing for the effects of negative consequences when and if they occur.
  
  The consequences fall into two categories:
  
  Unintended consequence - those consequences that can reasonably be predicted, but are ignored and/or unanalyzed.
  
  
  Example: California mandates a fifth year for prospective teachers to remain in school before earning a Teaching Credential. The intent of the law was the improvement of teacher quality. Unfortunately, the unintended consequence has been to increase the number of unqualified teachers in California classrooms.
  
  • Fewer teacher applicants enter the field due to the duration of schooling compared to compensation
  • Reduction in out of state veteran teacher applicants due to the requirements for a 5th year of schooling
  
  
  Unanticipated consequences - those consequences, which are unpredictable due to our inability to process the vast number of possible outcomes that may arise from an intervention.
  
  Example: No Child Left Behind (NCLB) itself is the model of best intentions resulting in unanticipated consequences. NCLB is over 650 pages, applies to 50 states, and is designed to be a comprehensive school improvement intervention. It covers topics as far ranging as reading scores, teacher qualifications, staff development, high stakes testing, compensation for performance, school report cards, school safety, and the establishment of evidence-based standards for education. The unprecedented scope of NCLB guarantees that no one could plan for and anticipate all possible consequences that would result from the implementation of this law. Some examples of unanticipated consequences are:
  
  • Closure of poorly performing schools with few alterative choices for transfer
  • Few evidenced-based school improvement strategies for poor performing schools
  • Under funding of NCLB crippling reform efforts
  • Formidable resistance from teachers to high stakes testing
  • Poor performing schools losing money and resources required for improvements assuring further slippage
  • Demanding that disabled students reach "proficiency" on standardized tests sets students up for failure and conflicts with IDEA
  
  
• Insufficient availability of rigorous evidence based research.
  
  • Insufficient rigorous research being conducted
  • Inadequate dissemination

• Performance outcomes - a direct measurement of the worthy and intended purpose of an intervention.
  Those outcomes that impact directly on improved student performance. Example: graduation rates
• Process outcomes - a measurement of a mission critical process that leads toward a performance outcome. Those outcomes that lead to improved student performance.
  Example: Percent of teachers fully qualified
  
  Other examples of process outcomes:
• Laws, policies, and regulations that support an evidence-based intervention model
• Established network of evidence-based stakeholders
• Common criteria for selecting practices adopted by researchers, politicians, practitioners, and consumers
• Research on efficacious and effective methods for getting interventions adopted
• Clearinghouse(s) to disseminate evidenced-based research easily accessible by decision-makers and consumers
• Widely available training in best methods for achieving adoption

• Competing practices
  When a well-established practice exists that will be replaced by a new intervention, opposition will inevitably arise to the new intervention.
• Policy makers select practices for reasons that are not necessarily evidence based
  It is not enough to be right. Policy-makers are constantly being lobbied to adopt practices based on innumerable motives, with evidence most often at the bottom of the list:
• Being re-elected (quick fixes)
• Lobbyist influence
• Avoidance of controversy
• Fiscal constraints
• Ideological or ethical conflicts
• Conflicting laws, regulations, and/or policies
  Until such conflicts are resolved it becomes difficult to get new interventions adopted. Example: No Child Left Behind (NCLB) conflicts with IDEA, the Federal Law governing special education services. In this instance NCLB mandates high stakes testing as the measure of student success; whereas, IDEA mandates that success for special education students be individualized.
• Insufficient preparation of stakeholders
  A disorganized campaign by stakeholders will often result in failure. A well-organized plan that coordinates the efforts of all stakeholders increases efficiency and avoids blunders.
• Inadequate or insufficient evidence
  The lack of evidence when presenting to decision-makers is a serious obstacle. In addition there currently exists serious flaws in the dissemination of evidence to all stakeholders.
• Fiscal constraints
  Change always comes at a price. The cost of any intervention is assessed in relationship to the cost of implementation. If the resources are not available to the decision-maker the practice is unlikely to be adopted.
• Values
  Each educational system (classrooms, schools, districts, states) has its established values (formal or informal) which can support or hinder the adoption of an intervention.

• Those who are responsible for the implementation of the new intervention must (when necessary) acquire the active support of all levels concerned with the results of the practice
• Policymakers (federal and state)
• School Boards (state and local)
• School Districts (Administrators)
• School principals and/or program managers
• Service providers (DIS, health professionals)
• Classroom staff (teacher, paraprofessional)
• Parents
• Stakeholders agree on specific and objective outcomes. These outcomes identify what the stakeholders hope to achieve and act as the yardstick for determining the success or failure of the intervention.
• There is a reporting system in use to assess progress.
• Stakeholders contribute to development of a written plan with goals, responsibilities, and timelines. The goals and objectives include specific strategies for overcoming obstacles that specify goals, resources, personnel responsible for specific performance, and timelines.
• Contingencies are aligned (in accordance with the contingency analysis) in support of individual staff performance that is directly connected to the intervention outcomes.
• Personnel required to implement the intervention are identified (may be existing staff).
• Training designed to insure fidelity to the model is implemented.

• Measurable progress is initially achieved and maintained over time. Stakeholders are informed on a regular schedule of the results.
• An efficacious and effective solution is available.
• The intervention continues to be cost effective.
• There are clearly defined goals and objectives for stakeholders.
• The new intervention cannot result in greater effort than the original practice without serious consideration being paid to addressing the imbalance that this creates.
• Advocates can reasonably achieve stakeholder support.
• Politicians
• Administrators
• Practitioners
• Consumers

• Common vision: a mission and purpose.
• Common language: establish a vocabulary for describing the vision, actions, operations, and feedback.
• Common goals: develop specific goals for all stakeholders identifying what they specifically need to accomplish for the practice to succeed.
• Common practices: defined actions, routines, procedures, or operations practiced by members of the organization.
• Initial and ongoing training
• Data management systems that provide timely feedback to all stakeholders on performances against outcomes
• Leadership team
• Policy and practices manual
• Common knowledge: keep people informed and up-to-date on the achievements as well as setbacks.

• Cost exceeds benefits
  Some solutions to problems work, but the cost are perceived as too great for the results that are achieved. Example: Providing an individual tutor for each student may improve test scores, but the cost far exceeds what society is willing to pay for the results.
• Non-relevant outcomes
  If the stakeholders and, in particular, the service providers do not see the relevancy of the change, it is very unlikely the practice will endure.
• Inadequate planning
  Not enough of the critical areas have been addressed in the planning stage of implementation.
• Ineffective planning
  There are serious flaws in the plan that leads to poor decision-making.
• Ineffective leadership
  It is important that competent leadership is available to communicate the vision, purpose, and outcomes to the staff providing the service. If leadership half-heartedly endorses the intervention, it undermines the practice and results will reflect this fact. Poor results are inevitable if management pursues an autocratic top down approach that does not involve practitioners in the process of implementing a new intervention.
• Turnover
  Bureau of Labor Statistics of the U.S. Department of Labor reported, for the period December 2004 through December 2005, that there was approximately a 30% turnover rate for educational personnel. This results in need for frequent and on-going professional development to maintain an effectively trained work force.
• Insufficient funding
• Funding must be available to meet the minimum resource requirements of a new intervention. It is also important that the funding continues past the initial application period and be available in subsequent years.
• Performance feedback
  Exemplary performance relies on feedback to both maintain current skills and to improve proficiency. If feedback is not frequent and specific, performance will suffer.
• Poor training
  Training of personnel prior to the introduction of a new intervention is imperative. The training must be mandatory. It needs to be specific to criteria, and staff must achieve fluency in the new procedure(s). If the offered training doesn t meet these standards it is likely that the practice will not be sustained.
• Poor fidelity to the intervention
  Programs that are not well implemented will achieve poor results, often leading to their dismissal.