Does a Systematic Decision-Making Process Facilitate Adoption?

Education decision makers have to consider many variables when adopting an intervention.  In addition to evidence of effectiveness, they must consider local context, the capacity of the school to implement the program, resource availability, and stakeholder values.  The complexity of the decision-making makes it likely that without a decision-making framework the decision-making task is so complex it is probable that some decision-makers will rely on processes that are influenced by personal biases rather than a systematic approach.  There are several decision-making frameworks available to guide the process but many have not been empirically evaluated.  Hollands and colleagues (2019) evaluated a cost-utility framework as a tool to guide decisions.  This approach relies on multiple sources of evidence to identify values of the decision-makers, the “experiential evidence” of stakeholders that have implemented similar interventions, the problem the alternative solutions are to solve, and the criteria for evaluating each dimension of a decision.  

In this project, the users evaluated the framework in three phases.  In the first phase principals, assistant principals, teacher leaders, and teachers enrolled in a principal preparation program, were assigned to small groups to implement the first six steps of the decision-making framework.  Although performance on each of the steps ranged considerably, approximately one-third of the groups completed each of the steps in the decision-making framework within the available time.  The authors suggested that factors such as complexity of the decision, alignment of the vision of the group members, and the emergence of a leader to keep the process moving forward, influenced performance on each of the six steps in the framework.

The second phase of the project was to survey participants about the usefulness of the cost-utility decision-making framework.  A large majority of the participants had a positive view of the process and thought it would be valuable to apply in their day-to-day work.  A few participants identified that the process was time consuming and may limit the application of the framework.  

The final phase of the study selected three assistant principals to apply the cost-utility framework in their work in their schools.  Two of the three participants reported that although it was time-consuming, it helped clarify decision options, and the stakeholders to be involved in the decision.  The third participant was not able to reach a decision problem within the available time.  This participant also reported that some decisions were imposed by district administration, subverting the cost-utility decision-making process.  

It seems that this framework has the potential value to guide decision-making in the complex environments of public schools.  The time-consuming feature of the process suggests that educators may need additional coaching and support as they develop competencies in applying the framework. Streamlining the steps in the process will be a significant step toward increasing the usability of the tool.

Citation:
Hollands, F., Pan, Y., & Escueta, M. (2019). What is the potential for applying cost-utility analysis to facilitate evidence-based decision making in schools? Educational Researcher, 48(5), 287-295.