Creating evidence is important. Creating usable evidence is even more so. But is it enough to just create high-quality usable evidence? If the evidence we generate is not used, we minimize our ability to achieve our shared mission of improving the lives of children and families.
There is a growing body of research that suggests investment is needed in “evidence-using systems” that facilitate the use of evidence. Such systems, or interventions , to facilitate evidence use span from linear (i.e. traditional dissemination efforts) to relational (i.e. fellowships or research-practice partnerships) and systems (i.e. leadership, infrastructure) approaches.
Kim Dumont recently prompted evidence use advocates to consider three actions necessary to foster research use:
- Consider how the research is conducted, to make it as relevant as possible.
- Incorporate what we already know about the conditions to support use.
- Identify whether we can facilitate evidence use in existing systems.
In one of the first blog posts in this series on evidence use, we discussed some of the research on the organizational conditions that support evidence use. The five key dimensions of ACF’s evidence capacity framework included organizational evidence culture, leadership, evidence infrastructure, human capital and engagement at the individual, interpersonal and organizational level. These align with Dumont ’s six nurturing conditions and a recent systematic review of the organizational conditions to support evidence-informed decision-making.
Knowledge mobilization seems to be most effective when embedded within existing structures and processes and not one-off projects. As one example within child welfare , a study found relationships between the investments in organizational supports for evidence use (defined as ties with external partners who could provide evidence), technical resources for staff to access evidence, staff dedicated to facilitating evidence use, and whether the agency refers to evidence in places such as its mission, values, strategic plans, job descriptions, performance reviews, and professional development materials. Similarly, OPRE works to insert evidence into regular, routine processes of government including budget formulation, regulation or sub-regulation development, funding opportunities, and accountability processes.
There is remarkable consistency of findings on systems approaches to facilitating evidence use across disciplines. These are the same whether you are in a school district, government agency, legislative branch, or community-based non-profit. I believe the conditions to support evidence use come down to human behavior and the behavior of organizational systems. Often our first inclination is to try to change the systems but systems change is very hard. Instead of disrupting systems, what if we identify how things already happen and develop interventions or strategies that move with them, like water flowing around boulders rather than trying to move the boulder out of the way.
What would it look like to facilitate evidence use in a system?
A wide range of strategies are being implemented by organizations and though some appear to show promise , few have been rigorously researched . While still early, there are some research teams taking these findings and creating and testing interventions to create the conditions for use:
- One type of intervention being tested is a coordinated knowledge system that aims to build a knowledge mobilization tool into existing routines. In this specific example, a coordinated knowledge system is built into clinical supervision, a consistent routine for mental health counselors, to answer just-in-time questions and needs with relevant evidence. The intervention shows initial promise in a pilot study to improve identification of the causes of the clinical problem, identify evidence-based solutions to address that specific problem, and support on how to implement those solutions. The intervention condition used evidence-based resources with a much greater frequency than the control condition and evidence-based resources were discussed and used in supervision significantly more frequently than in the control condition. The team is in the middle of a larger randomized control trial to further test this promising intervention.
- An example of an intermediary organization model is the Research to Policy Collaborative (RPC). RPC built infrastructure to facilitate collaboration between policy and research communities to improve research use in legislative activities. The intervention identified opening policy windows on topics of greatest relevance to policymakers, provided training for researchers on how to work in a policy context, and then supported matchmaking between researchers and policymakers. The initial randomized trial found the legislative offices who received the intervention valued research more and introduced more legislation using research, and the participating researchers advanced their own understanding of policy to create more relevant research in the future. The model is now being tested across multiple states in another randomized study to attempt to replicate the findings.
- Another model is connecting university researchers and agencies that administer services in a long-term partnership. One example being tested is a Community Academic Partnership model which builds infrastructure to connect university-based researchers and agency leadership. One example, CAPTURE , uses a coordinating committee to support relationships between the university and child welfare agency, support child welfare leaders to identify organizational facilitators to research use, and leverage continuous quality improvement activities to test research-informed strategies to agency identified problems. This project should be sharing findings soon.
- Another intervention model focuses on organizational work routines. This intervention supports school district agency leaders to reimagine core work processes to integrate access to and use of research. The District Leadership Design Lab focuses on redesigning school improvement planning, human resources processes, and teacher professional development. This project is beginning to share findings from this work.