r. Reshma Trasi, Monitoring, Evaluation, and Research Team Director, LMG
At the recent Third Governance for Health Roundtable, I listened to what over 50 thought leaders had to say about measuring governance and generating evidence that it matters and the challenges and opportunities around generating evidence. Here are the takeaways from the rich discussions:
-
The inherent tensions around generating and using evidence need to be acknowledged
Donors and development partners have equal interest in seeing rigorous evidence of the longer-term impact of interventions and generating ‘quick wins’—two diametrically opposing objectives—showing that governance produces positive health outcomes. Rigor (that is, attributing health outcomes to governance interventions) requires resources and time. Civil society watchdog groups are, relatively, less concerned about rigorous evidence, but want to use real-time data for action, decision making, and holding the public and private sectors accountable. Public sector officials may be less interested in being monitored or assessed. These tensions must be acknowledged when defining an evidence agenda around governance. During discussions around the use of evidence, participants challenged the notion that policy decisions are made based on evidence. There was agreement that governments make policy decisions based on short-term political agendas rather than evidence. As a result, governments are not investing in generating evidence to inform policy decisions.
-
Generating evidence around governance interventions is complex
Among the experts present in the room, there was agreement that terms like ‘safe deliveries’ are better understood than ‘good governance’. Governance as a concept means different things to different people. Some participants talked about good governance as a set of values, while others described it as a set of rules. Some talked about it as formal and informal structures, while others talked about it as a set of norms for a system. Some talked about it as a process, while others felt that good governance was defined by its outcomes. Essentially, ‘governance’ is everything! Given this diversity in understanding and definition, it is not surprising that the interventions addressing governance are diverse and cross-cutting, and often located outside the health sector (e.g., procurement transparency in the public sector, expenditure tracking, etc.). All this variation makes measuring and evaluating governance interventions very challenging, especially when there are limited resources available.
-
Don’t underestimate the utility value of performance monitoring
In the absence of significant investments to rigorously demonstrate that governance interventions produce health system performance improvements, we talked about the ‘lower-cost option’ of using project or program data to demonstrate the effects of governance interventions. We discussed the importance of defining, at the start of the intervention, a program theory of change—articulating what we think will change, when, and how—and designing performance metrics or indicators around this. A well-articulated, coherent, measurable, realistic theory of change (or conceptual framework that is supported by theory or practice) provides a good foundation on which to build an M&E plan from the beginning. Implementers and activists in the room talked about the importance of using performance data in programmatic feedback loops to define action and inform decisions.
-
Strategic evaluations can help move the field forward
Participants called on development partners to fund ex-post evaluations—evaluations conducted after the project or program has ended. Ex-post evaluations would help unearth what worked and why, and what did not and why not. The experts felt that this is a significant gap, and it’s the area that provides the most opportunities to move the field forward.
The experts in the room were unanimous in agreeing that context matters. In generating evidence then, perhaps, we should be looking at methods that integrate context rather than isolate it. Governance interventions—unless there is a clear, proximal, cause-effect relationship—will benefit from mixed methods assessments, case studies, and newer methods like contribution analysis and realist review that are being used to examine complex interventions. Colleagues felt that understanding what works, why, and in what context, provides an essential grounding to the field of governance interventions in health.