Yesterday, I had the pleasure and privilege of moderating a stimulating discussion, hosted by the USAID-funded AIDSTAR-Two project entitled, “Organizational Capacity Development: How Do We Measure Impact?” The thoughtful and brilliant panelists Sara Bennett, Associate Professor in the Health Systems Program at Johns Hopkins Bloomberg School of Public Health; Eric Sarriot, founder and director of the Center for Design and Research in Sustainable Health and Human Development at ICF International; and Kenneth Sklaw, Team Lead for the Capacity Building Team in the Office of HIV/AIDS within USAID validated and challenged my thinking on ways to measure the results of organizational capacity development—which is core to the work that the LMG Project does around the world.
These were my take-aways—the “Seven Cs” for measuring results of organizational capacity development:
- Organizational capacity development is COMPLEX and dynamic; measuring its results must take this into consideration. Organizational health, like a person’s health, is influenced by the system in which it functions and is more than just the sum of the individuals that make up the organization. So, when we measure organizational capacity development results, we have to remember that these may change over time as the organization and the organization’s needs evolve or change over time. One set of measures used at the baseline may no longer be relevant a year or two down the line.
- Organizational development is CONTEXTUAL: organizational development results for a small community based organization maybe different from those for a larger civil society organization, district health team, university or network or federation of organizations.
- The results being measured must be COMPREHENSIVE: results ought to look beyond output measures and focus on the performance of the organization. One-time assessments are not useful; results need to be measured over time. Projects need to think about what happens if the change occurs after the life of the project.
- The process of deciding what to measure, when, and how often, and for how long must be COLLABORATIVE: evaluators are but one player in the mix. Evaluators must also reflect on whether the framework they are using to measure organizational capacity helps the organization to meet its own capacity development goals.
- CAUSALITY is hard to prove. We should be looking at plausible associations rather than attribution of results to organizational capacity development efforts.
- The evidence base for capacity building needs to be along a CONTINUUM: Quantitative indicators must be combined with (perhaps) case studies of success and failure so we can understand not only what has changed, but how and why the change has occurred. Using mixed-methods creatively – like realist evaluation – may be the next frontier.
- And most importantly, the organization whose capacity is being developed must be CENTRAL to measurement and evaluation efforts. We must remember to measure what the organization wants to see change, in its own capacity, over time.