A new paper by the Proteus Initiative
Trench warfare probably best describes the quality of conversation in the results-measurement debate on development. On the one side of the trench line, we find the advocates of ‘hard’ quantative data as the final judge of efficiency, impact and effectiveness. Riding the waves of the neoliberal project, they argue that the social sectors need to get their act together and deliver ‘value for money’ or else. On the other side, we find those who persistently argue that the social realm demands a more qualitative reflective approach to monitoring and evaluation. Referring to systems- and complexity theory, they suggest that methods such as randomized control trails and social return on investment are a waste of time. In this paper titled ‘The Singer, not the Song: The Vexed Questions of Impact Monitoring and Social Change’, Sue Davidoff and Allan Kaplan offer a refreshing account that starts bridging this gap.
Reflecting on a 3-day conversation that they led among 25 development practitioners, they take the reader on a journey into ‘the enigmatic dance between achievement and emergence’. Short term results have to measured. Resources need to be accounted for. Yet, the paper vividly illuminates the potential danger of an overly quantitative mechanistic approach to monitoring and evaluation. Its counterweight: continuous open reflection is at least equally valuable, if conducted rigorously and with discipline, a challenge that is often underestimated. Indeed, in the words of the authors: ‘Impact monitoring, with its procedures and preset indicators, appears, at first glance, rigorous and consequential. The kind of open reflection… too subjective, too arbitrary, too unfocused, too inconsequential. Yet the opposite may be true. Measurement, especially quantitative measurement… is easy, like painting by numbers, compared with the challenging task of reading for what is really going on, in all its complexity and with all its nuances and contradictions, and making sense of this.'
The starring role then, to perform this dance, resides with the development practitioner. This brings us back to the title of the paper, taken from the Rolling Stones: It’s the singer, not the song. It’s the practitioner, not the project. It’s the practice, not the tools, that ultimately makes a difference. If you’re interested in the work of the Proteus Initiative, including some more reflective accounts into the craft of development practice, then have a look at www.proteusinitiative.org