Professor Michael Woolcock from the World Bank shared his experiences of reporting and communicating results from development programmes at a well-attended seminar at Sida.
Photo: Kerstin Becker/Sida
Results discussion at a high level
"Sida should be a creative laboratory with an open and tolerant atmosphere where you can try different methods and have a long-term perspective. There must be room for creative and unexpected initiatives, even if it means taking risks, was one of Woolcock’s comments."
The issue of results in development programmes is indeed a hot topic among politicians and employees in the field of development cooperation. Which results can be expected to come out of various initiatives, what interventions are most effective and which methods are most effective for communicating these results, are some of the main questions.
Those questions are quite relevant, but the simple word "results" can mean many different things – this was the conclusion that participants drew during the Development Talk on November 8th, entitled Capturing and Communicating Results of Complex Contributions; Opportunities and Challenges for Sida.
"We need to make clear what we mean when we talk about results: Results of what, for whom and what kinds of results, underlined Mr Lennart Peck, Sida, who opened the seminar."
"Are we talking about results for an individual in the host country? For groups such as women, poor or youth? Or perhaps for a village, the entire country or the region?"
Another aspect to consider is the timing of the assessment. Results are not static, but grow over time, continued Mr Peck, addressing more than 80 focused and diligently note-taking listeners.
The strength of the relationship in the recipient country is also important. Has the effort to be scrutinized been both necessary and sufficient to cause the change? Or necessary, but not sufficient – meaning that other donors also participated? Or neither necessary nor sufficient, but it has contributed to making the change possible?
"The aim of the Swedish development cooperation is to help create opportunities for poor people to improve their lives. And it is in this perspective that we should interpret the results, concluded Mr. Peck."
Then it was time for Dr. Michael Woolcock, a specialist in social development who works at the World Bank’s Research Department. He is frequently lecturing around the world and began his speech by praising Sida.
"You've come a long way in your discussions about the results of development cooperation. I am often asked to give lectures about this, but rarely on this level!"
Politicians in many countries ask for fast and “easy to interpret” performance reports on various development cooperation programmes efforts, continued Dr. Woolcock. But this demand is likely to steer many initiatives towards directions that are neither effective nor meaningful.
Development cooperation is, said Dr. Woolcock, like a "huge machine with many knobs in an unpredictable environment." What wheels you should start turning, and how hard you should push in order to achieve well-functioning states with happy people is a question with no right answer.
"Aid workers are not technicians that apply known solutions. We work in a complex and unpredictable environment, and this must be accepted as a true condition for our job."
Decisionmakers often have unrealistic notions about how quickly social change can come about, continued the professor. For example, how long it takes to establish the rule of law in countries previously governed by other, less democratic principles.
What does history teach us? That it is a fast process? Not at all, stressed Woolcock, and quoted the former British Prime Minister, Mr. Gordon Brown, who said: "the most difficult period when you are building the rule of law is the first five hundred years."
"Should we then give up after five, ten or fifteen years just because there are no clear and measurable results? Absolutely not! But when we are engaged in such long processes, we must learn to distinguish between changes that are important for achieving a positive outcome and that we therefore need to support, and changes that can damage a positive development and that we should try to minimise or counteract."
Also Dr. Woolcock stressed the importance of timing when assessing results.
- When dealing with issues such as democracy and gender equality, the situation can get worse before it gets better, because the work arouses opposition among established power groups. Other efforts might show impressive results in the short term, but leave no lasting trace. If we consider results at a wrong time, we risk drawing wrong conclusions.
Woolcock also warned against exaggerated reliance on generalisations, and gave an example of a programme in Indonesia that he had been involved in.
"The programme worked in Indonesia, but that does not mean that it would also work in Afghanistan, for example! In fact, we had to make local changes also in Indonesia."
Even if easily comprehensible results cannot be produced, this does not mean that development programmes should be excluded from evaluation, underlined Dr. Woolcock and noted that the trend to demand rapid and quantifiable results is becoming more widespread.
But, he said, it is often possible to create "pockets" in the business with a more nuanced, serious and long-term view of performance. He emphasised his own workplace, the World Bank, as a good example of such an environment. His advice to Sida was to build on a culture of learning in all development programmes, in order to make it possible to make relevant changes when needed.
The seminar programme also included specific examples of Sida's work with performance in complex environments, namely result-based management (RBM) in the Western Balkans and a joint evaluation of development support to Sri Lanka, Vietnam and Laos that Sida has been providing for nearly 50 years.
Charlotte Örnemark from Nordic Consulting Group who worked with RBM in the Balkans warned that this management style often becomes a form of “expert control”.
"We must give the control over the work back to those working in the field. Result based management should be a method that facilitates the daily management, not a system of control from above, she emphasised."
The seminar was organised by Sida’s unit for evaluation and monitoring as a part of the agency's work to become even better at assessing and analysing results.