Syspons GmbH helps organisations evaluate and improve the effectiveness of their projects. The consulting firm recently conducted an impact study of CmiA for the Aid by Trade Foundation. In this interview with Motje Seidler, a senior consultant at Syspons, we discuss her experiences as an evaluator and the outcomes of the investigation.
For international development projects, it is always important to take into account the major power imbalance or dependency relationship between donors and beneficiaries. This can also influence the responses of some interview partners. As an evaluator from Germany who is both white and a foreigner, I need to frequently remind myself of my own position and its potential impact on others.
What was the goal of the CmiA impact study commissioned by the Aid by Trade Foundation, and when was it conducted?
The study’s aim was to investigate the impact of the initiative so far and to what extent it aligns with the expectations embedded within CmiA’s training. In order to further improve the programme, we also assessed which elements of its implementation work well and which do not. We did this by analysing a large number of recent documents and conducting numerous interviews between October 2019 and September 2021. Furthermore, in two case studies in Zambia and Côte d’Ivoire, we asked 466 cotton farmers about their experiences.
In common with many international endeavours undertaken over the past two years, the CmiA impact study also faced constraints as a result of the COVID-19 pandemic. What challenges did you encounter?
Due to travel restrictions, we were unable to begin collecting data in Zambia when we had expected to do so in April 2020, and we had to change our approach at short notice. This caused a delay, but our on-site surveyors ultimately succeeded by working with local experts to collect the data. We discussed every detail of the survey questions and logistics with them, trained local experts via Zoom, and joined them in their work from our desks at home through regular telephone calls.
How are local experts selected, and how does onsite collaboration actually work?
We have developed a network of expert evaluators in various countries who can also refer us to other specialists. When selecting experts, we pay special attention to each individual’s evaluation experience, expertise in the relevant topic, and independence from conflicting interests. Good personal chemistry is also important to us. In Zambia and Côte d’Ivoire, we were fortunate and quickly found two experts whose work was excellent.
What lesson have you learnt from the CmiA impact study that has stuck in your mind?
It became apparent that contextual factors and market structures in partner countries play a central role. CmiA’s training efforts can maximise their impact where a country’s pricing policies and institutions also provide incentives to small-scale farmers to grow cotton using sustainable methods. These contextual factors also need to be considered during the selection process and when working with partner countries. As was the case with CmiA, it is always nice to see clients following an evaluation with great interest, critically examining their own work, and setting in train new learning processes.
The data on improving living conditions for small-scale farming families is drawn from the Syspons impact study (2021). Published under: https://cottonmadeinafrica.org/wp-content/uploads/CmiA-Impact-Study-2021.pdf