On 6-8 July, I attended a conference at the World Agroforestry Centre (ICRAF) on Impacts of International Agricultural Research: Rigorous Evidence for Policy organized jointly by the CGIAR Independent Science and Partnership Council (ISPC) Standing Panel on Impact Assessment (SPIA) and the CGIAR Research Program on Policies, Institutions, and Markets (PIM). I welcomed the delegates on behalf of ICRAF and ILRI, the two CGIAR Centres headquartered in Nairobi. I took the opportunity to give some personal reflections on impact assessment in CGIAR. This is a modified version of my address (the presentations from the conference are available here).

‘We in the CGIAR have committed to tackling some of the greatest challenges that the human population has ever faced.

How do we feed a growing population not only with calories but with nutrients from a balanced diet that are essential for good health, in the face of climate change? We know that protein, vitamins, and minerals are essential for not only growth in children but also for intellectual development, and cognitive and learning ability, and malnutrition can permanently cause not only physical stunting but also have long-term effects on a nation’s economic development.

Agriculture will form, and already forms, the basis of economic development in many part of Africa, and it is the route by which millions of people will escape poverty, not just by improving the livelihoods of poor farmers but also by commercialization of smallholder agriculture creating employment in input services and in value addition along the value chain.

‘And while agriculture is often seen as a destroyer of the environment, in fact it holds the key to effective natural resource management and the provision of essential environmental services, whether carbon sequestration by well managed rangelands or by trees that reduce greenhouse gas (GHG) emissions intensity, and in some cases absolute emissions through productivity increases and more efficient use of inputs.

‘Now, while we might recognize the importance of agricultural research, it is important that we can persuade the rest of the world of the case, as agricultural research can only deliver these benefits if there is investment in it. A few decades ago agriculture used to receive about 15% of official development assistance (ODA). Today it receives only 4% and livestock receives only 4% of total agriculture ODA, despite accounting for 40% of agricultural GDP in developing countries.

‘Many studies have shown the very high rates of return to agricultural research but we need more specific evidence on what investment in agricultural research actually delivers. So at a time when many donors are under pressure from political changes in spending priorities whether, for example, from domestic issues or the refugee crisis, it is timely to consider the role of impact assessment in CGIAR.

Agricultural research deals with complex systems – agriculture is a complex socio-ecological system with complex problems. We use complex research methodologies to solve these problems.

A major challenge in impact assessment is having robust methods to generate robust evidence. We have methods to assess rates of uptake of a technology or the welfare benefit of that technology but not all research is focused on a single technology. How do we assess the impact of research that delivers a mix of new technologies which may be adopted and adapted in different ways by different farmers? For example, in the livestock sector delivering improved genetics will not have much impact if it is not accompanied by better feeding and animal health services, coupled with new organizational and institutional marketing arrangements which in turn will only be effective if there is a conducive policy environment. In such cases how do we know what impact our research might have?

Some CGIAR research is focused on influencing and supporting policy or investment decisions where the effects of research on a complex political process may be difficult to assess. Twelve years ago I was at a workshop on the interface between research and policy organized by the chief scientist at the Scottish Government Rural Affairs Department who at that time was Maggie Gill, now the chair of the CGIAR ISPC. One presenter showed a list of things a minister has to consider when devising a new policy. The technical or scientific evidence was only one of 23 things on that list. How do we know what impact our research is having among the other 22 influences?

The point I want to make is that as we consider impact assessment in CGIAR let us also continually ask ourselves how we deal with complex questions about impact as there is a risk that we focus only on the things that are easy to measure.

To tackle the global challenges that CGIAR is researching, we will need transformational change in agriculture, not incremental change. There is a risk that if we focus only on the things that are easily measured we will not achieve those transformational changes. Do we have the tools and methods that will allow us to measure impacts of complex solutions to complex problems? I have my doubts about that and believe we need to see more methodological development in quantitative and qualitative impact assessment and that there is much to learn from other sectors including public health and education.

So when we consider impact assessment, let us look at what we have achieved, but also look forward to how we will demonstrate our achievements in the future.’