Evaluation challenges and innovative evaluation methods to improve the governance of health systems in Western Africa

Evaluation is a critical element in improving the governance of health systems and guiding the design, implementation and scaling-up of appropriate health interventions to progress towards universal health coverage. However, in many Western African countries, evaluation capacity is still limited, stakeholders have poor knowledge of different concepts related to monitoring and evaluation, evaluations are often commissioned by donors without national ownership, and therefore evaluation reports are rarely utilised to support decision-making. Moreover, evaluations face important financial, operational and methodological challenges, notably due to the complexity of health interventions, absence of “true” control areas and lack (or poor quality) of data. Health programmes are usually multifaceted and spread over a long period of time, and many outcomes change gradually, hence evaluation beyond the time line of the program is necessary. Faced with these challenges, this session intends to present a number of innovative methods that are adapted to support the development of an evaluation culture in Western Africa. The introduction sets the scene of the evaluation practice in Western Africa and points out that most of the time, health programmes in these contexts are evaluated as part of donors’ requirement, because development aid effectiveness has been questioned by donors and beneficiaries alike. This communication addresses two issues linked to evaluation: making ethics a dimension of evaluative approaches and the need for a global, integrated framework for evaluating development policies. Then, the methodological challenges of causal impact evaluation are discussed. In absence of randomised experiment, difference-in-differences are used to mimic an experimental research design using observational data to evaluate causal effects of health policies. In this context, it is key to identify comparable control areas or individuals. This presentation presents how geo-referenced data available in the Demographic Health Survey could be used to identify control individuals and match them with programme participants to evaluate a family planning programme in Nigeria. The third presentation deals with the principles of the meta-evaluation methodology. Its objective is to highlight, for a given political-administrative regime, the basic criteria for a quality evaluation. Then, a meta-evaluation research protocol in Burkina Faso is presented, targeted on both the methodological quality of evaluations, and the process quality, including utility, of evaluations.