Catalytic Initiative to Save a Million Lives
Improving Evaluation Methods—Holding Programs Accountable
The Catalytic Initiative to Save a Million Lives is an international partnership whose overall mission is to help countries reach their Millennium Development Goals for maternal and child health. International donor agencies work with developing country governments to increase coverage of life-saving interventions proven to help reduce maternal and child mortality.
IIP is a major research partner tasked with not only evaluating programs but developing improved evaluation methods to track changes in child mortality. IIP’s work is ongoing, but has already produced important tools and methods for the Initiative that help it make programming and funding decisions. Moreover, this work has contributed to innovations and methods that are being used more broadly by the evaluation and donor communities.
The Catalytic Initiative commissioned IIP to perform two related activities:
- Independent Prospective Evaluations: to provide information on whether delivery strategies for proven interventions are accelerating increases in coverage and reducing child mortality in focus areas. Full evaluations are being conducted in Burkina Faso, Ethiopia and Malawi. In Burkina Faso and Malawi, the work is funded primarily through the Bill & Melinda Gates Foundation, with WHO and UNICEF leading the implementation effort in collaboration with governments. UNICEF is funding the full evaluation in Ethiopia.
- Rapid Mortality Monitoring (RMM): to develop and test “real-time” methods for tracking under-5 mortality in periods of 1 year or less. Methods will be tested in Ethiopia, Ghana, Malawi, Mali and Niger. These methods will be used as part of the full prospective evaluations where possible. RMM is funded through a grant from the Canadian International Development Agency (CIDA).
IIP collaborates with local research institutions to carry out its in-country evaluation work. This ensures that the concerns of local policymakers are addressed by the evaluation, while also building capacity for ongoing assessments at the national level.
Based on lessons learned from the Accelerated Child Survival and Development Program (ACSD, see article in this issue), IIP researchers set out to make sure that the evaluations collect clear descriptions of what activities are being implemented, how, and at what level of quality. A country’s health needs, as well as its capacity, will factor into which programs are and can be scaled up. For a multi-country evaluation to contribute to future programs, evaluations must clearly define what is being implemented. These guidelines are being used by IIP researchers, governments, their partners, and in-country research staff.
Cost and Cost-Effectiveness Guidelines
IIP developed generic recommendations for the economic component of the larger impact evaluations. The economic analysis has two interrelated objectives:
1. Determine the cost of activities funded by the Catalytic Initiative
2. Estimate the cost and cost-effectiveness of providing the prevention and care services that Catalytic Initiative activities are intended to influence.
At present the economic component is only being implemented in Malawi in collaboration with the Department of Economics at the University of Malawi, but it may be expanded to other countries if needed funding is obtained.
Rapid Mortality Monitoring
In the first two years, IIP has conducted a thorough review of potential methods for obtaining data on under-5 mortality in real time and worked with in-country research partners to develop practical designs. Further refinement and data collection is ongoing.
Global Contributions to the Field
Common Framework for the Scale-Up to MDG 4 & 5
Research and lessons learned from IIP’s work on the Catalytic Initiative have contributed to the development of a common framework for the scale-up to Millennium Development Goals (MDGs) 4 & 5 for maternal and child health.
The framework establishes guiding principles, defines terms to promote understanding and presents an overall evaluation framework linked to achievement of the health-related MDGs. Its overall aim is to guide study designs, measurement approaches and analysis plans, and to provide a set of “core” indicators to be measured at country and district levels. The framework also reflects active collaboration with the GAVI Alliance, IHP+, and Countdown to 2015.
National Evaluation Platform Approach
The proposed evaluation framework was recently described in a Lancet article calling for a new approach to measure the impact of large-scale interventions.
Based in part on its Catalytic Initiative work, IIP lead researchers Cesar Victora, Robert Black, and Jennifer Bryce, with other international experts, developed an evaluation approach based on district-level data on inputs, outputs, outcomes and impact. This new platform approach has several innovative features:
- the district is the unit of design and analysis.
- based on continuous monitoring of the different levels of indicators.
- additional data are gathered before, during, and after the period to be evaluated by various methods.
- a range of analytical techniques are used to deal with data gaps and biases.
- interim and summative evaluation analyses are undertaken.
The platform approach is presented as part of a Lancet series on large-scale evaluations which IIP is leading. IIP’s work from ACSD has also been published and work related to modeling in evaluations by Senior Scientist Neff Walker and other colleagues is in press. And the Malawi Ministry of Health has requested assistance developing a country-specific version to implement.
For more information about IIP’s work in the Catalytic Initiative, visit http://www.jhsph.edu/iip
--Brandon Howard, September 2010