An Afghan Success Story:
The Balanced Scorecard and Improved Health Services
Since 2003, the Department’s Health Systems Program has been a major force in the reconstruction of the Afghanistan health sector. After the fall of the Taliban, the Afghanistan Ministry of Public Health was left with a system in near ruins. The Ministry decided that to achieve its vision of a sustainable, transparent, and equitable health system it needed independent technical assistance to monitor and evaluate the delivery of health services. After successful bidding and negotiations with the government and other stakeholders, Health Systems faculty, led by Drs. David Peters and Gilbert Burnham, began developing an innovative system for assessing the performance of Afghanistan’s national health system: the Balanced Scorecard.
While the Scorecard has been used for years in the healthcare sector, it had never before served as a management tool for a country’s entire health system. More than a mere tool to monitor and evaluate service delivery, the Scorecard defined an approach to help the Ministry assess its progress in realizing its core goals of quality, access and equity.
Afghanistan Basic Package of Health Services
In 2002, after decades of conflict, the Afghanistan health system was among the worst in world. The infrastructure was in shambles and trained personnel were scarce. In rural areas the situation was even more severe. With the help of the European Commission, United Nations Children’s Fund (UNICEF), United States Agency for International Development (USAID), World Bank (WB), and the World Health Organization (WHO), the Ministry was able to start rebuilding.
|The Balanced Scorecard had never before served as a management tool for a country’s entire health system.|
Owing to capacity, infrastructure and administrative issues, the Ministry decided that the best course for ramping up service delivery across the country was to contract with non-governmental organizations (NGOs) which already provided about 80 percent of health services in the country. The Ministry would therefore play an oversight role to ensure that agencies were providing services effectively and efficiently. As a roadmap for these providers the Ministry developed the Basic Package of Health Services (BPHS), which includes a set of cost-effective primary health care services with emphasis on the needs of rural areas and women and children. With clear goals defined, the next step was to develop a system to assess how well NGOs and Ministry providers performed in order to make evidence-based decisions with regard to future contracting and funding.
The Balanced Scorecard
A. Patients & Community
C. Capacity for Service Provision
D. Service Provision
E. Financial Systems
F. Overall Vision
With no routine national health information system in place, the Ministry of Public Health had to start from scratch. In a rather unusual arrangement for the Department, the Ministry contracted directly with a team of researchers from our Health Systems Program and the Indian Institute of Health Management Research (IIHMR) to provide independent technical assistance in building a new monitoring and evaluation system from the ground up. Through a collaborative process that included representatives from the government, international agencies and NGOs, the research team facilitated the development of a set of indicators that the government could use to objectively measure performance. The decision to implement the Scorecard at the national level was both an exciting and daunting challenge for all parties involved, not only because it had never been done before but because the circumstances inside Afghanistan—from security to resources—presented many extraordinary difficulties.
Business Management Roots
Robert Kaplan and David Norton conceived of this new management tool in their book entitled, The Balanced Scorecard: Translating Strategy into Action. The authors developed the Scorecard to measure more than just the bottom line. They also wanted to help organizations translate overall vision and strategies into practice. For an organization to realize its mission statement, objective milestones had to be defined. And to get buy-in from all levels of the organization, the development process had to be collaborative to ensure confidence in what was being measured. Organizations that have successfully used the Balanced Scorecard include Cornell University, IBM, Verizon, and Walt Disney (http://www.balancedscorecard.org/).
The Afghanistan Balanced Scorecard
After the fall of the Taliban, the Ministry of Public Health declared its top priority
to strengthen the delivery of sustainable, quality accessible health services, especially targeted at women, through planning for, and the effective and efficient implementation of the basic health services package.*
Because the Scorecard is designed explicitly to align an organization’s actions with its vision, the Ministry decided to adapt it with the technical assistance from the team led by the Health Systems Program. While the Department faculty were hired by the government, their role was to provide independent advice and mediate between the various stakeholders in the development of indicators. In order to engender trust in this new system among personnel at all levels of the system, a series of workshops were held with the Ministry, NGOs and other partners in the health sector.
Six domains to assess were identified during the development process:
- Patients and community perspective
- Staff perspective
- Capacity for service provision
- Service provision
- Financial systems
- Overall vision for the health sector
Twenty-nine core indicators along with two composite scores were agreed upon by the stakeholders. For each indicator, achievable benchmarks were set based on the realities of the country. Upper benchmarks were set at a level that was currently being achieved by 6 of the 33 provinces in 2004.
Every year, a sample of over 600 facilities from across the country is surveyed—the largest independent facilities survey globally. The domains are analyzed at the provincial level in large part because the government contracts with NGOs are arranged at this level.
To make the Scorecard easy to interpret, it is color-coded. Green=top quintile; yellow=middle 3 quintiles; red=bottom quintile. This makes comparisons easier across provinces and allows staff at all levels to quickly see where extra effort is needed or where government support is lacking relative to other provinces. “In the red zone” has even been appropriated by health sector personnel to describe areas that need special attention.
Although travel never proved easy—with Soviet-era highways that abruptly end and poor access to many rural regions—Department staff were able to visit survey sites in relative safety when the project first began. Over the last few years, however, the security situation has deteriorated, making it nearly impossible to visit many parts of the country. Local staff must be relied on to make visits to most rural facilities.
On top of the many security challenges that exist in the county, recruiting women to conduct maternal interviews has been a continuing challenge. Many women in Afghanistan do not have a formal education and are not literate. Moreover, women who are literate and permitted to work require a male family member to escort them. In addition to limiting the available workforce, this also creates exceptional demand for qualified women.
While project faculty and staff have made capacity building a priority—establishing the first Institutional Review Board ever in the country, for example—long-term capacity building may prove difficult. Because of the security situation, many highly trained Afghanis emigrate. The country has no School of Public Health to participate in these efforts. The Afghan Public Health Institute, however, has recently been established and it is hoped that institutional capacity can be built.
A stable evaluation system used at all levels
The Balanced Scorecard has become the cornerstone of the government’s monitoring and evaluation system. The NGOs that provide about 80 percent of the services are also utilizing the Scorecard to monitor and improve their own performance. Provincial managers are even using the Scorecard as a negotiating tool with the central government. Areas such as understaffing are reflected in the indicators and can be compared to other provinces. Therefore, managers can show how variables controlled centrally, such as staff funding or equipment availability, are adversely affecting a province’s overall performance.
Continued improvement in health delivery
The latest Balanced Scorecard Report from 2008 shows continued improvement since 2004. Overall the national median score across all the indicators has increased from 50 percent in 2004 to 72 percent in 2008. In fact there has been notable progress in each of the domains since 2004:
- nearly 20-point improvement in the patient and community domain which captures patient satisfaction
- Over 25-point improvement in service provision, which includes antenatal and delivery care
- A 28-point improvement in capacity for service provision, with large gains in availability of family planning supplies and lab functionality
|The Balanced Scorecard has become the cornerstone of the government’s monitoring and evaluation system.|
|Doctoral Research in Afghanistan|
Faculty and students have conducted a wide range of research in Afghanistan since 2002 with funding from government and international agencies. Below are four of the doctoral students whose dissertations focus on the Afghan health system. Their current position is listed after the dissertation title.
|Peter Hansen, PhD: Health service performance assessment in Afghanistan; GAVI chief of evaluation|
|Shivam Gupta, PhD: Methods for Population-based Assessments in Post-conflict Settings: Health Service Performance, Economic Status and Equity of Utilization in Afghanistan; Research Associate in the Department|
|Aneesa Arur, PhD: Contracting for Health Services in Afghanistan: An Analysis of Changes in Service Utilization and Quality of Care between 2004 and 2005; Researcher with Abt Associates|
|Laura Steinhardt, PhD: Determinants of Access to Primary Health Care Services in Afghanistan; To begin as CDC EIS fellow|
The Health Systems Program and its partner the Indian Institute of Health Management Research were recently awarded the follow-up grant to continue their Balanced Scorecard work. As part of this new 4-year project, they will be redesigning the Scorecard to reflect the health sector’s achievements. New benchmarks need to be developed since provinces have surpassed nearly every benchmark based on 2004 performance levels.
Plans are also underway to capture more provider treatment and community-level information. For instance, in the past, surveys recorded information about whether a correct diagnosis was made. However, no separate assessments of the patient’s diagnosis or prescription given were collected. Household surveys conducted by Hopkins and IIHMR have shown that coverage of health services is increasing while child mortality is falling. It is hoped that in future iterations, more in-depth information such as this can be regularly collected and used by the NGOs and Ministry of Public Health.
The success of the Scorecard has also created demand at lower levels of the health system. While administrators at the provincial level have utilized the Scorecard to make decisions about policies and staffing, efforts are being made to use the Scorecards at the community, district and facility levels. The Scorecard has contributed to a more accountable and effective health system in Afghanistan—questions are also being asked whether these efforts can contribute more broadly towards creating stable and trusted public institutions that look out for disadvantaged groups. The Future Health Systems research consortium, another project based in the Health Systems Program, will be looking at these issues in more depth.
The success of the health sector has been a beacon of hope for an Afghanistan government that has been struggling in recent years. The Ministry of Public Health’s bold choices have fostered a resurgent health system in a post-conflict environment that presents many ongoing challenges, including basic security. In the face of the many obstacles, the Ministry established a results-based system rooted in transparency and accountability, which has led to better quality healthcare for its citizens and better access for those who need it most.
Further Reading: Faculty Articles
Measuring and managing progress in the establishment of basic health services: the Afghanistan health sector balanced scorecard. Hansen PM, Peters DH, Niayesh H, Singh LP, Dwivedi V, Burnham G. Int J Health Plann Manage. 2008 Apr-Jun;23(2):107-17. http://dx.doi.org/10.1002/hpm.931
A balanced scorecard for health services in Afghanistan. Peters DH, Noor AA, Singh LP, Kakar FK, Hansen PM, Burnham G. Bull World Health Organ. 2007 Feb;85(2):146-51. http://dx.doi.org/10.2471/BLT.06.033746
Determinants of skilled birth attendant utilization in Afghanistan: a cross-sectional study. Mayhew M, Hansen PM, Peters DH, Edward A, Singh LP, Dwivedi V, Mashkoor A, Burnham G. Am J Public Health. 2008 Oct;98(10):1849-56. http://dx.doi.org/10.2105/AJPH.2007.123471
The effect of wealth status on care seeking and health expenditures in Afghanistan. Steinhardt LC, Waters H, Rao KD, Naeem AJ, Hansen P, Peters. DH. Health Policy Plan. 2009 Jan;24(1):1-17. http://dx.doi.org/10.1093/heapol/czn043
Determinants of primary care service quality in Afghanistan. Hansen PM, Peters DH, Edward A, Gupta S, Arur A, Niayesh H, Burnham G. Int J Qual Health Care. 2008 Dec;20(6):375-83. http://dx.doi.org/10.1093/intqhc/mzn039
*Ministry of Health. A basic package of health services for Afghanistan. Kabul: Islamic Transitional Government of Afghanistan, February 2003.
—Brandon Howard, March 2010