Published in 1993 by the Aga Khan Foundation USA, Suite 700, 1901 L Street N.W., Washington DC and the Aga Khan Foundation, P.O. Box 435, 1211 Geneva 6, Switzerland. Additional copies are available at the Aga Khan Foundation Canada in Toronto; Aga Khan Foundation (United Kingdom) in London; and AKF offices in Dhaka, Bangladesh; Delhi, India; Nairobi, Kenya; Karachi, Pakistan; Lisbon, Portugal; and Dar-es-Salaam, Tanzania; the Aga Khan University Faculty of Health Science in Karachi, Pakistan; the Somboon Vacharotai Foundation and the ASEAN Institute for Health Development in Thailand A CHW monitors the weight of a mother during a prenatal home visit in a peri-urban slum of Dhaka, Bangladesh Photo by Jean-Luc Ray for AKF ISBN: 1-8828839-03-X LOC Catalog Number 92-75464 Dedicated to Dr. Duane L. Smith (1939-1992), Dr. William B. Steeler (1948-1992) and all other health leaders, managers and workers who follow their example in the effort to bring quality health care to all in need. The monitoring of levels of health awareness among women of all generations, particularly in rural areas like this village in China where contamination from farmyard manure is frequently a problem, can help track achievements, refine strategies and set improved priorities and action plans Photo by Jean-Luc Ray for AKF An overview of PHC MAP The main purpose of the Primary Health Care Management Advancement Programme (PHC MAP) is to help PHC management teams collect, process and analyse useful management information. Initiated by the Aga Khan Foundation, PHC MAP is a collaborative programme of the Aga Khan Health Network The Aga Khan Health Network includes the Aga Khan Foundation, the Aga Khan Health Services, and the Aga Khan University, all of which are involved in the strengthening of primary health care> and PRICOR.<$FPrimary Health Care Operations Research is a worldwide project of the Center for Human Services, funded by the United States Agency for International Development> An experienced am and equally experienced PHC practitioner teams in several countries, including Bangladesh, Chile, Colombia, the Dominican Republic, Guatemala, Haiti, India, Indonesia, Kenya, Pakistan, Senegal, Thailand and Zaire, have worked together to develop, test and refine the PHC MAP materials to make sure that they are understandable, easy to use and helpful. PHC MAP includes nine units called modules. These modules focus on essential information that is needed in the traditional management cycle of planning-doing-evaluating. The relationship between the modules and this cycle is illustrated below. Managers can easily adapt these tools to fit local conditions. Both new and experienced programmers can use them. Government and NGO managers, management teams, and communities can all use the modules to gather information that fits their needs. Each module explains how to collect, process and interpret PHC-specific information that managers can use to improve planning and monitoring. The modules include User's guides, sample data collecting and data processing instruments, optional computer programs, and Facilitator's guides, for those who want to hold training workshops. The health and management services included in PHC MAP are listed below: Health and management services, +, + HEALTH SERVICES, +, MANAGEMENT SERVICES ^, ^, ^ GENERAL PHC household visits Health education MATERNAL CARE Antenatal care Safe delivery Postnatal care Family planning CHILD CARE Breastfeeding Growth monitoring Nutrition education Immunization Acute respiratory infections Diarrhoeal disease control Oral rehydration therapy, OTHER HEALTH CARE Water supply,, hygiene and <~>sanitation School health Childhood disabilities Accidents and injuries Sexually transmitted diseases HIV/AIDS Malaria Tuberculosis Treatment of minor ailments Chronic,, non-communicable <~>diseases, Planning Personnel management Training Supervision Financial management Logistics management Information management Community organisation Several Manager's guides supplement these modules. These are: Better management: 100 tips, a helpful hints book describing effective ways to help managers improve what they do; Problem-solving, a guide to help managers deal with common problems; Computers, a guidebook providing useful hints on buying and operating computers, printers, other hardware and software; and The computerised PRICOR thesaurus, a compendium of PHC indicators. The Primary Health Care Management Advancement Programme has been funded by the Aga Khan Foundation Canada, the Commission of the European Communities, the Aga Khan Foundation U.S.A., the Aga Khan Foundation's head office in Geneva, the Rockefeller Foundation, the Canadian International Development Agency, Alberta Aid, and the United States Agency for International Development under two matching grants to AKF USA. The first of these grants was "Strengthening the Management, Monitoring and Evaluation of PHC Programmes in Selected Countries of Asia and Africa" (cooperative agreement no. OTR-0158-A-00-8161-00, 1988-1991); and the second was "Strengthening the Effectiveness, Management and Sustainability of PHC/Mother and Child Survival Programmes in Asia and Africa" (cooperative agreement no. PCD-0158-A-00-1102-00, 1991-1994). The development of Modules 6 and 7 was partially funded through in-kind contributions from the Primary Health Care Operations Research project (PRICOR) of the Center for Human Services under its cooperative agreement with USAID (DSPE-6920-A-00-1048-00). This support is gratefully acknowledged. The views and opinions expressed in the PHC MAP materials are those of the authors and do not necessarily reflect those of the donors. All PHC MAP material (written and computer files) is in the public domain and may be freely copied and distributed to others. Contents QUICK START 1 INTRODUCTION 5 Types of indicators to be monitored 6 How long should you monitor PHC activities? 15 Keep it straightforward and simple (KISS) 17 Limitations of service monitoring 17 MONITORING PROCEDURES 19 Step 1: Specify the monitoring objectives 20 Step 2: Decide on the scope of the monitoring 23 Step 3: Select the indicators and performance standards 24 Step 4: Choose information sources and develop data collection procedures 27 Step 5: Collect the data 29 Step 6: Tabulate and analyse the data 30 Step 7: Present the findings 33 Step 8: Take appropriate action 34 Step 9: Decide whether to continue monitoring 35 APPENDICES A. PHC service delivery indicators 39 B. PHC management indicators 65 C. Impact indicators for monitoring mortality, morbidity, disability, and fertility 75 D. Summary list of indicators for PHC activity monitoring 79 E.Blank worksheets 103 REFERENCES AND BIBLIOGRAPHY 107 ACRONYMS AND ABBREVIATIONS 108 GLOSSARY 110 Acknowledgements The first draft of this module was developed by Lynne Miller Franco and Jack Reynolds in early 1991. It was reviewed by a number of outside experts and field tested in several PHC programmes in mid and late 1991. Paul Richardson took responsibility for revising the module in early 1992. The revision was reviewed by participants at the International Conference on Management and Sustainability of PHC Programmes, held in Bangkok in May, 1992. The participants recommended the addition of indicators on a number of other PHC services, including sexually-transmitted diseases, childhood disabilities, and accidents and injuries. Neeraj Kak prepared these lists of indicators and Paul Richardson made the final revision. Special thanks are due to Maria Francisco, who helped develop several of the indicator lists, and Mary Millar, who developed the Facilitator's guide for the module. External Reviewers: Donald Belcher. Veterans Administration, Seattle WA, USA David H. Peters Johns Hopkins University School of Hygiene and Public Health, Johns Hopkins University, Baltimore MD, USA Paul Zeitz Johns Hopkins University School of Hygiene and Public Health, Johns Hopkins University, Baltimore MD, USA Michael Bernhart. University of Puget Sound, WA, USA Internal Reviewers and Contributors: Thailand: Jumroon Mikhanorn Somboon Vacharotai Foundation (SVF) Thongchai Sapanuchart SVF Donna Robinson SVF Yawarat Porapakkham ASEAN Institute for Health Development (AIHD) Chaweewon Boonshuyar AIHD Nawarat Suwannapong AIHD Som-Arch Wongkhomthong AIHD Orapin Singhadej AIHD Peerasit Kamnuansilpa Health and Population Research Company (HPRC) Butsabar Subongkot HPRC Field tests: Countries: Participating organisations, field test facilitators Bangladesh:Concerned Women for Family Planning; The Asia Foundation; Facilitator: Barkat-E-Khuda, URC Bangladesh. Pakistan: Aga Khan University, Karachi; Facilitator: Khatidja Husein, Aga Khan University. Thailand: Ministry of Public Health, Srisaket, SVF; AIHD; HPRC; Facilitator:Peerasit Kamnuansilpa, HPRC. India: Junagadh PHC Project; Sidhpur Sustainable Health System Project, Gujarat; Aga Khan Health Service, India; Facilitator: Neeraj Kak, URC; Vijay Moses, Aga Khan Health Service, India. Kenya: Mombasa PHC Project; Kisumu PHC Project; Facilitators: Paul Richardson, URC; Esther Sempebwa, Mombasa PHC Project; Matthew Onduru, Kisumu PHC Project. Senegal: Ministry of Health; Facilitator: Mounir Toure. Indonesia:Department of Health, Jakarta; Facilitator: Sandi Iljanto, University of Indonesia. Quick start If you want to get started quickly, follow these instructions for monitoring PHC activities. Monitoring is the periodic collection of data to determine if activities are being implemented as planned. Most PHC managers already have a monitoring system that records data regarding programme inputs and outputs. In developing a practical monitoring system, you should consider the following guidelines: Keep the data collection and reporting simple for managers and workers with limited training. Collect only essential data. If there is no immediate need, do not collect the data. There are limits as to how much information a manager can collect and use effectively. Monitoring should not over-burden operational and MIS staff. Provide timely feedback and use the information. Make sure that managers at all levels receive, analyse and utilise the information. This basic Quick start analysis makes two assumptions: the analysis will use existing data collection forms, or existing forms will be modified; and only input, output and effect data will be monitored. Inputs are the resources which enable the PHC programme to produce outputs. Outputs are the immediate services and products that your programme provides. Effects are the changes in knowledge, skills, motivation, behaviour (including coverage) that result from your products and services. You should already have an idea about the type of information that you need. The Quick start involves three stages: planning, implementing, and reviewing. Stage 1: Planning Specify the objectives by clarifying which PHC services or management services will be monitored, why the data will be gathered, and who will use it. You should be clear about the purpose of monitoring and who needs the information. Determine the scope of monitoring by specifying the administrative areas, the types of facilities or service components, and the duration of data collection. In most cases data will be gathered for a short term to determine if programme resources and activities have been implemented as planned.<%0> Select input, output and/or effect indicators for PHC services or management support services. The following are general indicators which can be used for most PHC services. Appendices A and B provide detailed lists of indicators. PHC services (For example, ANC, growth monitoring, immunizations, ORT, and curative care for ARI, malaria and other common diseases.): Effects: Number or percent of target group covered by a PHC service, e.g., percent of children << 2 years fully immunized Number or percent of mothers who gain PHC skills, e.g., number who can prepare ORS solution correctly Outputs: Number of services and products provided to new and continuing users, e.g., number of ORS packets distributed Number of contacts to inform and motivate eligible clients, e.g., number of community visits by programme teams Inputs: Number of personnel per facility, e.g., number of nurses per clinic Number of supplies and equipment per facility or health worker (e.g., number of health workers with IEC materials) Management support services For example, planning, training, supervision, personnel, MIS, logistics, and finance Effects: Number or percent of health workers who gain needed skills, e.g., percent of trained CHWs who can counsel mothers on nutrition Number or percent of staff who follow program policies, e.g., number of drivers who use seat belts Outputs: Number or frequency of management support activities completed, e.g., number of training sessions completed Number of products produced, e.g., number of financial reports distributed Number of supplies inventoried and distributed, e.g., number of BCG ampoules distributed to clinics Inputs: Number of facilities with requisite personnel and skills, e.g., number of workers employed with required education level Number of supplies procured, e.g., number of family planning methods received from donors Select performance standards. For each indicator select a "target." These standards are compared with actual performance. For example, your target for immunization coverage may be set at 70 percent. You would compare your actual coverage to that target. Choose the information sources and the data-gathering procedure. Most of this information will probably come from existing or modified sources such as treatment data, logistics records, and activity reports. In some cases, a new indicator will be added to an existing form. Whenever possible use the existing data gathering, compilation, and reporting system. Stage 2: Implementing Collect the data. When a new form is developed or an existing form is substantially modified, pre-test it on a small scale. Data collection should be carefully supervised to ensure that the information is accurate and complete. This often involves training and re-training field staff that collect and compile data. WORKSHEET FOR COMPARING ACTUAL PERFORMANCE WITH ITS STANDARD OF PERFORMANCE, +, +, +, +, + Components/ Indicators, +, Actual performance, Standard performance, % Achieved of standard, Action to be taken TEXT PHC services, +, , , , 1., No. HH with latrine, 400, 500, 80%, Organise teams to construct latrines 2., No. of mothers who can interpret GM card, 600, 600, 100%, Increase target to 750 Management, +, , , , 1., No. of CHW's with improved performance, 15, 13, 115%, Provide incentives for performance Tabulate and analyse the data. Compute the results by comparing the actual with the performance standards, i.e., divide the monitoring indicator by the standard. For example, if the performance target for community activities was 50, and 30 were actually completed, the performance achieved would be only 60 percent. You can use the computer files in modules 4, 6 and 7 to process your data quickly. Look for discrepancies between the input and output indicators and the targets; trends over time that are increasing or decreasing; and administrative areas or facilities that fall substantially above or below the norm for performance. Present the results of monitoring to those involved in service management and delivery, and take action. Each monitoring report should include actions to be taken and the staff responsible for mplementing those actions. Stage 3: Reviewing Indicators should be reviewed periodically to determine if they should be dropped, modified, or continued. When monitoring results are not being used, you should consider discontinuing the indicator. In most cases, monitoring data are not effectively utilised because managers have not been trained and supervised to analyse the data and to develop an action plan. Thus you should determine if information is not being used because it is no longer useful or because managers have not been trained, directed, or supervised. If your original intention was to monitor indicators over a short period of time in order to ensure that activities were implemented as planned, you may decide to incorporate one or two key indicators into the routine monitoring system. INTRODUCTION Monitoring is the periodic collection and analysis of selected indicators to enable managers to determine whether key activities are being carried out as planned and are having the expected effects on the target population. Monitoring provides feedback to project management in order to improve operational plans and to take corrective action. Indicators can be used to: measure achievement of targets; assess changes/trends in health status; compare the level of achievement between working areas or project sites; and identify currently under-served areas. An indicator is defined as an indirect measure of an event or condition. For example, weight-for-age is an indirect measure (indicator) of a child's nutritional status. Although most managers already have a monitoring system, it may not allow them to monitor some of the PHC and management services that they deem to be especially important. This module is designed to fill that gap. Managers can use it to select a limited number of indicators from lists that have been compiled for each PHC service and management service. Guidelines in the module explain how to design and implement simple monitoring "systems" using these (or other) indicators. Most PHC managers have to oversee a large number of programme services. In this series of modules we have divided those services into two categories: PHC services (immunization, antenatal care, etc.) and management support services (planning, supervision, etc.). Types of indicators to be monitored It is helpful to classify indicators into broad categories and then to select one or two from each category so that key parts of the activity can be monitored and overlap can be avoided. PHC MAP uses a "systems framework" to describe PHC programmes in broad categories of inputs, processes, outputs, effects, and impacts (see Module 1). This module suggests some generic indicators for four of these categories: inputs, resources needed to carry out the programme; outputs, the services or goods produced by the programme; effects, the knowledge, attitude, and behavioural changes that result, including coverage; and impacts, changes in health or fertility status due to the effects. By "generic," we mean indicators that can be applied to most PHC services. This module focuses on monitoring inputs, outputs, and effects. Generic impact indicators for mortality, morbidity, disability, and fertility are included, but the user who wants to monitor these should consult Module 4, Surveillance of morbidity and mortality. The recommended generic indicators are summarised below. Appendix A provides an annotated list of suggested indicators for each of the specific PHC services. Appendix B provides a similar annotated list for each of the specific PHC management support services. Appendix C provides a separate list of annotated PHC impact indicators to monitor mortality, morbidity, disability, and fertility. Appendix D is a summary list of PHC service and management service indicators with cross-references to other PHC MAP modules. PHC service indicators Since the primary purpose of this module is to monitor PHC services, the lists of generic indicators start with effects to encourage linking the effect with the outputs and inputs needed to carry out that activity. Table 1 illustrates this linkage. Obviously, not every PHC service needs to be monitored with all of the generic indicators. Indicators should be selected to fit specific needs. Table 1: Generic indicator categories for PHC services Effects , Behaviour (also called coverage) , Knowledge and skills Outputs , Utilisation of services , Quality of care , Contacts,, visits , Access to services , Unit costs Inputs , Availability of personnel,, supplies,, equipment,, funds PHC service effect indicators Effects are changes in knowledge, skills, attitudes, and behaviour that result from the PHC service. Since attitudes are very difficult to measure, three types of indicators are emphasised: knowledge, skills, and behaviour, or practice. Behaviour indicators tell whether the target group has done or is doing what is needed to be protected by the PHC intervention. You can also call this coverage, which is a measure of the proportion of the target group that is following a prescribed behaviour or practice, e.g., using contraception. In PHC, coverage also refers to the proportion of the target group that has received the prescribed treatment in a correct and complete manner, e.g., fully immunized. Coverage measures usually include all people in the target group in the denominator, including those who do not receive services from the PHC programme. Thus, the two sub-categories of behaviour indicators are: Practice, e.g., proportion of eligible couples using modern contraceptives Treated, e.g., proportion of children under five years of age who are fully immunize Knowledge indicators tell the manager whether mothers have understood key health education messages. Skill indicators tell whether they know how to perform important health tasks. Knowledge, e.g., number or percent of TB patients who know why it is important to complete the treatment for tuberculosis Skills, e.g., number or percent of mothers who can correctly administer ORT Record systems that are based on complete household registration may be able to provide some data on effects (see Module 3) but, by and large, this information is rarely available from service statistics and will usually require a community survey (see Module 2). Usually, effects data should not be collected frequently since significant changes would not normally occur rapidly. Furthermore, data collection can be time-consuming and expensive. Annual monitoring of effects may be sufficient, unless changes in strategy are being implemented and short-term monitoring is being carried out. Analysis of effects would look at changes over time, comparisons with project norms, comparisons among health centres and breakdowns of distributions by characteristics such as age, parity, socio-economic status, and geographic area. PHC service output indicators Outputs are the services and products that the PHC programme generates with its resources. They are what lead to the effects on mothers or children. Outputs can be seen as the immediate result of PHC processes, or services. The result of a process can be measured as an output. For example, the process of visiting pregnant women can be measured as the number of pregnant women visited, an output. The most important types of outputs in PHC are: utilisation of the service; quality of the service provided; contacts of those in need of, or eligible for, the service; and access to the service. The generic indicators we suggest fall into these four sub-categories. Utilisation: To have an effect on knowledge, skills, and behaviour, PHC services must be utilised. This set of indicators tells the manager whether they are utilised and, if so, how. Utilisation is the initial link between the effort that goes into providing a service and the improvement in health that should result from using the service. Utilisation indicators are important for both outreach and clinical services. Depending on the intervention, utilisation can be expressed in several ways. For services which require a long period of constant contact, such as family planning, tuberculosis treatment, and growth monitoring, it may be useful to distinguish between those ever using (number of acceptors, number enrolled, number registered) and those currently using (number of current users, number of active cases). Useful negative indicators include the number of drop-outs, past users, and inactive users.<%0> Acceptors/users, e.g., number or percent of children enrolled in growth monitoring Continuation, e.g., number or percent of current users of family planning Drop-outs, e.g., number or percent of ANC drop-outs Information on utilisation can usually be obtained from clinic and outreach reports as well as community surveys. The distribution of users can be compared to targets and broken down by geographic area, age, sex, ethnic group, socio-economic status, etc. Quality of care: Information on the quality of service delivery is often important to managers for two reasons. First, poor service quality can lead to drop-outs, low service utilisation, and poor knowledge, attitudes, and practice. Second, gaps in service quality imply that staff training and supervision may be required. Quality indicators that managers will usually want to monitor are in diagnosis, education, treatment, and counselling. Diagnosis, e.g., number or percent of health workers who screen fever patients for signs of other serious illnesses, e.g., meningitis, pneumonia, etc. Education, e.g., number or percent of women who receive correct information on immunization schedules Treatment, e.g., number or percent of babies weighed correctly Counselling, e.g., number or percent of health workers who counsel mothers on nutrition needs of their children<%0> Data for these indicators are rarely collected routinely. The best information usually comes from direct observation of health workers and can be compiled from service delivery quality assessment checklists (Module 6). For some specific indicators, information may be available from health centre registers or health cards. Exit interviews may also provide the necessary information. Analysis typically focuses on health worker performance over time, comparisons with project norms, and comparisons among health centres. Contacts: Services may not be utilised if target groups are neither informed nor motivated to use them. This type of indicator tells the manager the proportion of the target group that is being contacted or visited by the programme, usually through outreach workers, group health education, or mass media. Such contacts, however, do not necessarily result in utilisation. Low contact rates may indicate that health workers are not visiting all households in their catchment areas, are visiting them too infrequently, or are not adequately conveying educational and motivational messages. This indicator is especially important for outreach services and can be further refined to focus on high-risk women and children. That is, the programme could give priority to those most in need of services by scheduling appropriate outreach and clinic sessions. Contact, e.g., number or percent of households contacted by malaria workers High-risk contact/visit, e.g., number or percent of malnourished children followed up by health workers Data can come from clinic records, outreach worker and supervisor records, and community surveys. Analysis is similar to that already described. The distribution of people contacted or visited can be compared to targets and broken down by geographic area, age, sex, ethnic group, socio-economic status, etc. Access: This type of indicator will assist managers to determine whether the services are accessible to the intended target groups. This is likely to be of greater concern in rural and remote areas than in urban areas. But even in urban areas, PHC services may not be accessible due to distance, schedules, costs, and cultural obstacles. Indicators in this group are particularly important for preventive services and for outreach programmes covering large catchment areas. Access indicators are often critical for monitoring new or revised services, since people must have access to a service, before they can use it. Accessibility can be measured in terms of distance from a service, time required to get to a service or the number of service sessions held in a given time period. Physical distance, e.g., number or percent of population living within five kms of a health facility Time, e.g., number or percent of population within a 15-minute walk of clean water Frequency of service: e.g., number of sessions held per week Access indicators often require first mapping out the catchment area to determine the proportion of the target group that is and is not being served (Module 3). How frequently these indicators should be monitored depends on how quickly access can be expected to change. In many cases, monitoring on an annual or semi-annual basis would be sufficient. Data on accessibility can be collected from community surveys, health session activity reports, and outreach worker reports. Analysis of accessibility data usually involves examining the distribution of the target group served and unserved by geographic distance from the service site. This information can also be gathered by a population-based survey (Module 2). The distribution can be compared to targets and broken down by geographic area, age, sex, ethnic group, socio-economic status, etc. PHC service input indicators Inputs are the resources needed to carry out the project's services. They enable the project to produce its outputs. Most PHC services require certain essential resources. In addition to personnel, nurses, physicians, outreach workers, volunteers, community leaders, etc., there are often key supplies that are needed to provide the service, vaccines for immunization, scales for growth monitoring, contraceptives for family planning. And for some services, it is imperative to have the proper equipment, e.g., X-ray machines for TB, vehicles for outreach, refrigerators for immunization. Managers and staff usually know which resources are essential and which are the most likely to be unreliable. If problems are being experienced or are anticipated, it may be useful to monitor the availability of these key resources. We suggest formulating the input indicators in the negative to trigger immediate action. The most common input indicators would monitor the availability of personnel, supplies, equipment, and funds. Personnel, e.g., number of trained TBAs; population per active CHW Supplies, e.g., number or percent of health centres without sufficient ORS packets Equipment, e.g., number or percent of outreach workers without operational motorbikes) Potential information sources for these indicators include logistic supply records, stock inventory forms, supervisor reports, and CHW reports. An item could be added temporarily to these or other routine reports if it is not already available. Data can also be collected with Module 6 and 7 checklists. If the resources are supposed to be available in households, e.g., growth monitoring cards, ORS packets, the required data could come from home visit forms or could be included in a community survey (Module 2). Analysis and interpretation of input indicators are very straightforward. If supplies are found to be inadequate, there is a problem. This information should stimulate the obvious solution of supplying the needed inputs. If the cause of the shortage is unknown, this should trigger further investigation into the logistics system. The logistics checklist in Module 7 could be of help in that case. PHC management Indicators for the eight management services can also be categorised as effects, outputs, and inputs. Table 2 illustrates this linkage. Table 2: Generic indicator categories for PHC management services Effects , staff behaviour (and "coverage") , Staff knowledge,, skills Outputs , Services or activities completed , Frequency of management services , Quality of management services Inputs , Availability of trained personnel,, supplies,, information , Guidelines/protocols Management effect indicators The management services are designed to support PHC service delivery. Thus, the effects of the management services can be measured in terms of the knowledge, skills and behaviour of service providers and support staff. Figure 1 shows how management services affect PHC services. Appendix B lists effect indicators you should consider: Staff behaviour, e.g., number or percent of PHC staff who follow travel guidelines Staff knowledge, e.g., number or percent of trained PHC providers who know how to prepare a work plan Staff skills, e.g., number of PHC nurses who received training in the correct use of an autoclave and can correctly use autoclave equipment Figure 1: A systems diagram of management and services relationships Management services PHC services Management output indicators Outputs are the services and products that PHC management services generate with their resources. There are basically three types of management output indicators: Services or activities completed, e.g., the number of PHC providers trained Frequency of activity, e.g., the percent of facilities completing quarterly reports Quality of management activity, e.g., the number of supervision visits that included review or follow-up on problems from previous visit. Information for these indicators can often be obtained from review of programme plans, interviews and discussions with health workers and managers, review of supervision and training reports, personnel records, account books, stock records, and activity plans. Module 7 provides a more detailed description of the various data sources for these indicators. As with PHC service indicators, the analysis will typically focus on comparisons over time, comparisons with performance standards, or comparisons among health units. Management input indicatoRS Inputs are the resources needed to carry out the management services. We have suggested a few that often pose problems: Personnel, e.g.,the number or percent of PHC providers recruited with requisite skills Supplies, e.g.,the number of IEC materials received for training outreach providers Information, e.g.,the number of monthly service reports received from private physicians and midwives Guidelines/protocols, e.g.,the number of supervisors with the written protocols for supervision. Management may come from outside the PHC organisation or from another department inside the organisation. Information for these indicators can be obtained from interviews, observations, personnel records, and programme plans. Again, Module 7 discusses information sources and use of this information. How long should you monitor PHC activities? PHC managers carry out one of two types of monitoring, which we will call "routine" and "short-term." Example: Thailand A simple monitoring system was put in place at the village level in a north-eastern province to track coverage of six child survival interventions (immunizations, growth monitoring, ORT, antenatal care, family planning, and water and sanitation). This monitoring system was based on "neighbourhood health inventory cards." Data were collected by local village health volunteers, and cards were displayed at the volunteer's home to make it easy for the community to track progress. Supervisors collected the data every two months, did quick on-the-spot analyses, provided feedback to the volunteer and community leaders, and forwarded the results to the province health office for data entry and aggregate analysis. This system enabled PHC administrators, health workers, and community leaders to remain informed about the PHC services and achievements made in each village. Routine monitoring involves compiling information on a regular, ongoing basis for a core set of indicators. The number of indicators should usually be kept to a minimum but should still provide the manager with sufficient information to track progress. Routine monitoring can be used to identify where programme implementation is or is not proceeding as planned. Short-term monitoring is done for a limited period of time and usually for a specific activity. Often when new activities or processes are implemented, managers need to know whether they are being implemented as planned and whether they are having the desired effect. Managers normally use this information to make adjustments in the new intervention. Once implementation is underway, key indicators are incorporated into routine monitoring. Short-term monitoring is also used when managers have identified a problem in the delivery of inputs and services and additional information is needed. Monitoring in this case is used to examine a problem and pinpoint gaps in service provision and management support services. Example: Zaire A health zone manager set up a special monitoring system to assess the effectiveness of a solution he had implemented to address a nutrition problem. He found that mothers of malnourished children often left growth monitoring sessions without understanding their child's nutritional status and what they should be doing about it at home. He believed that the cause of this problem was that the health workers had insufficient time to provide counselling to mothers. Time spent with mothers averaged only one minute. As a consequence, he decided to reorganise the growth monitoring sessions. Children were weighed by community volunteers who performed a triage, directing malnourished children to be seen by the nurse. This should have allowed the nurse to spend more time counselling those mothers with children most in need of attention. The manager monitored the effects of his strategy by collecting information on whether the new triage approach was being properly implemented, the average amount of time that nurses spent with mothers of malnourished children, and mothers' knowledge of their children's nutritional status and of activities they needed to carry out at home. He found that the time per mother for counselling tripled, and mothers' knowledge upon leaving the growth monitoring session improved substantially. There can be a complementary relationship between routine and short-term monitoring. For example, problems that have been identified through routine monitoring can lead to a special intervention to solve those problems. A short-term monitoring system might then be set up temporarily to make sure that the intervention is effective. Once that has been achieved, short-term monitoring may be reduced, eliminated, or partially incorporated into the routine monitoring system. Keep it straightforward and simple (KISS) In designing a routine or short-term monitoring system, a few guidelines should be considered: 1) select only key indicators that will be used by managers and other key users; 2) do not overburden staff by collecting too much data; 3) provide feedback in a timely manner; 4) use a reporting format that facilitates data interpretation and action. Limitations of service monitoring Some important PHC activities may be difficult to monitor. For example, it is difficult to collect data about the content and effectiveness of services provided by CHWs at the household level. Collecting information for some indicators, such as changes in infant and maternal mortality, may not be feasible, especially over a short period of time. For this reason, we suggest focusing activity monitoring on input, output, and effect indicators to determine whether a PHC service or management service is being carried out as planned. Community surveys (see Module 2) can also be used periodically to assess the effects and impact of these services on knowledge, behaviour, and health status. A surveillance system (see Module 4) can be utilised to track mortality, morbidity, disability, and fertility, and to investigate their causes. Although the indicators suggested in the appendices include some of the most common and useful indicators currently used in PHC, it is expected that you will modify them to meet the specific needs of your programme. Monitoring procedures Nine steps to activity monitoring are listed below and are described in detail later in this guide. The first four cover the planning stage and seek to ensure that the information collected is relevant, specific, feasible to collect, and can be analysed. The next four cover the implementation of monitoring: collecting information, compiling and analysing it, reporting, and using the results for management action. The ninth step suggests taking a look at the system periodically to decide whether to continue monitoring activities at the current level. Steps in activity monitoring Planning Step 1: Specify the monitoring objectives Step 2: Decide on the scope of the monitoring Step 3: Select the indicators and performance standards Step 4: Choose information sources and develop data collection procedures Implementation Step 5: Collect the data Step 6: Tabulate and analyse the data Step 7: Present the findings Step 8: Take appropriate action Assessment Step 9: Decide whether to continue monitoring These steps assume that the user will normally want to select a limited number of indicators to monitor one or more PHC services for a short period of time. However, the same process would be followed by those who might want to monitor all PHC services on a routine basis. Those who already have an MIS may find it useful to review the steps to determine if improvements can be made in their system, for example, by reducing the number and frequency of collection of indicators or by streamlining the current system. Step 1: Specify the monitoring objectives The first step in designing a monitoring system, no matter how small, is to determine what services are going to be monitored, for what purpose, and for whom. Different users will have different information needs and purposes. For example, a donor or board member may only be interested in monitoring programme coverage on a routine basis to determine whether to continue funding. On the other hand, a manager may want information about service inputs and outputs, largely to make sure that the PHC services are being carried out as planned, and to make adjustments if they are not. The following worksheet is used to clarify: 1) what services will be monitored, e.g., immunization, logistics; 2) for what purpose, e.g., routine or short-term information; decisions about continuing or modifying the service; and 3) who will use the monitoring results, e.g., manager, community, or health workers. @BLOB = Activities If you have completed Module 1, you have already set priorities among your monitoring information needs. You should start by reviewing the PHC and management services that you selected. If you have not completed Module 1, find the lists of the most common PHC services and management services (both in Module 1 and the PRICOR Thesaurus), and use these lists to decide which services to monitor. Appendices A, B, and C of this module can be used as references, as well. In Modules 6 and 7, each PHC service and management support service is broken down into major activities and tasks. These lists might be of use to those who are interested in monitoring only a specific part of a service. After you have reviewed these modules, rank each PHC service and management service on a scale of 1 to 10 (highest) to determine what should be monitored. Next, determine if you are going to monitor input, output, or effect indicators for these PHC and management services. Purpose In specifying your purposes, you should clarify why you need information in each area. Many monitoring systems merely produce data for general information, often because it is "required," nothing more. It is quite possible that the user(s) only want to know what the PHC services are doing (e.g., how many people were served this month, what were the major illnesses reported). If this is the intended use of the data, then data collection may be quite simple. Only numerator data may be needed. For example: 45 children vaccinated; 13 CHWs trained. In other cases the user may want to know whether a service is being performed as planned. A judgement about the effectiveness of that service can be made based on a comparison between what was planned and what actually happened, such as in the case of special monitoring. This will require collecting numerator and denominator data, for example: 45 of 60 eligible children vaccinated (75%); 13 of 15 CHWs trained (87%). Finally, if specific decisions are to be made based on the results, then "decision rules" should be determined at the outset so that the user will be assured of getting enough data to make the decision, for example: close the health centre if attendance falls below 100 visits per month and revenues are less than Rp 3,000. Users Identifying the users is important and worth checking to make sure that there are no misunderstandings and that the information collected will be of real use. The users should be involved in the selection of the indicators, and must be able to explain how the information will be used. If there are multiple users, their various needs may have to be negotiated. The following worksheet may help you summarise the monitoring objectives of each user. WORKSHEET FOR SPECIFYING THE MONITORING OBJECTIVES What to monitor, Purpose (routine,, R, or short-term,, S), Internal users, External users, ^, ^, Providers, Managers, Donors, Others PHC services, , , , , EPI (effects), measure coverage of immunization, Nurses Physicians, Clinic, AKF, MOH ORT (outputs), determine no. clients served (s), Physicians Director, Clinic, CIDA, ANC (inputs), determine availability of TT (r), Nurses Director, Clinic, USAID, MOH Management, , , , , Training (effects), determine % of staff trained (s), Trainers Director, Programme, CIDA, Planning (effects), determine plans produced/distributed (s), Planners, Director of planning, UNFPA, BKKBN Logistics (inputs), measure purchases of FP methods (r), Purchaser Director, Logistics, USAID, Step 2: Decide on the scope of the monitoring After specifying the purpose of monitoring, managers need to determine how broad the monitoring should be: what geographic area will be included in the monitoring (the entire region, a sub-district); which facilities or sub-projects are to be included; which staff or workers will be included in selected facilities; and how long monitoring will continue. WORKSHEET FOR SPECIFYING THE SCOPE OF MONITORING, 1., What geographic area will be covered? Kisumu and Mombasa Districts 2., Which facilities or sub-projects will be monitored? Clinical services 3., Which personnel (managers,, providers & volunteers) will be selected? Physicians and nurses 4., How long will the monitoring continue? Six months It may not be necessary to monitor all of the service facilities or staff in a programme area. For routine monitoring, it may be a good idea to test the new monitoring procedures in a few areas before instituting them throughout the project area. For short-term monitoring, it may be sufficient to limit the monitoring to: a) low performance facilities, sub-projects, or staff, or b) facilities or staff from each geographic area. Just as important as the scope of the monitoring is the duration of data collection and analysis. The duration will usually depend on whether the monitoring is routine or short-term. Routine monitoring procedures usually continue for an indefinite period, or at least until the user(s) determine that the data are no longer needed. Short-term monitoring, as the name implies, is more likely to be used for evaluation and decision making and to have a limited duration. Duration would be determined by the deadlines the user(s) have (or set) for making a judgement or decision about the activity being monitored. For example, if a new case-finding procedure is being tested, the manager may want to know whether it is effective before expanding it to the whole programme. The manager might set a three- or six-month test period, after which a decision can be made to continue testing, expand the procedure, or drop it. By setting a specific time frame for short-term monitoring, the project can avoid continuing data collection beyond its utility. Step 3: Select the indicators and performance standards This module emphasises input, output and effect indicators. Appendices A and B provide extensive lists of broad and narrow indicators for each PHC service and management service. The user should examine these lists and choose a limited number of indicators for inclusion in the monitoring system. It is expected that most programmes will need to modify the indicators presented in this module. Field tests have shown that the most typical modifications are in: 1) the definition of the target group (for example, children under age 2, under age 4, under age 5, between 12 and 60 months); 2) terminology (for example, "diarrhoea" may be defined differently in different cultures); and 3) phrasing, to make the indicator culturally acceptable. Although the indicators listed in Appendices A and B are thought to include most of those that PHC managers will need, there may be additional indicators that some programmes will need to develop. The worksheet on the next page can be used to specify PHC service and management indicators, formulate the indicators, set performance standards, and determine the frequency of collection. Formulation Indicators can be formulated in the following ways: Count: The simplest type of measure; just count the number of events or objects. For example, 56 visits, 432 CHWs, 9,765 children immunized. Rate: A measure of the frequency with which some event occurs, such as household visits/day. Ratio: Two numbers related to each other in a fraction or decimal, such as the number of ANC visits made per pregnant woman (3.2:1) or the number of trained TBAs per population (1:490). Proportion: A special type of ratio expressing a relationship between a part and the whole. For example, the 3,250 children immunized out of 5,000 (3,250/5,000 = 0.65). The numerator is the portion of the total, the denominator is the total. Percentage: A proportion multiplied by 100. For example, (3,250/5,000) * 100 = 65 percent of eligible children immunized. WORKSHEET FOR SELECTING INDICATORS AND STANDARDS, +, +, + Indicators, Formulation, Standards, Frequency PHC services, +, +, + Effect, % of households using latrines, 50%, annually Output, No. of mothers who can interpret GM card, 600, 6 months Input, % of training sessions with materials, 80%, monthly Management services, +, +, + Effect, % of CHWs with improved performance, 30%, quarterly Output, No. of training sessions with clinic staff, 20, monthly Input, No. of FP methods received from donors, 20,000 CYPs, quarterly Performance standards: In addition to selecting the indicators, the manager should set performance standards for each indicator. This step is often overlooked, but it is very important, especially for where the objective is to make an evaluative judgement or decision. If performance standards are not set at the outset, then it will be impossible to determine whether the activity has been successfully carried out or has been effective. Use the above worksheet to define standards for each indicator. A performance standard is often called a "target" and is usually quantified. For example, suppose that the activity to be monitored is growth monitoring and three indicators are selected: number of GM sessions held, number of children weighed, and percentages of mothers counselled. "Targets" need to be set for each of the indicators. How many sessions should be held during the monitoring period; how many children should be weighed; what proportion of mothers should be counselled? Only if these "performance standards" are set will it be possible to reach agreement on whether performance is acceptable. These "standards" are the denominators in the computation of performance. An indicator involves a numerator (what was achieved or the actual performance) and a denominator (the target or planned performance). For example: No. GM sessions actually held = 20 * 100 = 80% No. GM sessions planned to be held = 25 In immunization coverage, a common objective is to fully immunize 80 percent of the target population. That "standard" is the denominator. The numerator would be the actual proportion of eligible children immunized, say 65 percent. The performance would be: No. of eligible children immunized 3,250 * 100 = 81% No. of eligible children to be immunized 5,000 * .80 The result shows that the programme met 81 percent of its target. It also shows that actual coverage was 65 percent (3,250/5,000), or 15 percentage points below the target. Frequency of data collection: Information for some indicators is easier to collect than for others and frequency of collection should be kept to a minimum so as not to overburden staff and raise costs. Frequency is important to consider in planning because each round of data collection involves effort that may reduce the amount of time available for services and other important management services. If it is important to know exactly how many times an event happened, e.g., how many children were immunized, how many latrines were built, then the data would have to be collected continuously. But if the objective is to determine if a procedure is being carried out as expected, e.g., are CHWs counselling mothers properly, are there enough ORT sachets on hand, then the data can be collected at periodic intervals. However, data collection and examination of results do not have to occur simultaneously. The users should also determine how often they want to examine the results. For some users, an annual or semi-annual assessment may be enough. Others may want weekly reports. There are no hard and fast rules about frequency. It depends on the user's needs. Step 4: Choose information sources and develop data collection procedures For each indicator, determine the source and the procedure for collecting monitoring data. For many indicators, more than one source could provide the necessary data. The following worksheet can be used to specify sources and techniques for collecting data for each indicator. Techniques for monitoring PHC services In addition to routine records, several other techniques are available for gathering data for monitoring purposes. Observation Direct observation at the site of service delivery offers the opportunity to collect information on health worker performance and on the presence of inputs. For many indicators, this is the most reliable source of information, although it also requires a caveat: people tend to improve their performance when they are being watched, and thus, the experience may not be representative of what routinely happens. Observation data can be collected during supervisory visits, home visits, or household surveys. Rapid surveys Rapid surveys provide information on the population of users and non-users. It is particularly useful for information on coverage and on home treatments, as well as mothers' knowledge. Module 2 presents a whole series of rapid household surveys. These models can be used to carry out comprehensive surveys, or a few questions could be extracted on a specific topic and used in a small, quick survey. Exit interviews Exit interviews involve asking questions of patients as they are leaving the clinic or session. This type of interview offers the opportunity to collect information on what the health workers did and how well the patient/mother was able to absorb the information given. This information source can be incorporated into routine supervisory visits as well as carrying it out as an independent activity. Interviews with health workers Much information can be gained just by talking to health workers, whether in structured individual interviews, informal conversation, or focus group discussions. It offers a quick, inexpensive way to get information on such things as problems with inputs, knowledge about treatments, and identification of high-risk patients. This source offers sensitive information; if a problem is identified this way, it is most likely a real problem. However, if no problems are identified, that does not mean they do not exist. The information generated from this information source may require verification through routine records or direct observation. The choice of information sources will depend on the manager's existing information system. As such, most of the data needed for monitoring will probably come from existing sources. Routine records include the information sources that make up the health information system, such as logistics records, treatment registers, individual patient records, activity reports, and population-based records, including family files. The form of these routine records will vary from programme to programme, but the first place to look for information is among what is currently being collected. Many input indicators could be derived from the routine record system or by making a few modifications of routine records so that they produce the specific information desired. Some process/output indicators could also be calculated from routine records. For example, reviewing treatment records or the treatment register could provide information for the indicator: % of health workers using antibiotics only in cases of pneumonia, strep throat, and otitis. If the needed data are not already collected, it is often possible to modify one of the current forms so that they will provide the desired information. This may require nothing more than a slight change in the wording of the current form. For example, change "number of ORS packets distributed" to "number of mothers of children under age 5 given ORS packets." In other cases, a new item could be added to an existing form. Example: add a question to the CHW's home visit register to ask mothers, "Do you have a growth monitoring card for your child?" Or instruct supervisors to ask CHWs a few key questions during their routine visits, for example: "Have you been trained this past month how to counsel mothers about side effects from immunizations?" In some cases, it may be necessary to develop new instruments to collect the data. Modules 2, 6, and 7 provide guidelines, examples, and instruments that can be adapted to fit monitoring needs. They describe the most common data collection approaches: how to collect new data from surveys of mothers, from observations of PHC services, and from reviews of available records and reports. Informal interviews and discussions with health workers, administrative staff, community leaders, and others may be a practical way to collect general impressions of how the programme is going, if more formal and specific data collection procedures cannot be afforded. Time and financial constraints must be taken into account when choosing where and how to obtain monitoring data. WORKSHEET TO SELECT DATA SOURCES AND TECHNIQUES, +, +, +, +, + Data source:, Records, Provider client interface, Client, Health worker Data collection technique:, +, Review, Observation, Survey/Interview, Interview PHC services indicators, +, , , , % HH use latrines, +, , , rapid survey, No. mothers who can interpret GM card, +, , , rapid survey, % of training sessions with materials, +, service data, , , Management services indicators, +, , , , % of CHWs with improved performance, +, supervision records, , , No. of training sessions with clinic staff, +, service records, , No. of FP methods received from donors, +, logistic procurement records Step 5: Collect the data When new data monitoring forms or instruments are developed, the user should first pre-test them on a small scale. This includes the selection and training of data collection personnel and the establishment of procedures for data collection supervision and processing. Module 2, in particular, includes some guidelines on data collection, including pretesting, training of interviewers, and supervision. Module 4 presents guidelines for establishing an impact surveillance system. Modules 6 and 7 also include relevant suggestions for collection of input and process data. The following worksheet can be used to specify responsibilities for data collection, supervision, and processing. Again, the easiest approach would be to collect the needed data through the existing system. This would eliminate the need to set up a special data collection effort. CHWs, supervisors, physicians, and others could easily collect a small amount of additional information, especially if it is only for a limited period of time. However, it will probably be necessary to carry out at least minimal training to ensure that workers understand the changes in the existing system and its purpose. WORKSHEET FOR SPECIFYING DATA COLLECTION, SUPERVISION AND PROCESSING, +, +, + Staff, Responsibility, +, + Collection, Supervision, Processing Health Worker, Collect, , Send to district Clinic Staff, Collect, Supervise health worker, Send to district District Manager, Receive, Use data to manage, Compile MIS Manager, Receive, , Process and report Step 6: Tabulate and analyse the data The monitoring data should be tabulated and analysed to meet the objectives and scope specified in steps 1 and 2. Step 3 showed how to compute the results by comparing actual with planned achievement. The following worksheet can be used to specify the actual performance observed, the performance standard or target, and the percent of the standard achieved. Most tabulation will be straightforward counts and/or percentages. There are several ways in which data analysis can facilitate interpretation of results. You can identify problems by examining frequency distributions (Table 3), by comparing the actual values of the indicators to the performance standards or targets (Figure 2); by comparing performance between health units or administrative districts (see Figure 4); and by comparing administrative units over time (see Table 4). The data can be displayed in various ways. The most common are lists, tables, and graphs. Table 3 shows the number and percentage breakdown of selected pregnancy-related indicators for a six-month period. The following examples are taken from a semi-annual report of a PHC programme in Bangladesh. From the "Aga Khan Community Health Programme, Dhaka, Bangladesh. Progress Report, April 1990 - September 1990." Figure 2 shows data in graphic form compared to standards or targets set for the same periods. The next example (Figure 4), shows how data from different health centres can be compared. These data are of Vitamin A-capsule coverage in 14 disaster-prone areas in Bangladesh. Data can be presented in a tabular form to compare results over time by administrative areas. The final example (Table 4) shows the number of pills distributed each month for eight districts. Overall, month of July 1989 had the highest level of pill distribution. Among the eight districts, the Canca district had the highest overall level of pill distribution. Figure 2: Eligible children fully immunized: Sept 1989-Sept 1990 Table 3: Pregnancy-related indicators: April-September 1990 Figure 4: Vitamin A - capsule coverage August, 1990 Step 7: Present the findings The results of activity monitoring should be presented to those involved in service implementation: supervisors, health staff, community members. You should review the "Worksheet for specifying objectives" in Step 1 to determine if the users, in fact, need the information. Although results can be presented graphically or in tables, the presentations should be adapted to the level of the users. They should be simple and point out major findings. The participants should be allowed to contribute their viewpoints on potential causes of and solutions to problems identified. Meetings with the users of information can promote the sharing of perspectives among the different PHC partners. Step 8: Take appropriate action When the monitoring results have been presented and discussed among the various PHC partners, you must use your findings to decide whether action needs to be taken, and if so, which action. This means that you must first explore causes and then develop solutions. In many cases, the causes may be obvious or well-known to the staff. In Thailand, for example, health workers did not counsel mothers during growth monitoring sessions because the sessions were too noisy and there was not enough time. The checklists in Modules 6 and 7 can be helpful guides for exploring the causes of problems. For example, if immunization coverage is low, the immunization checklist could be used to examine the service delivery process to determine whether: health workers are contacting and following up eligible households; mothers are coming to immunization sessions; children are being immunized properly and mothers counselled about possible side-effects; and if vaccines, syringes, and other supplies are in adequate supply. When the cause of the problem and/or the most likely solution are not known, the PHC MAP Problem-solving guide and the PRICOR Operations Research manuals can help managers work their way through this problem-solving process. Once a solution has been agreed upon, its implementation must be planned. This involves setting objectives and determining what activities must take place, who should carry them out, where the services should be provided, when the services should be offered, and what resources are required. The action decided upon may itself require additional short-term monitoring to make sure that: a) the interventions selected are implemented as planned; and b) they have the desired effects. Step 9: Decide whether to continue monitoring Activity monitoring is designed to meet the current needs of managers and other users. As those needs change, so these should monitoring requirements. As programmes improve over time, managers may have less need for frequent, detailed information on some inputs and outputs. Effect indicators usually take on more importance as input and output indicators show improved implementation. The number of indicators can ordinarily be reduced when sustained, satisfactory performance has been achieved. Periodic review of all MIS indicators can help managers keep their MIS streamlined by reducing the number of indicators, reducing the frequency of collection of data, and reducing the aggregation and reporting of data to higher levels. Appendix A: PHC service indicators This appendix presents an annotated list of selected indicators for the following PHC services: GENERAL PHC General PHC household visit Health education MATERNAL CARE Antenatal care Safe delivery Postnatal care Family planning CHILD CARE Acute respiratory infections Breast feeding Diarrhoeal disease control/oral rehydration therapy Childhood disabilities Child immunization Growth monitoring/nutrition education COMMUNITY HEALTH Water supply, hygiene, and sanitation OTHER HEALTH CARE Accidents and injuries Chronic, non-communicable diseases: Malaria Treatment of minor ailments (general curative care) Tuberculosis Sexually transmitted diseases, HIV/AIDS The indicators for each PHC service are divided into three categories: effects, outputs, and inputs. In general, these categories include indicators that cover one or more of the generic criteria described in the text. Examples are shown below. Effect indicators: coverage number or percent of children fully immunized behaviour number or percent of households that use a sanitary latrine knowledge number or percent of TB patients who know why it is important to complete the treatment for tuberculosis skills number or percent of mothers who can correctly administer ORS Output indicators: utilisation number or percent of children under two years enrolled in growth monitoring quality of care number or percent of health workers using sterile needles and syringes for each immunization injection contact/visit number or percent of pregnant women contacted/visited by CHWs access number or percent of population living within five kms of health facility Input indicators: personnel number of trained TBAs; population per active CHW supplies number or percent of health centres without sufficient ORS packets equipment number or percent of health units without functioning laboratory equipment Although these indicators represent practical and expert recommendations, they may not be universally applicable and may need to be adapted to fit local circumstances. For some indicators, local standards for treatment, training of staff, and equipment must be reviewed and incorporated into your data-gathering instrument in order to measure PHC inputs, outputs, and effects. For other indicators which require the measurement of knowledge and skills, the user may need to review the content of PHC IEC and training efforts to determine key knowledge areas. Footnotes provide suggestions for data sources and, in cases where it is not obvious, explain what the indicator is designed to measure and why it is important. In some cases, more than one indicator is suggested for a similar area. As stated in the introduction, the target group and period of observation need to be specified for many of these indicators. Those given are illustrative only. The term "health units" applies to health centres, health posts, or community-level health services. This term should be interpreted to fit each programme's health system. Most of the indicators are designed to be simple counts of activities or simple percentages. In many cases counts will be sufficient, since the objective will often be to determine whether a target has been met. Percentages are recommended where possible, and ideally, both counts and percentages would be calculated. GENERAL PHC HOUSEHOLD VISIT Output indicators number or percentage of households visited by CHW in last three months number or percentage of PHC household visits during the last three months where the health worker discussed:1 <_>- growth monitoring/nutrition (if malnourished child) <_>- antenatal care (if pregnant woman) <_>- family planning (if woman of child-bearing age) <_>- water and sanitation <_>- immunizations <_>- oral rehydration therapy (if diarrhoea case) number or percentage of the target group visited by their CHW in the last three months by type:2 <_>- women 15-49 years of age <_>- children under 5 years of age <_>- high-risk cases Input indicators number of households per CHW3 population per CHW2 number of active health workers by type: <_>- CHW <_>- public health nurse <_>- midwife Notes on the general PHC household visit indicators 1 Care should be taken in analysing and interpreting results for this indicator, since not all households visited would require all the messages listed. The denominators should be adjusted to reflect this. 2 Information could be obtained from home visit forms, if they exist, or from rapid surveys (by modifying the PHC rapid survey form). 3 Information for this indicator can be derived from population/census information. See Appendix G of Module 2 for estimations of the target population. HEALTH EDUCATION Effect indicators number or percentage of respondents who practise health behaviour outlined in the health education objectives1 number or percentage of the target population who remember health education messages on mass media, television, radio, billboards, posters, etc., during the last 1-2 weeks2 number or percentage of respondents who know the importance of the health education topic discussed by CHW during the last visit3 Output indicators number of the target population that was visited at home and received health information during the last three months4 number or percentage of health workers using 1 or more health education techniques (role playing, demonstration, flip charts, and others used in the local area) number or percentage of clients receiving health information in a community or group setting5 Input indicators number or percentage of CHWs trained in health education6 number of community organisations which provide health education services number of CHWs compared with 1,000 households or each village7 number or percentage of respondents that wish to receive additional health education information/topics8 Notes on health education indicators 1 Review the local norms for positive health behaviour, as explained in health education materials. Survey respondents to determine if they practise each type of behaviour. 2 Respondents can be asked about specific messages from various sources of mass media. 3 This indicator is for households that were visited during the last 3 months (see Module 2). 4 Target groups may include women 15-49 years, children under age 5, high-risk cases (tuberculosis, malaria, STD/HIV), and others. 5 Module 2 Assessing community health needs and coverage, provides interview questions for measuring health education among reproductive age women, children under 24 months, children 24-60 months, and other household members. 6 Health workers include CHWs, public health nurses, midwives, physicians and others. 7 The ratio of CHWs to households or number per village should be defined in the local context and in accordance with local needs. 8 This indicator measures the demand among the population for more information. ANTENATAL CARE Effect indicators number or percentage of pregnant women identified that are "high-risk"1 number or percentage of women who made three or more prenatal visits during their last pregnancy2 number or percentage of women who received two doses of tetanus toxoid to confer protection prior to delivery3 number or percentage of women who complied with iron folate supplementation regime during last pregnancy4 number or percentage of women gaining less than 1 kg/month during the second and third trimester Output indicators number or percentage of women who received at least one antenatal visit while they were pregnant average number of contacts per pregnant women5 number or percentage of pregnant women seen in antenatal care who were counselled about danger signs indicating the need to seek further care number or percentage of workers who regularly track high-risk pregnancies6 number or percentage of high-risk women seen by a health worker who were identified as such7 number or percentage of health workers providing medical attention to high-risk pregnant women or referring them8 Input indicators number or percentage of health units which experienced stock shortages of iron supplements/malaria prophylaxis9 number of days when iron supplements/malaria prophylaxis were out of stock in the clinic number or percentage of women of reproductive age weighing less than 38 kg before pregnancy10 Notes on antenatal care indicators 1 High risk includes: maternal age less than 16 or over 35, first pregnancy over 30 years of age, 5-8 past pregnancies, over 10 years since last pregnancy, previous caesarean section, previous delivery complications, previous still birth, 2 or more previous miscarriages, previous neonatal death, 3 or more abortions, 2 or more infant deaths, previous low birth weight baby, maternal height less than local standard, small pelvic outlet, maternal limp/polio leg, bleeding since last period, clinically anaemic, fever, blood pressure greater than 140/90, sputum AFB positive, diabetes, heart disease, pre-eclampsia, abnormal foetal presentation, sickle cell, malaria, AIDS, breech presentation or transverse lie, large for date pregnancy, suspected twins. 2 Information for these indicators can be obtained from rapid or mini surveys of women having delivered within the last 12 or 24 months, depending on local concerns. The norm for the number of visits should be adapted to local policy. 3 This indicator shows how well women complete the necessary tetanus toxoid injections during their last pregnancy. The numerator is the number of women that receive the full coverage; the denominator is all ever-pregnant women. 4 This information can be gathered in a survey by asking if iron pills were taken during pregnancy. 5 This indicator shows how often those women using antenatal care seek it. The numerator would be total antenatal visits, and the denominator would be the number of women making at least one antenatal visit. 6 "High-risk" will need to be defined locally. Information can be obtained from supervisory checklists. 7 This indicator measures how well the high-risk system works. Information on high-risk pregnancies could be compiled from antenatal cards or other individual service records kept in the health facility which record high-risk factors. 8 Whether health workers need to refer, or can provide advice without referral, will depend on the educational and training level of health workers and the sophistication of the health facility. 9 Both indicators point out problems in the supply system, although the second provides a more detailed picture of the extent of the problem. Data can be obtained from supervisory checklists for antenatal care (interviews with health workers or review of stock records and inventories). 10 This is a measure of the risk of pregnancy. This information can be gathered by survey or review of records. SAFE DELIVERY Effect indicators1 number or percentage of deliveries in preferred locations (e.g., hospital, maternity clinic, health clinic, midwifery or birthing centre)2 number or percentage of births attended by trained health provider (physician, nurse, midwife, CHW, TBA)3 number or percentage of mothers with knowledge of danger signs and where to go if complications arise (danger signs include malaria, diabetes, hypertension, liver disease, and others) number or percentage of families with members (men, women, mothers-in-law) aware of danger signs of pregnancy, labour, delivery, and puerperium ratio of positively treated obstetrical complications to all complications during the last 3-6 months4 percentage of women with optimum weight gain (i.e., no more than 13 kg and no less than 6 kg from pre-pregnancy to childbirth)5 Output indicators1 number or percentage of pregnant women who were trained about the danger signs of delivery and instructed where to go6 number or percentage of obstetrical complication cases treated7 Input indicators8 number or percentage of TBA trained in family planning, recognition of obstetrical complications, and hygienic birthing practices, and linked with the formal health service delivery system9 number or percentage of district hospitals equipped and functioning as first referral centres10 number or percentage of facilities and staff using standardised referral protocols to manage obstetrical complications11 number or percentage of health cadres and staff trained in care of obstetrical complications, especially emergency cases12 number or percentage of communities with organised transport systems in place to effect referral13 Notes on safe delivery indicators 1 Most of the effect and output indicators can be obtained from survey interviews of women having delivered during the last 12-24 months. 2 This indicator measures women's practice in using preferred facilities. The indicator is only relevant if the women have reasonable access. This indicator could be used to focus only on mothers from the low socio-economic status. 3 The indicator is only relevant if the woman has reasonable access to a trained provider. The numerator is the number using a trained provider; the denominator is the number of women with access. 4 Positive treatments refer to complications which are successfully treated versus unsuccessful treatments which result in maternal mortality or chronic morbidity. This information can be gathered from a review of records in obstetrical care facilities. If the information is gathered directly from patients, a large sample of women will be required to collect data from those with complications. 5 This indicator can be measured by using hospital records and surveys. 6 This information can be gathered from a survey of clients or routine service records of MCH facilities or providers. 7 This indicator requires a review of obstetrical care facility records. 8 Information on input indicators can be obtained from providers and/or a review of service records. 9>This indicator is only appropriate where TBAs provide a large proportion of deliveries and their capability is of major concern. 10 Periodic service records of health facilities usually provide information about the types of services provided, i.e., obstetrical referral services, and equipment available, i.e., to manage obstetrical complications. 11 To measure this indicator, first determine if standardised referral protocols exist and what they are. Next, providers and clients can be surveyed to determine if the protocols are known by the providers and followed. 12 To gather this information, determine the types of complications handled by various types of health staff. Then survey health cadres and staff to determine if they have been trained (their knowledge can also be measured). Results of this indicator permit an evaluation of the obstetrical complication capability of various types of health staff and facilities. 13 This indicator measures clients' access to emergency obstetrical services and is obtained by surveying obstetrical care providers or facilities. POSTNATAL CARE Effect indicators number or percentage of women receiving postnatal care from health workers number or percentage of postnatal women who return for follow-up visits number or percent of women who have delivered and know when and where to return for a postnatal follow-up visit Output indicators number or percentage of women who have delivered and were seen at least once during the postnatal period1 number or percentage of health workers counselling mothers on potential danger signs in postpartum period requiring consultation with health worker number or percentage of health workers using sterile materials for cutting and bandaging the umbilical cord, OR number or percentage of mothers who said that health worker used clean materials for cutting and bandaging umbilical cord2 number or percentage of mothers delivered by a trained birth attendant who received counselling on child services after delivery Input indicators number or percentage of clinics/local health workers experiencing shortages of sterile equipment and supplies for appropriate care of the umbilical cord (razor blade, bandages, etc.) number or percentage of facilities with one or more health staff trained in postnatal care and counselling3 Notes on postnatal care indicators 1 This indicator measures whether women are receiving postnatal follow-up. This information can be obtained from rapid or mini surveys. 2 When sterile or clean materials are used, the incidence of postnatal infections is reduced. Information for the first indicator can be obtained from supervisory checklists for deliveries or from exit interviews with mothers. Although the latter may be less reliable, the difficulties of observing delivery practice in many settings may make this the only source of data available. 3 The type of person classified as a "trained birth attendant" will depend on local programme norms. This could be a re-trained traditional birth attendant, a public health nurse, or a midwife. FAMILY PLANNING Effect indicators1 number or percentage of eligible women knowing at least one modern family planning method and where to obtain it2 number or percentage of women of child-bearing age currently using modern family planning methods3 number or percentage of last pregnancies not intended4 average length of time current contraceptors of modern methods have used the method5 number or percentage of births, with less than 24 months' spacing, among younger women 15-29 years6 ratio of births to women below 19 or above 34 years7 Output indicators number or percentage of eligible women contacted by health worker, for outreach8 number of women receiving methods from CHW, by contraceptive method number of new acceptors by method, particularly longer acting methods9 number or percentage of women seen who were referred from other providers for clinic-based services10 number or percentage of health workers who explain where and when to go for routine follow-up for family planning services11 number or percentage of acceptors followed up12 number or percentage of women who received counselling on possible side-effects of the contraceptive chosen, or who were asked about side-effects on follow-up visits Input indicators number or percentage of health units experiencing stock shortages of each contraceptive method in the last month13 number or percentage of health units who had less than two types of contraceptives in stock at anytime during the month14 number or percentage of villages without a regular supplier of contraceptives15 Notes on family planning indicators 1 The effect and output indicators can be measured by surveying eligible women or couples. The woman should live in a conjugal relationship and still be at-risk to pregnancy. 2 Modern methods include pills, injectables, IUDs, implants, sterilisation, and condoms. 3 This indicator measures the coverage of contraceptive use among reproductive-age women, usually 15-49 years. 4 This indicator measures unwanted pregnancies resulting in unwanted births or abortions. It provides a measure of the unmet need for family planning services. The numerator is unwanted last pregnancies; the denominator is the number of women reporting one or more pregnancies. 5 This measure is obtained for each contraceptive method and indicates the continuity of effective use. An alternative indicator is to determine if the current method has been used continually for the last 6 or 12 months. The numerator is current contraceptors continually using; the denominator is all current users (by method). 6 This indicator measures the spacing of children among young women to improve both the health of mother and baby. Information can be gathered from a survey of mothers. 7 Women 15-19 and 34+ years should have substantially lower fertility levels. As women delay marriage and older women limit additional pregnancies, fertility should be concentrated in the ages 20-34 years. This indicator is appropriate for programmes which are targeting young women either to delay marriage or to space births, and older women to use long-term or permanent contraceptive methods. 8 This indicator measures the level of contact that family planning services have with their target population. This information can be obtained from rapid or mini surveys. 9 IUD, sterilisation, and implant 10 This indicator measures how well women using other health services are being encouraged to use family planning as well. It is a measure of service integration. This information could be obtained by modifying the service records (to show if the women were referred and by whom) or through exit interviews with clients. 11 Information can be obtained from supervisory checklists using observation of health worker-patient encounters. 12 This indicator provides more in-depth information about how well family planning clients are being followed up to ensure continuation. Information could be obtained by compiling information from individual service records to see whether patients are returning or other actions are being taken to ensure they return. 13 This indicator can point out problems in the supply system. Data for each method can be obtained from supervisory checklists for family planning; interviews with health workers or review of stock records and inventories. If monthly stock levels have been established for each method, another indicator can be developed to measure low stocks and the risk of shortages. 14 Some programmes may want to use this indicator which addresses the issue of contraceptive choice in family planning programmes. This information can be obtained from supervisory checklists. For facilities that have a wider range of methods (5-6) the minimal number can be raised to, for example, three methods. 15 Information for this indicator presupposes knowledge of the villages in the catchment area and their locations. Information about specific availability can come from rapid/mini-surveys or from logistical reports and mapping of the area. ACUTE RESPIRATORY INFECTIONS Effect indicators number or percentage of ARI cases (cough, fever, difficulty breathing) treated by health worker correctly1 number or percentage of ARI patients or their caretakers who know the correct dosage and duration of treatment for ARI average number of serious symptoms associated with ARI recognised by mothers with children under age five2 number or percentage of mothers who know the warning signs of ARI and when to seek care3 number or percentage of pneumonia cases receiving standard case management at health facilities4 number of ARI hospitalisations per year among children under age five Output indicators number or percentage of women with children under age five that were informed of serious ARI symptoms by CHW or other health provider number or percentage of health workers using antibiotics only in cases of pneumonia, strep throat, and otitis (per local policy)5 Input indicators number or percentage of health units which experienced stock shortages of antibiotics in the last month6 number or percentage of health units/health workers without functioning timepiece with second hand (to count respirations) number or percentage of health workers or facilities with health care workers trained in ARI case management number or percentage of health facilities that have an ARI treatment chart on the wall number of percentage of health facilities with national treatment guidelines Notes on treatment of acute respiratory infections indicators 1 This indicator measures the level of coverage for ARI treatments by health workers. Information for this indicator can be obtained only from rapid surveys. Some care should be taken in interpreting this indicator since the denominator would be self-diagnosed ARI based on a history of cough, fever, and difficult breathing. The exact criteria to be used to determine what is considered an ARI case will need to be defined locally. 2 ARI symptoms include high fever, cough with sputum, rapid breathing, difficult breathing, determined by chest indrawing or retracting, inability to drink, earache or discharge, seizures, weakness or lethargy. 3 Mothers should seek care before the symptoms become serious. Symptoms of moderate ARI include; a moist cough (often frequent), and frequent or difficult breathing, sometimes accompanied with wheezing or other sound. Moderate ARI is often accompanied by a fever and weakness (lethargy), but these symptoms are associated with a variety of illnesses not just ARI. 4 This indicator measures the population that has access to standard ARI case management through a health worker in a facility who is trained in standard ARI case management with a source of free or affordable antibiotics. As ARI case management requires the administration of antibiotics, emphasis remains on access to providers within the health system. 5 The denominator in this indicator is health workers prescribing antibiotics, and the numerator is those health workers prescribing antibiotics for pneumonia, strep throat, or otitis. 6 This indicator can point out problems in the supply system. A second, more detailed indicator could be "the percentage of health units with stock-outs of one day or more during the last period." Data for both can be obtained from supervisory checklists for ARI; interviews with health workers or review of stock records and inventories. BREAST FEEDING Effect indicators number or percentage of mothers breast feeding babies up to 12 (18) months of age number or percentage of mothers who began breast feeding within 24 hours after birth1 number or percentage of mothers who gave the baby colostrum (local word) number or percentage of mothers who know why it is important to give colostrum2 number or percentage of mothers who breast fed and did not provide food supplements during the first 4 months3 number or percentage of mothers starting to give supplemental foods (water, other liquids, solid foods) to infants 4-6 months of age4 proportion of infants 6-9 months of age who received both breast milk and complementary foods number or percentage of mothers who continued to breast feed during the last case of diarrhoea5 number or percentage of women with children under age two who know how long to continue breast feeding number or percentage of currently breast feeding women who know how to position the child and care for her breasts6 number or percentage of currently breast feeding who know what to eat during the lactation period7 Output indicators number or percentage of women who were informed during their pregnancy by a health provider about the value of breast feeding and when to start number or percentage of postnatal women with children under age two who received breast feeding brochures, pamphlets, or other educational materials number or percentage of mothers who received information about breast feeding during the neonatal period; 28 days after birth Input indicators number of workers who have been trained in breast feeding education number of CHWs or other health providers with correct knowledge about the benefits of breast feeding and when to start8 number of health units which have prenatal and postnatal educational materials for pregnant/postnatal mothers which explain breast feeding (both benefits and procedures) Notes on breast feeding indicators 1 This indicator shows correct knowledge of mothers about when to initiate breast feeding. 2 The numerator is the number of women who answer correctly; the denominator all mothers. The target group can be limited to women with children under ages 2-3 to provide more current information. 3 Mothers should breast feed without supplements during the first 4 months of age. 4 Mothers should begin to introduce supplemental foods at 4-6 months of age. 5 Mothers should continue feeding during diarrhoea. The numerator is mothers who breast feed; the denominator is mothers who reported a case of diarrhoea. 6 The baby's head should be slightly elevated during feeding. The breast should be washed with soap and water (before and after) then dried. 7 The recommended diet for lactating women includes proteins, vegetables (particularly local vegetables high in vitamin A), and calcium (from milk, other dairy products, bones, or supplement). 8 Develop a list of benefits; then ask the provider to identify the benefits to the child's health. Benefits of breast feeding include improved nutrition, increased resistance to diseases, psychological security of the child, and others. Breast feeding should start within the first 24 hours and include the introduction of colostrum. DIARRHOEAL DISEASE CONTROL/ORAL REHYDRATION THERAPY Effect indicators number or percentage of mothers who have heard of ORT, ORS solution, SSS, or local names number or percentage of mothers who can state three rules of home case management for diarrhoea (fluids, feeding, care seeking) number or percentage of mothers who know how to prepare ORS solution, SSS, or local name number or percentage of mothers who know how to administer ORS solution, SSS, or local treatment1 number or percentage of mothers who used ORS solution, or a recommended home fluid (total volume increased), and continued feeding during their child's last diarrhoea episode (last month)2 number or percentage of breast feeding women who know to continue breast feeding during diarrhoea episode number or percentage of mothers who know how often to feed a child with diarrhoea (at least every 3-4 hours) Output indicators number or percentage of health workers who counsel mothers on preparation and administration of ORS solution3 number or percentage of health workers who correctly showed the mother how to prepare and administer ORS solution4 Input indicators number or percentage of health units which experienced stock shortages of ORS in the last month5 number or percentage of mothers who live within a reasonable distance, defined by local norms and expressed in terms of time, of a health facility or provider that has a regular supply of ORS and antibiotics, and practises correct case management6 Notes on diarrhoea disease control/oral rehydration therapy indicators 1 ORS solution, SSS or local treatment should be administered until the diarrhoea stops. 2 This indicator can be used to reflect health unit treatment, community health worker treatment, or home treatment, or all three combined. The decision of what to include will depend on programme objectives and norms. 3 Data from some countries have shown that health workers often discuss how to prepare ORS solution with mothers, but few provide them with information on how to administer ORS solution. This indicator requires information on more than one health worker task and would require supervisors to judge health workers on whether they performed both tasks. 4 This indicator examines whether health workers actually demonstrate preparation and administration to the mother. If health workers are not administering ORS solution in the clinics, mothers will have little chance to see what ORS solution preparation and administration involve. This information can be obtained through ORT supervisory checklists. 5 Information can be obtained from supervisory checklists, interviews with health workers or review of stock records. The second, more precise indicator, number of days when ORS was out of stock in the clinic or village, would require review of stock records and inventories. Another indicator that could be useful in programmes which promote home distribution of ORS would be the "number or percentage of villages without a regular source of ORS." Information could be compiled from programme reports, maps, and stock records. If the programme distributes ORS through a community-based worker during home visits, the indicator would be "number of days CHW experienced stock shortages of ORS," and information could be obtained from records, inventory, or supervisory checklists. 6 Information can be obtained by 1) reviewing the stocks of health facilities and/or providers, and 2) determining the target population that is accessible to the facilities. The numerator is the target population with access to stocked facilities; the denominator is the total target population. CHILDHOOD DISABILITIES Effect indicators number or percentage of disabled children enrolled in special schools or community programmes1 number or percentage of disabled children covered by appropriate health services number or percentage of clients/mothers with knowledge of special schools, community programmes, and NGOs involved with disabled children number or percentage of mothers who know appropriate care for disabled child number or percentage of mothers with disabled children who know how to prevent a similar disability2 Output indicators3 number or percentage of clients with disabilities referred for diagnosis or treatment number or percentage of health workers currently providing counselling/treatment for disabilities Input indicators number or percentage of health workers who can identify signs and symptoms associated with disabilities4 number or percentage of health workers who know how childhood disabilities can be prevented5 number or percentage of health workers with correct knowledge of national treatment/referral/counselling guidelines for disabilities number or percentage of health centres without treatment/counselling facilities for clients with disabilities number or percentage of health centres without established treatment, referral, and counselling guidelines Notes on childhood disabilities indicators 1 These coverage indicators can be modified to reflect different sub-groupings of disabilities, e.g., physical disabilities, mentally handicapped patients, etc. Numerators can be derived from reliable information on enrolment or client participation in disability-related activities and programmes. Denominators can be derived from an inventory or survey of disability-related services in the working area. 2 Disability impairments include movement, deformity, hearing, blindness, speech, behaviour, and other. 3 Information for these indicators can be obtained from supervisory checklists of disability-related service providers. 4 Information for these indicators can be obtained from supervisory checklists of disability-related service providers. 5 To obtain this information, list childhood disabilities of concern, how they occur, and what can be done to prevent them. For each type of disability, develop two or more preventive measures. Respondents are asked to identify preventive measures for each disability and are scored for the number of correct responses. CHILD IMMUNIZATION Effect indicators number or percentage of children age 12-23 months who are fully immunized with BCG, DPT, measles, and polio vaccines1 number or percentage of children age 12-23 months never immunized with BCG, DPT, measles, and polio vaccines2 number or percentage of mothers who know the age at which children should be immunized against measles (9-12 months of age) number or percentage of mothers whose children are not completely immunized who know when to return for the next immunization number or percentage of children age 12-23 months whose mothers can present a completely (per local standards) filled immunization card, given the age of the child Output indicators number or percentage of health workers using sterile needles and syringes for each injection for outreach programmes number or percentage of immunization sessions held (per area)3 number or percentage of health workers counselling mothers on possible side-effects of vaccines Input indicators number or percentage of health units [or number or percentage of immunization sessions] which experience shortages of vaccines, needles, syringes, and/or immunization cards4 number or percentage of health units whose refrigerator has not been at a temperature between 0C and 8C at all times during the previous period5 Notes on child immunization indicators 1 This is a standard format for assessing immunization coverage. However, if your programme is performing well according to this indicator, you may want to modify it to focus on younger children, e.g., children under one year of age. This more specific information could be obtained from household surveys, by modifying slightly the information collected with the Module 2 rapid or mini surveys. The reason for emphasising younger children is that the risk of dying is much higher in the younger age group. In addition, many older children may have already contracted the disease, clinical or sub clinical, before being vaccinated. Data for either version of this indicator could be collected using Module 2 rapid or mini-surveys. Data could also be compiled from clinic-based records if these are organised by child, and if an estimate of the total target population exists. If no clinic-based records exist, and surveys are impractical, information on the number of children fully immunized by age 23 months could be tallied as children come in for immunization. Their cards could be marked to show they had already been counted. Doing this over a one year period would provide a measure of the number of children completely immunized by a certain age, using an estimate of the number of children in that age group in the target area as the denominator. 2 This indicator helps managers identify which groups might be missed altogether by the immunization activities. It can only be collected through household surveys, because that is the only way to collect information from those who do not receive immunization through the programme. In addition, the household survey will also identify unimmunized children, and that information can be used to plan service strategies. 3 The minimum standard for this indicator would be quarterly immunization sessions. Mapping the area may be necessary to ensure that outreach sessions are planned in a way that affords access to the whole population. Access must be defined locally, either in terms of distance or travel time. Information on the number and location of immunization sessions can be obtained from work plans or immunization reports. 4 A high percentage points out problems in the supply system. Data can be obtained from supervisory checklists for immunizations and/or interviews with health workers or review of stock records and inventories. 5 This information can be obtained from supervisory checklists when supervisors inspect temperature logs. An alternative to this would be to divide the numerator into three groups: those without temperature logs, those whose temperature logs show constant temperature between 0C and 8oC, and those whose temperature logs indicate unacceptable temperature variations. GROWTH MONITORING/NUTRITION EDUCATION Effect indicators number or percentage of children under two years in the target area weighed at least once during the past quarter1 number or percentage of mothers breast feeding babies up to 12 (18) months of age number or percentage of mothers starting to give supplemental foods to infants between 4 and 6 months of age number or percentage of children whose weight-for-age is below the normal range or whose growth is faltering (per local norms)2 number or percentage of mothers with children under age two who interpret growth chart information3 Output indicators number or percentage of children under two years enrolled in a growth monitoring programme number or percentage of mothers with children under two years to whom the growth monitoring chart was explained by CHW during the last 3-6 months (for outreach programmes) number of growth monitoring sessions held (per area)4 number or percentage of health workers who track malnourished children5 number or percentage of high-risk children (malnourished ) followed up6 number or percentage of mothers told child's nutritional status7 number or percentage of mothers with growth-faltering children who received counselling on appropriate feeding Input indicators number or percentage of health workers/health units which do not have a functioning scale (accurate to 1/10 kg) number or percentage of health units (or growth monitoring sessions) experiencing shortages of growth cards in the last month8 Notes on growth monitoring/nutrition education indicators For indicators reflecting the target population, the denominator should be adapted to local programme objectives: children under two, children under five, etc. 1 This standard for frequency of weighing should follow programme guidelines, and ideally would be linked to the number of sessions held. Some programmes may want to concentrate their coverage measures on those age groups most at risk, e.g., 18-36 months, or those who may require more frequent weighings. The intervals between compilations/analyses should be long enough for differences to appear in frequency of weighing. For example, if the indicator is of quarterly weighing, the data should not be collected more than once, or at most, twice a year. If the number of children in the target area is known, and clinic-based records exist, information could also be compiled from service records. 2 The phrasing of this indicator should reflect current programme objectives. Information can be obtained from rapid surveys (if weights are taken). 3 This indicator measures mothers' ability to correctly interpret growth monitoring charts. Information can be gathered in a survey of mothers with children under age five. The interviewer uses a chart designed for the survey to determine if the mother can identify an underweight child. The numerator is mothers able to interpret the growth chart; the denominator is all women participating in growth monitoring. 4 The minimum standard for this indicator would be quarterly growth monitoring sessions (required to ensure coverage with quarterly weighings). Mapping the area may be necessary to ensure that outreach sessions are planned in a way that affords access to the whole population. Access must be defined locally, either in terms of distance or travel time. Information on the number and location of growth monitoring sessions can be obtained from work plans or growth monitoring reports. <%0> 5 "High-risk" will need to be defined locally, e.g., level 2 and 3 malnutrition, or not growing in the last 3 months. Information can be obtained from supervisory checklists. 6 If a high-risk system exists, this indicator measures how well it works. Information on high-risk children could be compiled from growth cards or other individual service records kept in the health unit. 7 This indicator could measure counselling at growth monitoring sessions or during routine curative sessions where the child is weighed. 8 A high percentage can signify problems in the supply system. Data can be obtained from supervisory checklists for growth monitoring (interviews with health workers or review of stock records and inventories). WATER SUPPLY, HYGIENE, AND SANITATION Effect indicators number or percentage of households receiving drinking water from a clean source, faucet, tap, pipe, covered well, or other safe source, within 15 minutes' walk1 number or percentage of households using a clean facility (water-seal latrine, pit privy, or WC) number or percentage of mothers knowing the importance of hand washing2 number or percentage of mothers with children under age two who use a baby potty Output indicators number or percentage of health workers inspecting latrines (per local standards) number or percentage of health workers inspecting community management of local water sources (per local standards)3 number of latrines built during the last year4 number of wells or other water sources constructed during the last year5 Input indicators number of wells or other water sources constructed per 1,000 population6 number of latrines built per 1,000 population1 number or percentage of communities with access to health staff or technicians with resources (information, funds, supplies) for building safe water supply systems and latrines7 Notes on water supply, hygiene, and sanitation indicators 1 This indicator measures the population's access to water, an important factor in sufficient water usage. Information can be obtained using rapid surveys. 2 Information on mothers knowledge can be obtained from supervisory checklists for water and sanitation which call for "exit" interviews with mothers, or by adding a question to the rapid survey forms. 3 This indicator measures the performance of health workers who either inspect the water sources themselves (if that is part of their tasks) or discuss maintenance of local water sources with the community. This information can be obtained from supervisory checklists (observations or interviews with health workers) or discussions with community members about health worker activities. The indicator should be adapted to the workers' water and sanitation tasks and job descriptions. 4 Information can be obtained from activity reports. 5 Information can be obtained from activity reports. 6 This indicator measures only the accessibility of water and sanitation facilities to a population, not the use of those facilities. It also indicates access to inputs in the local area. 7 This indicator measures the access of communities to critical inputs. ACCIDENTS AND INJURIES Effect indicators number or percentage of respondents with knowledge of how to care for accidents and injuries1 number or percentage of respondents with knowledge of an emergency care facility2 number of respondents who use safety precautions and equipment on a daily basis3 Output indicators number of respondents treated for an injury (by type)4 (in a work place or other high-risk environment) number of safety regulations implemented and enforced number or percentage of health workers (CHWs, factory or plant cadres, traditional practitioners, etc. ) trained in injury care number or percentage of health workers who received formal training in hazard identification and intervention Input indicators number or percentage of health care facilities without 24-hour emergency care services number or percentage of health care facilities without treatment and referral procedures for clients with injuries Notes on accidents and injuries indicators 1 This indicator is most useful in a context where risk of accident or injury is relatively high, such as a factory, farm, or other place of work. Start by determining the profile of common accidents and injuries; then determine appropriate first aid treatment. Ask respondents how they would treat key accidents and injuries and compare against the standard. Respondents are the target population of accident/injury prevention and treatment efforts. 2 This includes phone number, address, or directions on how to go to a facility. 3 This indicator requires a revision of prescribed safety precautions and equipment in a particular context such as a factory, farm, hospital, chemical plant, etc. Respondents are interviewed about the safety precautions followed and equipment available. 4 Respondents can be interviewed with a rapid survey (see Module 2). CHRONIC, NON-COMMUNICABLE DISEASES Hypertension1 Effect indicators number or percentage of patients with hypertension number or percentage of patients who developed other complications of hypertension number or percentage of patients, diagnosed with chronic hypertension, who are in a systematic anti-hypertensive therapy programme number or percentage of patients with moderate hypertension2 who follow a prescribed diet on a daily basis number or percentage of patients who take medication as prescribed number or percentage of patients who practise the appropriate diet on a daily basis number or percentage of patients who were given a special regimen sheet for the use of medication and diet Output indicators number of clients diagnosed for hypertension number of patients with hypertension who were treated number of patients who switched from prescribed medicine to diet number of health workers who are clinically knowledgeable in providing diet and medication regimen to patients with hypertension number of systematic visits by patients with hypertension number or percentage of patients who have been assessed for renal function number of patients with hypertension who were provided with instructions for the use of medication and diet Input indicators number of health workers who can provide diet and medication information to patients with hypertension number of primary health centres with daily/weekly availability of trained family doctor number of health centres with sphygmomanometers number of primary health centres with functioning laboratory equipment and medication for hypertension3 Notes on hypertension indicators 1 Developed by Walid Abubaker, MD, Senior Scientist, University Research Corporation 2 Diastolic blood pressure between 90 and 115, or local definition 3 As determined by local standards Diabetes1 Effect indicators number or percentage of diabetic patients with annual assessment of renal function number or percentage of diabetic patients with semi-annual measurement of glycosylated Hgb number or percentage of diabetic patients with documented home glucose monitoring number or percentage of diabetic patients with discrepancies between home and clinic monitored results2 number or percentage of diabetic patients with family members who know how to handle a diabetic emergency3 Output indicators number or percentage of persons diagnosed for diabetes mellitus number or percentage of patients treated for diabetes mellitus by a physician number or percentage of patients treated for diabetic ketoacidosis number or percentage of patients treated for diabetic foot lesions Input indicators number of general practitioners or family physicians per 1,000 households presence of equipment and other materials in the health units (glucose monitoring, blood pressure, protocol for home glucose monitoring, and insulin schedule) Notes on diabetes indicators 1 Developed by Walid Abubaker, MD, Senior Scientist, University Research Corporation 2 This data requires a comparison of clinic records and results of home monitoring. The indicator measures the client's ability to monitor diabetes. 3 This information is gathered from interviewing household members (see Module 2). Anaemia Effect indicators number or percentage of anaemic patients with semi-annual evaluation and follow-up of haemoglobin and haematocrit number or percentage of anaemic patients with documented improvement in anaemic condition number or percentage of anaemic women who know the treatment for anaemia Output indicators number or percentage of persons diagnosed for anaemia number or percentage of patients treated for anaemia number or percentage of patients treated for symptomatic anaemic conditions (pale, weak, etc.) number or percentage of patients who received education for nutritional health Input indicators number of general practitioners of family physicians per 1,000 household with knowledge or understanding of anaemia number of providers/primary centres with staff, equipment and other materials for monitoring haemoglobin, haematocrit, blood count (WBC/CBC) MALARIA Effect indicators number or percentage who know how malaria is spread1 number or percentage of population who are protecting themselves against malaria2 number or percentage of malaria (fever) cases treated with anti-malarial drugs at home (by mother who knows correct dosage)3 number or percentage of patients who were prescribed anti malarial drugs who know dosage, frequency, and duration of treatment4 Output indicators number or percentage of malaria (fever) cases treated at health unit5 [in areas where confirmation is the norm] number or percentage of health workers who take/request a blood slide from malaria/fever cases number or percentage of health workers who screen fever patients for signs/symptoms of other serious illnesses (meningitis, pneumonia, etc.) Input indicators number or percentage of health units experiencing stock shortages of anti-malarial drugs in the last month6 number or percentage of health units without functioning laboratory equipment Notes on malaria indicators 1 The indicator measures knowledge of modes of transmission. 2 The following can be used to protect against malaria: mosquito nets, household spray, eliminating standing water, using anti-malarial drugs, and other locally appropriate means. 3 This indicator measures coverage for home treatment of malaria for programmes which are promoting such an approach. Information for this indicator can only be obtained from rapid surveys. A drawback to this indicator is that the denominator would be based on self-diagnosed fever or malaria. Local definitions of malaria will have to be developed to determine which symptoms constitute a diagnosis of malaria. 4 The numerator is the number of clients who are knowledgeable of correct treatment divided by all patients prescribed anti-malarial drugs. Data is gathered by a survey of individuals diagnosed with malaria who received drugs from a CHW or health facility. 5 This indicator measures health worker treatment of malaria. Like the indicator above, information can be obtained from rapid surveys, and it, too, is based on self-diagnosed fever or malaria. 6 Both indicators can point out problems in the supply system. A more detailed indicator, number of days when anti malarial drugs were out of stock during last period, can measure the extent of the problem. Data can be obtained from supervisory checklists for malaria (interviews with health workers or review of stock records and inventories). TREATMENT OF MINOR AILMENTS Effect indicators number or percentage of illness episodes treated at health unit1 number or percentage of patients who were able to obtain all of the medication prescribed in the health unit number of patients receiving laboratory services in the same health unit per number requiring laboratory services number or percentage of patients or their attendants who understand the treatment prescribed <196> how much, how long, how often2 Output indicators number of treatments given during last period by condition3 number or percentage of health workers asking at least five history questions and carrying out at least one physical exam4 Input indicators number or percentage of health workers who know the explicit criteria for referring patients to a higher level (per local policy)5 number or percentage of health units without functioning:6 stethoscope thermometer watch with second hand scale blood pressure cuff microscope and slides number or percentage of cases where prescribed medicines were available at the health unit at the time of consultation, compared to all cases that were prescribed medicines7 availability of drugs for high volume conditions availability of laboratory tests used more frequently Notes on treatment of minor ailments indicators 1 This indicator is a rough measure of coverage for curative services. It should be interpreted with care, since some illness episodes may not require health care services. The types of illness episodes to be counted in the denominator will need to be decided locally. Such an indicator can suggest problems such as acceptability or financial accessibility. Interpretation can be aided by using this indicator to compare "coverage" over time or with other health units. 2 To apply this indicator develop a list of minor ailments to monitor. For each ailment determine the treatment protocol <196> how much, how long, and how often. This indicator measures the knowledge of patients/caretakers (effect) and can also be modified to gauge the knowledge of health providers (input). 3 This indicator, plotted over time and compared among health units, can help managers track trends and efficiency. Variations could be due to seasonal epidemiological patterns or financial resources of the population. Changes could also reflect problems or strengths in acceptability of services. Efficiency can be assessed by comparing the number of treatments among health units to see if personnel allocation reflects use rates. 4 The numbers of history questions and exams can be adapted to reflect local policy. However, a minimum number is necessary to ensure that workers are doing more than simply accepting the patient's diagnosis, and that they are able to identify potential multiple health problems. Information can be obtained from curative care supervisory checklists which use observation. In some cases, the supervisor may have to use some judgement about whether the worker did the appropriate history and physical exams. 5 This indicator measures the knowledge of health providers and their ability to treat minor ailments. Develop a list of key minor ailments of concern to your programme and the referral criteria. Then ask the respondent to recall ailment-specific referral criteria. 6 The equipment to be included in this indicator will need to be adapted to local policy and the level of health worker being evaluated. 7 This indicator attempts to evaluate availability of essential drugs. The indicator can be revised if local policy states that drugs are not to be distributed at the health unit, e.g., drugs are distributed at a nearby drug outlet or clinic/hospital. Data can be gathered by checking drugs prescribed with drug stocks. TUBERCULOSIS Effect Indicators number or percentage of children vaccinated with BCG1 number or percentage of target group who know how TB is spread and how to prevent infection2 number or percentage of detected tuberculosis cases followed to cure3 number of active tuberculosis cases4 number or percentage of tuberculosis patients knowing why it is important to complete treatment number or percentage of tuberculosis patients knowing the correct dosage and duration of treatment for tuberculosis number or percentage of population with persistent cough lasting more than two weeks who sought treatment for TB Output indicators number or percentage of suspected tuberculosis cases sent for confirmation5 number or percentage of health workers who have a system for following up suspected and confirmed tuberculosis cases6 number or percentage of suspected and confirmed tuberculosis cases followed up7 Input indicators number or percentage of health units without adequate equipment to diagnose tuberculosis8 if the health unit is a tuberculosis treatment centre, number of days when tuberculosis drugs were out of stock Notes on tuberculosis indicators 1 This measures the coverage of the immunization programme and the effectiveness of TB prevention efforts. 2 This information can be gathered from a survey. Asking probing questions to explain (see Module 2). 3 Information for this indicator, which is a partial measure of coverage, can be obtained by compiling information from tuberculosis service records. The denominator for this indicator would be the number of tuberculosis cases (detected by the health services) that should have completed treatment during the period being evaluated. A truer measure of coverage might be constructed if reliable information can be collected about the number of tuberculosis cases in the community. The vital events rapid survey does contain information about point prevalence for tuberculosis, but the value of this information depends on the ability of the population and the interviewer to classify tuberculosis cases from survey data. If this information is felt to be reliable enough, the indicator could be modified by changing the denominator to all tuberculosis cases in the community, rather than those detected by the health services. 4 This indicator allows managers to follow changes in tuberculosis case detection over time or among health units. Information can be compiled from tuberculosis registers. 5 Confirmation can be defined as laboratory examination of sputum or X-rays, depending on local policy. This information can be obtained from supervisory checklists for tuberculosis or, if the information is available, from compilation of curative consultation or tuberculosis records. 6 Information can be obtained from supervisory checklists. 7 If a follow-up system exists, this indicator measures how well it works. Information on suspected and confirmed cases could be compiled from tuberculosis service records kept in the health unit to determine the percent of active cases that presented themselves voluntarily for their appointments or were visited/contacted by the health services if they defaulted on their treatment. 8 Equipment includes thermometer, stethoscope, tuberculosis te