THE PHC MAP SERIES OF MODULES, GUIDES AND REFERENCE MATERIALS Each module includes: a User's guide a Facilitator's guide computer programs Module 1 Assessing information needs Module 2 Assessing community health needs and coverage Module 3 Planning and assessing health worker activities Module 4 Surveillance of morbidity and mortality Module 5 Monitoring and evaluating programmes Module 6 Assessing the quality of service Module 7 Assessing the quality of management Module 8 Cost analysis Module 9 Sustainability analysis Manager's guides and references Better management: 100 tips Problem-solving Computers The computerised PRICOR thesaurus Production Managers: Ronald Wilson, Aga Khan Foundation, Geneva and Thongchai Sapanuchart, Somboon Vacharotai Foundation, Bangkok, Thailand Editor: Jennifer Sharples, Bangkok, Thailand Design & Layout: Helene Sackstein, France Desktop Publishing: Margaret M. Growe, Byte Type Services, Chapel Hill, North Carolina, USA; Michael S. Sunggiardi, Batutulis Computer, Bogor, Indonesia Printing: Thai Wattana Panich, Bangkok, Thailand Published in 1993 by the Aga Khan Foundation USA, Suite 700, 1901 L Street N.W., Washington DC and the Aga Khan Foundation,P.O. Box 435, 1211 Geneva 6, Switzerland. Additional copies are available at the Aga Khan Foundation Canada in Toronto; Aga Khan Foundation United Kingdom in London; and AKF offices in Dhaka, Bangladesh; Delhi, India; Nairobi, Kenya; Karachi, Pakistan; Lisbon, Portugal; and Dar-es-Salaam, Tanzania; the Aga Khan University Faculty of Health Science in Karachi, Pakistan; the Somboon Vacharotai Foundation and the ASEAN Institute for Health Development in Thailand Cover Photo: A community health nurse of the Aga Khan Community Health Programme in Dhaka meausres the blood pressure of a healthy pregnant mother during a prenatal home visit Photo by Jean-Luc Ray for AKF MODULE 7: USER'S GUIDE ASSESSING THE QUALITY OF MANAGEMENT Lori DiPrete Brown URC ISBN: 1-882839-05-6 Library of Congress Catalog Number: 92-75466 On the way to a rural health clinic in Peru. Photo by WHO Dedicated to Dr. Duane L. Smith (1939-1992), Dr. William B. Steeler (1948-1992) and all other health leaders, managers and workers who follow their example in the effort to bring quality health care to all in need. Near Kakamega, Kenya, a nurse is teaching a mother how to prepare maize-salt oral rehydration solution on a cooking fire outside her house Photo by Jean-Luc Ray for AKF An overview of PHC MAP The main purpose of the Primary Health Care Management Advancement Programme (PHC MAP) is to help PHC management teams collect, process and analyse useful management information. Initiated by the Aga Khan Foundation, PHC MAP is a collaborative programme of the Aga Khan Health Network<$FThe Aga Khan Health Network includes the Aga Khan Foundation, the Aga Khan Health Services, and the Aga Khan University, all of which are involved in the strengthening of primary health care.> and PRICOR. <$FPrimary Health Care Operations Research is a worldwide project of the Center for Human Services, funded by the United States Agency for International Development.> An experienced design team and equally experienced PHC practitioner teams in several countries, including Bangladesh, Chile, Colombia, the Dominican Republic, Guatemala, Haiti, India, Indonesia, Kenya, Pakistan, Senegal, Thailand and Zaire, have worked together to develop, test and refine the PHC MAP materials to ensure that they are understandable, easy to use and helpful. PHC MAP includes nine units called modules. These modules focus on essential information that is needed in the traditional management cycle of planning-doing-evaluating. The relationship between the modules and this cycle is illustrated below. PHC MAP modules and the planning-evaluation cycle PHC MAP MODULES 1. Information needs 2. Community needs 3. Work planning 4. Surveillance 5. Monitoring indicators 6. Service quality 7. Management quality 8. Cost analysis 9. Sustainability Managers can easily adapt these tools to fit local conditions. Both new and experienced programmers can use them. Government and NGO managers, management teams, and communities can all use the modules to gather information that fits their needs. Each module explains how to collect, process and interpret information that managers can use to improve planning and monitoring. The modules include User's Guides, sample data collecting and data processing instruments, optional computer programs, and Facilitator's Guides, for those who want to hold training workshops. The health and management services included in PHC MAP are listed below. OTHER HEALTH CARE Water supply, hygiene and sanitation School health Childhood disabilities Accidents and injuries Sexually transmitted diseases HIV/AIDS Malaria Tuberculosis Treatment of minor ailments Chronic, non-communicable diseases HEALTH SERVICES Planning Personnel management Training Supervision Financial management Logistics management Information management Community organisation MANAGEMENT SERVICES Health and management services GENERAL PHC household visits Health education MATERNAL CARE Antenatal care Safe delivery Postnatal care Family planning CHILD CARE Breast feeding Growth monitoring Nutrition education Immunization Acute respiratory infection Diarrhoeal disease control Oral rehydration therapy Several Manager's Guides supplement these modules. These are: Better Management: 100 Tips, a helpful hints book that describes effective ways to help managers improve what they do; Problem-solving, a guide to help managers deal with common problems; Computers, a guidebook providing useful hints on buying and operating computers, printers, other hardware and software; and The computerised PRICOR thesaurus, a compendium of PHC indicators. The Primary Health Care Management Advancement Programme has been funded by the Aga Khan Foundation Canada, the Commision of the European Communities, the Aga Khan Foundation U.S.A., the Aga Khan Foundation's head office in Geneva, the Rockefeller Foundation, the Canadian International Development Agency, Alberta Aid, and the United States Agency for International Development under two matching grants to AKF USA. The first of these grants was "Strengthening the Management, Monitoring and Evaluation of PHC Programs in Selected Countries of Asia and Africa" (cooperative agreement no. OTR-0158-A-00-8161-00, 1988-1991); and the second was, "Strengthening the Effectiveness, Management and Sustainability of PHC/Mother and Child Survival Programs in Asia and Africa" (cooperative agreement no. PCD-0158-A-00-1102-00, 1991-1994). The development of Modules 6 and 7 was partially funded through in-kind contributions from the Primary Health Care Operations Research project (PRICOR) of the Center for Human Services under its cooperative agreement with USAID (DSPE-6920-A-00-1048-00). This support is gratefully acknowledged. The views and opinions expressed in the PHC MAP materials are those of the authors and do not necessarily reflect those of the donors. All PHC MAP material (written and computer files) is in the public domain and may be freely copied and distributed to others. In rural Egypt where primary health care extends to some remote areas: women grinding corn under a wall decoration welcoming husbands back from Mecca Photo by P. Boucas for WHO Contents QUICK START 1 INTRODUCTION What is management assessment? 3 How you can use this module 3 Management checklists 5 Management discussion guidelines 9 An overview of management services and checklists 10 Using service quality checklists for management assessment 14 Strengths and limitations of the module 17 THE MANAGEMENT ASSESSMENT PROCEDURES Step 1: Specify the objectives of the management assessment 19 Step 2: Determine assessment methods 21 Step 3: Select and adapt appropriate checklists or guidelines 23 Step 4: Carry out the assessment 24 Step 5: Tabulate and analyse the data 26 Step 6: Present and review preliminary results 29 Step 7: Analyse the management process in more detail 30 Step 8: Take action to improve management 31 APPENDICES A. Management assessment discussion guidelines 35 B. Management assessment checklists 51 C. Worksheet 83 REFERENCES AND BIBLIOGRAPHY 84 ACRONYMS AND ABBREVIATIONS 85 Acknowledgements The first draft of this module was reviewed and field tested in 1991 by a number of PHC specialists and field managers. The module was also reviewed and critiqued by the PHC MAP Technical Advisory Committee at a meeting in Bangkok in September, 1991. Special thanks to the staff of URC's International Division for their critiques and suggestions, especially to Paul Richardson and Neeraj Kak for their help in conducting field tests and in compiling and analyzing all of the field test feedback. Thanks also to Jack Reynolds who conceptualized the module and provided guidance throughout its development and revisions. Dr. Reynolds also prepared the Quick start for this module. The contributions of the individuals below were invaluable and greatly appreciated. External reviewers and contibutors: Donald Belcher Veterans Administration, Seattle, WA, USA Paul Zeitz Johns Hopkins School of Hygiene and Public Health, Baltimore, MD, USA William Reinke Johns Hopkins School of Hygiene and Public Health, Baltimore, MD, USA Bobby Zachariah Price Waterhouse Associates, Kingston, Jamaica Michael Bernhardt University of Puget Sound, WA, USA Internal reviewers: Thailand Jumroon Mikhanorn Somboon Vacharotai Foundation (SVF) Thongchai Sapanuchart SVF Donna Robinson SVF Yawarat Porapakkham Asian Institute for Health Development (AIHD) Chaweewon Boonshuyar AIHD Som-Arch Wongkhomthong AIHD Orapin Singhadej AIHD Peerasit Kamnuansilpa Health and Population Research Company (HPRC) Butsabar Subongkot HPCR Field tests: Countries Participating organisations, field test facilitators Bangladesh Test 1: 4 The Asia Foundation (TAF) sub-grantees: Progoti Samaj Kallyan Protishthan (PSKP); Unity through Population Service (UTPS); Polli Shishu Foundation of Bangladesh (PSF); Facilitator: Barkat-e-Khuda, URC/Bangladesh. Test 2: 3 TAF subgrantees: UTPS; PSKP; CWFP; PSF. Pakistan Aga Khan University, Facilitator: Khatidja Husein, Aga Khan University, Karachi, Pakistan Thailand Ministry of Public Health, Srisaket; Somboon Vacharotai Foundation (SVF), ASEAN Institute for Health Development (AIHD); Health and Population Research Company (HPRC); Facilitator: Chaweewon Boonshuyar, AIHD India Junagadh PHC Project; Sidhpur Sustainable Health System Project, Gujarat; Aga Khan Health Service India (AKHSI); Facilitators: Neeraj Kak, University Research Corporation (URC), Bethesda, MD, USA; Vijay Moses, AKHSI'S Kenya: Mombasa PHC Project; Kisumu PHC Project; Facilitators: Paul Richardson, URC; Esther Sempebwa, Mombasa PHC Project; Matthew Onduru, Kisumu PHC Project Colombia: Fundacion Santa Fe de Bogota; Facilitator: Jorge E. Medina Dominican Republic CARE, MUDE, Adoplafan, Aguas Vivientes, Caritas; Facilitators: Jorge E. Medina; Linda Ashburn, URC Indonesia Department of Health, Jakarta; Facilitator: Sandi Iljanto, University of Indonesia Children in the U.K. take plentiful clean water for granted Photo by Jean-Luc Ray for AKF Planning Personnel mgmt. Training Supervision Financial mgmt. Logistics mgmt. Information mgmt. Community organ. Number of YES observations An animated conversation between an auxiliary health worker and a villager in West Azerbaijan, Iran Photo WHO/Ministry of Health of Iran Introduction What is management assessment? Management assessment is the process of evaluating the strengths and weaknesses of such activities as planning, supervision, training and logistics. This module presents a series of management checklists, discussion guidelines, and analytical techniques for management assessment. With adaptation, managers can apply the checklists to determine whether management systems are functioning according to the norms and standards set by the programme. The PHC MAP discussion guidelines and analytical techniques complement the checklist approach by helping the PHC team to explore the causes at the root of the problem and to develop creative and appropriate solutions. The aim of management assessment is to identify specific ways in which the management system could function better, rather than to point out mistakes or blame those responsible. The approach presented here is intended to help teams work together to improve management which will, in turn, improve the effectiveness, efficiency, and quality of PHC services. While this module concentrates on the quality of PHC management services, Module 6: Assessing the quality of services, outlines a similar process to examine and improve directly the quality of service delivery. How you can use this module This module enables managers and supervisors to carry out management assessment with the aid of management checklists, discussion guidelines, and analytical techniques. The management assessment checklists presented here are based on explicit criteria about the various management processes. These checklists operationalise management assessment activities into discrete tasks, enabling managers and service delivery teams to conduct objective, thorough, and systematic assessments. In-depth discussion with the staff involved in the processes is another way to assess and analyse management performance. The PHC MAP discussion guidelines can be used to structure an open-ended discussion and analysis of management services. This management assessment approach can be used in a variety of ways. Examples include: Management evaluation: These tools can form the basis of an evaluation of PHC management. Such an assessment would include observation of activities, a review of records, and a review of the health facility itself. It could also include interviews with members of the health team and community members who are or should be served by the programme. Structured interviews and open-ended informal discussions could be used. A multi-facility review could also be carried out based on the materials presented here. Guidelines for team self-assessment: PHC MAP tools can also be used for team self-assessment. Group discussion, based on the PHC MAP discussion guidelines, can serve as a starting point for sharing information and opinions and identifying areas that need attention. Guidelines for planning new management procedures: Some teams may find that important management services are not in place in their health programmes. Rather than beginning with assessment in such cases, the checklists can be used by a manager or team as a planning tool for a needed management procedure. Tools for monitoring management improvement efforts: Managers can also use the tools to monitor the impact of attempts to improve management. This could be especially valuable after an initial assessment has taken place, or as an interim activity between assessments. Management job aids: Managers or PHC teams can also use these guidelines as job aids in the various management areas addressed. In each management area, specific tasks that contribute to better management are listed. This list of tasks could be used as a checklist that managers can refer to as they plan and carry out management activities. Management training: These materials are also useful in the development of a management training programme. Management assessment activities are intended to complement the other elements of the PHC Management Advancement Programme. Areas for programme improvement will be identified through a management information audit (Module 1), service quality assessment (Module 6), or routine programme monitoring (Module 5). Problems encountered in these areas are sometimes caused by shortcomings in programme management. With the help of these management guidelines, problem areas can be explored in more detail so that appropriate action may be taken. Further, since service delivery problems are often solved through management interventions, these guidelines can be used to develop and monitor efforts to improve the programme. Management assessments can be carried out by managers, service delivery teams, community groups, or outside evaluators. They are most effective when they are used for internal assessment by the PHC team to identify areas for improvement and to develop plans for action. Management checklists PHC management checklists have been developed for 8 management services. Each checklist consists of an organised list of specific recommended tasks, along with brief instructions about how the information gathered can be used. These checklists can be easily adapted to specific public and private programmes by adding, modifying, or omitting items. They can be used to assess overall PHC management, specific management services or sub-functions, or vertical service delivery programmes. Information can be gathered through interviews, group discussions, document review, observation of management activities, or a review of the health facility. The management services listed on the following page are included. Each checklist divides the management service into activities, which are further divided into distinct tasks. For example, the planning checklist has a section on the mission statement, programme objectives, information needed for planning, financial planning, programme planning, and individual work planning. This structure will help the user to identify major sub-functions that are problematic so that appropriate action may be taken. The PHC MAP checklist for planning is presented in Exhibit 1 on the following page. Detailed checklists for all eight management services functions are included in Appendix B. In the form presented here, most questions are phrased for a yes/no response. The questions are formulated so that a "no" response indicates a potential problem area that may require further attention. The yes/no format was chosen so that analysis of the data would be straightforward. However, the checklist can be used in other ways if more flexibility is needed. For example, if a manager feels that the yes/no format is too rigid for some questions, those questions could be re-worded slightly and a scale of 1-5 could be used. For example, the question, "Do staff members understand the programme mission?", could be changed to, "How well do staff members understand the programme mission?" The 5 point scale could be defined as, 1 = very poorly, 2 = poorly, 3 = adequately, 4 = well, and 5 = very well. Alternatively, the question could be phrased as a declarative sentence, "Staff members understand the programme mission well", and the scale could be defined as 1 = disagree strongly, 2 = disagree, 3 = neutral, 4 = agree, and 5 = agree strongly. Other modifications on the scale are also possible. Some questions ask for opinions about the quality or adequacy of specific tasks. In such cases the perspectives of managers, health service providers, and community members should be taken into account. Some questions, such as, "Does the facility have a refrigerator?", are straightforward and can be answered reliably by one respondent, although a check on such information is sometimes useful. Others should be explored from more than one point of view. For example, the question, "Do health workers receive adequate technical support and supervision?", should be explored through consultation with supervisors and health workers themselves. Sources of information can include managers, health service providers, community health workers, clients, and other community members. Records of reviews, observation of activities, and site visits are also important sources of information. EXHIBIT 1: MANAGEMENT ASSESSMENT Checklist for planning "Health planning is the process of defining community health problems, identifying needs and resources, establishing priority goals, and setting out the administrative action needed to reach those goals". This checklist is intended for use in the assessment of planning activities. Its objective is to help managers to enhance the quality of their programmes by identifying and resolving problems in the planning area. They can be adapted for use in both vertical programmes and integrated PHC efforts. The questions below can be answered through interviews, document review, observation of management activities, or a review of the health facility. Some questions ask for the respondent's opinion about the quality or adequacy of specific tasks. In such cases the perspectives of managers, health service providers, and community members should be taken into account. Areas that are deemed inadequate can be further explored through focused discussions with key informants. With modification, the checklists can be used as a basis for an open-ended interview or group assessment. 1. Health facility 2. Service provider 3. Observer/Supervisor 4. Date Mission statement A mission statement describes the purposes and overall goals of an organisation. These questions will help you to determine whether the mission is clearly defined and understood by programme staff, the community, and donors: 5. YES NO Is there a written mission statement? 6. YES NO Does the mission statement include a clear primary health care strategy? 7. YES NO Is the mission statement understood by the health centre staff? 8. YES NO Is the mission statement understood by the community? 9. YES NO Is the mission statement understood by health centre management? 10. YES NO Is the mission statement understood by the board? 11. YES NO Is the mission statement understood by the donors? 12. YES NO Does the programme plan directly address the overall programme mission? Programme objectives Objectives are the specific results that are expected from a programme or activity. These questions will help you determine whether the objectives are defined adequately for the purpose of planning: 13. YES NO Is the geographic area to be served clearly defined? 14. YES NO Are the age groups to be served clearly defined? 15. YES NO Are high-risk groups to be served specified? 16. YES NO Are the services offered specified? 17. YES NO Are there explicit programme objectives? 18. YES NO Did the community play a role in setting goals and objectives? 19. YES NO Are the objectives reviewed and revised on a regular basis? 20. YES NO Do government programmes and/or donors understand and agree with the goals and objectives? 21. YES NO Are coverage objectives for each service stated and quantified? 22. YES NO Are coverage objectives understood by staff? 23. YES NO Are objectives for service quality stated and quantified? 24. YES NO Are service quality objectives understood by staff? 25. YES NO Are objectives for improvements in knowledge in specific PHC areas stated and quantified? 26. YES NO Are objectives for improvements in knowledge understood by staff? 27. YES NO Are objectives for behaviour change in specific PHC areas stated and quantified? 28. YES NO Are behaviour change objectives understood by staff? 29. YES NO Are objectives for health status improvement stated and quantified? 30. YES NO Are health status objectives understood by staff? Management discussion guidelines PHC MAP management checklists attempt to measure or quantify management performance. However, many management areas are difficult to quantify or isolate into a yes/no question. Because management is complex and multi-faceted, a more qualitative type of assessment tool is needed. The PHC MAP discussion guidelines complement the checklists by helping managers to capture the complexity of their management systems and to study them in depth. Discussion guidelines for all 8 PHC MAP management services have been developed (see Appendix A). Exhibit 2 presents an example of the discussion guidelines for training. The discussion guidelines can be used as a basis for an open-ended group discussion. The participatory nature of this method is especially important because the interaction among group members will make for a richer response and groups are better than individuals at identifying problems and their multiple causes. EXHIBIT 2: MANAGEMENT ASSESSMENT Discussion guidelines for training Training is the process of continually improving the knowledge, skills and competencies of health workers. Introduction: Welcome the group and briefly give the purpose of the discussion. Ground rules: Explain the following ground rules to the group: Ground rules for group discussion Everyone's ideas and opinions are important. There are no right or wrong answers. Both positive and negative comments are welcome. Participants should feel free to disagree with one another so that all points of view are heard. Don't wait to be called on; it's a group discussion. Please speak one at a time. These discussion guidelines are provided to help you to lead a group discussion about your training system. You may also refer to the detailed version of the management checklists (Appendix B) as a resource for the discussion. Discussion What works well in the way we do training? Guidelines What aspects of the way we do training do not work well? What kind of information do you use to plan or focus training? Do you use information about health worker knowledge, attitudes, and practices? Do you use information about community needs? Is this information available to you? What training methods are used? Are they used correctly? Are they effective? Is the content of the training appropriate for local norms? What resources are available to support training? What resources are needed? Key questions Is programme specific information (from MIS or supervisor) about service quality used to plan or focus the training? Did the training include a pre-test of health worker knowledge and skills? Does training include a post-test of health worker knowledge and skills? Is there a follow-up assessment of training (about six months later) to assess the impact of the training? Are training topics explained clearly? Are training topics demonstrated? Do trainees receive written materials? Are health workers satisfied with the training they receive? Will training be followed up through the regular supervision system? Will trainee performance in new skill areas be monitored through MIS? Is the training content complete and accurate according to local norms? Are resources adequate? An overview of management services and checklists This section defines the eight management services included in the PHC MAP programme and describes the key components of the management checklists. Clear understanding of these basic management concepts is essential for project managers and health centre teams who want to assess and improve their programmes. 1. PLANNING Definition: Health planning is the process of defining community health problems, identifying needs and resources, establishing priority goals, and setting out the administrative action needed to reach those goals. Description and explanation of guidelines: The guidelines are divided into five sections which address different aspects of the planning process. The first section assesses whether there is a mission statement and whether it is understood by programme staff, the community, and donors. The second section assesses whether programme objectives (specific results that are expected from a programme or activity) have been adequately defined for the purpose of planning. The next section considers whether the information needed for programme planning is available. Section four assesses the adequacy of the organisational structure, overall work schedule, and whether management activities are planned. The final section discusses individual work planning for staff members, which is dealt with in more detail in Module 3. 2. TRAINING Definition: Training is the process of continually improving the knowledge, skills, and competencies of health workers. Description and explanation of guidelines: The guidelines are divided into four sections which address different aspects of the training process. The first section deals with the planning of training activities. The second section assesses the training methods, and the third assesses training content. The final section includes questions about the adequacy of training resources. 3. SUPERVISION Definition: Supervision is the process of directing and supporting staff so that they may effectively perform their duties. Supervisors provide leadership, support, guidance, training, and assistance in the identification and solution of problems so that service quality and effectiveness may continually improve. Description and explanation of guidelines: The guidelines address four aspects of supervision. The first section can be used to describe or assess supervisory activ- ities. The second section includes questions about supervisory methods, as well as a general assessment of the adequacy of supervision in a variety of technical and management areas. The third section addresses supervision of technical service quality, and section four assesses supervisory style. Section five looks at supervisory action taken, specifically work planning and problem-solving. The final section assesses whether vital information for problem-solving is included in the supervisory records. 4. PERSONNEL MANAGEMENT Definition: Personnel management includes activities which relate to recruiting, hiring and supporting staff, and defining the roles of the individual. It also refers to standard procedures related to compensation, benefits, and work schedules. Description and explanation of guidelines: The guidelines are divided into four sections which address different aspects of personnel management. The first section looks at essential personnel management issues that reflect the quality of the personnel management system, such as whether salaries are fair, and whether staff turnover is kept at a reasonable level. The second section assesses whether key personnel policies and procedures are in place. The questions in section three will help to assess the adequacy of job descriptions. Section four addresses the ongoing activity of work planning and performance assessment. 5. FINANCIAL MANAGEMENT Definition: Financial management includes management of and accountability for programme finances, budgeting for planned activities, and, in some programmes, cost recovery or fund-raising. Description and explanation of guidelines: The guidelines are divided into seven sections which address different aspects of financial management. They permit a detailed assessment of processes related to general accounting, cash disbursements, accounts payable and purchases, payroll, petty cash, financial planning and management, and sources, and adequacy of funding. 6. LOGISTICS MANAGEMENT Definition: Logistic systems deal with the procurement, storage, and tracking of supplies and equipment. Description and explanation of guidelines: The guidelines are divided into five sections which address different aspects of logistics management. They are procurement, reception of materials, inventory control, stock issue, and transportation. 7. INFORMATION MANAGEMENT Definition: An information system is organised around key indicators that measure a programme's progress toward its goals. It is a systematic way of collecting, reporting, and using data at all programme levels. Description and explanation of guidelines: The guidelines are divided into eight sections which address various aspects of service delivery: planning the information system; record keeping (health centre-level and district-level); compilation and tabulation and analysis of information (at health-centre and district-level); use of information for management decision-making; reporting information and feedback; ad hoc data collection; compilation and analysis of ad hoc data collection; and use of information for management decision-making. 8. COMMUNITY ORGANISATION Definition: Community organisation is the involvement of the community in the design, planning, promotion, or delivery of health enhancing activities. Description and explanation of guidelines: The guidelines are divided into six sections which address different aspects of service delivery. The first section is a profile of the local health committee or community organisation. This profile will help supervisors or health workers to describe the context of community activities and to identify opportunities and constraints. The second section looks at utilisation, satisfaction, and participation. The third section looks at indicators of quality, such as the representativeness of the group and the level of the group's activities. The fourth section looks at the community's role in programme implementation, while the fifth section assesses whether communities are playing a leadership role in activities. The final section looks at the support received by community organisations and co-ordination with other programmes or community efforts. Using service quality checklists for management assessment In addition to using the checklists provided in this module for management assessment, the service quality assessment checklists in Module 6 can provide an important source of information about the quality of management. Service quality measures can be thought of as outcomes of management activities and, therefore, often provide insights into the adequacy of the management system. This relationship is especially clear in the areas of supervision and training. Service quality checklists used during regular supervision can be reviewed by higher-level managers as a source of information about service quality and performance. For example, if the checklists filled out by a supervisor indicate service delivery problems but do not record corrective action taken, this might be identified as a problem area in supervision by a higher-level supervisor. Following are some examples of how service quality checklists can be used in supervision: Secondary supervisors can conduct quality assessments in the responsibility areas of their subordinate supervisors. The results measure the outcome and effectiveness of first-level management/supervision. Secondary supervisors can use checklists to carry out supervision jointly with first-level supervisors. This will allow the second-level supervisor to assess directly how well a supervisor identifies and solves specific problems. Checklists can be used as a basis for a supervisory interview or performance review. Service quality checklists can be used as a reference for the development of detailed training assessment tools. Exhibit 3 shows a training assessment tool that combines the detail of the service quality checklist for ORT with the training quality criteria included in the management checklist. The asssessment tool could be used during observation of training or to interview participants after training. Subsequent supervision with service quality checklists could determine which training topics were not communicated effectively, and the training content or methods in that area could be adapted accordingly. EXHIBIT 3: QUALITY ASSESSMENT ORT training This is an example of how service delivery checklists could be adapted as tools for detailed assessment of a training course on ORT. Medical history E=explained, D=demonstrated, PC=practised, T=competency-based testing, MT=included in take-home material, MN= included in manual. E D PC T MT MN Duration of diarrhoea? E D PC T MT MN Consistency of stools? E D PC T MT MN Frequency of stools? E D PC T MT MN Presence of blood in stools? E D PC T MT MN Presence of vomiting? E D PC T MT MN Fever? E D PC T MT MN Home treatments? Physical examination E D PC T MT MN Assess general status (alert or lethargic)? E D PC T MT MN Pinch skin, examine fontanel and mucus membranes? E D PC T MT MN Weigh child? E D PC T MT MN Determine nutritional status to be sure the child is not severely malnourished? E D PC T MT MN Take temperature? Classification and treatment E D PC T MT MN Determine the degree of dehydration (none, moderate, severe)? E D PC T MT MN Prescribe safe ORS solution? E D PC T MT MN Recommend safe home treatment with ORS solution? E D PC T MT MN Refrain from using antibiotics, except when stools contain blood or mucus? E D PC T MT MN Refrain from using anti-diarrhoeals? E D PC T MT MN If child is dehydrated, administer ORS solution immediately or refer child for ORT? E D PC T MT MN Give sufficient amount of ORS solution? E D PC T MT MN Plan to reassess child's hydration status after an appropriate interval? E D PC T MT MN If dehydration is severe, rehydrate with intravenous fluid or naso-gastric tube? E D PC T MT MN If IV or NG tube are not available within 30 minutes of facility, try ORT? E D PC T MT MN If child cannot drink, refer/evacuate for IV, NG treatment? ORT education E D PC T MT MN Tell mother to give extra fluids during diarrhoea? E D PC T MT MN Tell mother how to prepare ORS solution? E D PC T MT MN Tell mother how much ORS to give and how often to give it? E D PC T MT MN Tell mother about appropriate feeding practices during and after diarrhoea? E D PC T MT MN Tell mother about at least signs of dehydration? E D PC T MT MN Discuss at least danger signs that indicate need for further medical attention? E D PC T MT MN Show mother how to prepare ORS solution? E D PC T MT MN Verify that mother understands key information? E D PC T MT MN Ask mother if she has any questions? Strengths and limitations of the module It is important to note a few limitations of the module. First, effective use of the tools provided in the module depends on a management environment that is open to frank assessment of programme strengths and weaknesses. Second, those involved in the assessment must have knowledge of the process under discussion. For this reason it is very important to include the staff members who actually carry out the task in the assessment. Finally, there are limits to the extent to which management assessment can be quantified, thus, the staff members are still dependent on qualitative information to a considerable degree. Even when performance can be quantified, results cannot be generalised because they flow from a specific management system with unique features. In spite of these limitations, the management assessment methods posed here are objective and concrete, and offer considerable advantages over many types of performance review and needs assessment methods currently in practice. The process of assessment also provides an opportunity for communication and a chance for a team to think together and gain insights about management. Finally, the identification of problem areas or opportunities for improvement is sometimes enough to lead to better performance. A management assessment is an effective way to direct staff attention to the importance of management and to help each member see opportunities to improve the system. Management assessment procedures This section describes an eight step process of designing and carrying out a management assessment. Step 1: Specify the objectives of the management assessment Step 2: Determine assessment methods Step 3: Select and adapt appropriate checklists or guidelines Step 4: Carry out the assessment Step 5: Tabulate and analyse the data Step 6: Present and review the preliminary results Step 7: Analyse the management process in more detail Step 8: Take action to improve management Step 1: Specify the objectives of the management assessment The general objective of management assessment is to identify strengths and weaknesses and to improve management which, in turn, will improve overall programme effectiveness. In planning a management assessment it is up to the manager or PHC team to determine the specific objectives and scope of the assessment. These decisions are important because they have implications on design, analysis, and documentation. The following questions should be addressed at the outset: What is the purpose of the assessment? The purpose of the assessment might be, for example, needs assessment in a particular management area, with the aim of identifying ways that the system could be improved. The assessment might also be used to assess the performance of an individual, a team, or a system. What management activities will be included? The selection of a management activity for assessment should be based on priorities identified by managers and the PHC team. Results from routine monitoring, service quality assessment, or other evaluations could be taken into account. The focus of the assessment could be an entire management system, for example, planning, or it could focus on a sub-task within planning, for example, the mission statement. Who will use the information gathered? The information generated from a management assessment can be used by local managers, PHC teams, health committees, the board of directors, regional or national directors, and donors. Who will use the information will have an impact on what information is collected, how much is collected, and how it is reported. On the one hand, a report that is intended for use by national directors and donors might require a large statistical sample and a formal report as an end product. A local manager, on the other hand, might use a small convenience sample and document the conclusions in a supervisory notebook. How will the information be used? The information collected can be used in a variety of ways. It can be used by programme staff to develop management improvement programmes and to better understand the root causes of service delivery problems. It can also be used to focus training efforts on the actual needs experienced in the programme. The specific use of the information will have implications on what is collected and how it is analysed. What will be the unit of analysis? What health facilities will be included? The appropriate unit of analysis for management assessment will depend on the objectives of the assessment and the nature of the management function in question. Several options are discussed below: Single health facility. In many cases the health facility will be the unit of analysis. Such management assessments often involve a study of management processes for only one health facility, with the aim of making improvements at that level. This type of assessment permits in-depth analysis of the local situation, participation of staff, and locally appropriate solutions. A group of health facilities. Management processes for a group of health facilities of an area or region may also be of interest. Managers may want to study the same management process in a group of facilities to identify systematic management problems that will require intervention from the area or region for resolution. Outputs related to the management function. Each management system has outputs which may also be a unit for analysis. For example, in the training system, outputs might be training courses or trainees. These could be examined as a way of assessing the performance of the training system. Over what period of time will the activities take place? Managers must consider the above decisions determining the time and duration of management assessment activities. Short rapid assessments may be employed, or more in-depth studies may be undertaken. Also, some activities may be carried out on an ongoing basis, while others will be done once, or on an ad hoc basis. Are adequate resources available for the assessment? Ideally, management assessment activities should be carried out with resources that have been allocated to management and supervision. However, additional costs, such as transportation, per diem, and supplies may be needed. In designing studies, managers should assess whether additional resources are available and keep in mind the importance of both the financial and technical feasibility of carrying out assessments. Step 2: Determine assessment methods Quantitative or qualitative assessment? This module presents two different approaches to management assessment. The use of PHC MAP checklists permits a comprehensive, quantitative assessment. This will help you to describe the management system and its problems. Some of the problems will be easy to solve as a result of the assessment, others will leave the team with many questions. Why do these problems occur? How can they be corrected or prevented? These questions are too complex to be answered with a checklist alone. They require a less structured format that allows people to share their insights and experiences. The PHC MAP discussion guidelines are better suited to this kind of exploratory qualitative assessment. WORKSHEET FOR PLANNING MANAGEMENT ASSESSMENT Step 1. Specify the objectives of the assessment a. What is the purpose of the assessment? To ensure a steady supply of drugs in the health centre b. What is the scope of the assessment? Management activity(ies): Logistics Sub-tasks: Procurement Inventory control c. Who will use the information gathered? Pharmacy, director of health centre d. How will the information be used? To determine if drug supplies are adequate and, if not, to identify changes that should be made in the system e. What programmes or facilities (units) will be analysed? 1 health centre f. Over what period of time will the activities take place? 6 months g. Are resources adequate? Needs are minimal Step 2. Determine assessment methods a. Unit of observation: Pharmacy (no. days when drugs are not available and the percentage of patients who receive all needed medications) b. Data sources: Discussion with health centre staff, including the person who manages the pharmacy. c. Sampling methods: 100% sample for last six months Step 3.Choose relevant checklists: Logistics - procurement (5-15); inventory (24-39) Modification required: yes Need to adapt questions so that they relate to drug supply. Need to add questions to measure the number of days when drugs were available for essential drugs, and to measure the percentage of patients who received the medicine needed. Managers may choose the structured checklist approach or the more flexible discussion to assess their management performance. This decision will depend on the objective of the management assessment and, to some extent, on the style of the manager. The two approaches can also be combined, so that the team can benefit from the advantages of both methods. For example, a manager or PHC team might first apply the checklists to get a general idea of how things are going. Based on these results, a group discussion could be held to assess the situation in more depth and from different points of view. Information sources. Information about management services can be collected from direct observation of job performance or inspection of the health facility. A review of programme records is another source of information. Interviews or surveys may also be helpful. These could be carried out with individuals or groups and could be open or close ended. In choosing information sources, consider the reliability of the information, the time it will take to collect it, and the cost involved. Step 3: Select and adapt appropriate checklists orguidelines While the checklists and discussion guidelines may be used as they are, they will usually require some adaptation. Field tests have shown that it should take no more than 2-3 hours to revise these tools. Adaptation can include rephrasing questions, omitting items that are irrelevant, revising, rearranging, rephrasing items to fit local conditions, and adding items that are important to your programme. In some cases you may want to combine items or sections from several checklists to form a new one. Several managers have found it useful to make these changes in a group session that involves the users, who are usually supervisors and administrative staff. Group work helps to ensure that the key users become familiar with the instruments and procedures and develop a sense of "ownership" of the assessment. It is also quicker. When an instrument is prepared by one person it is often circulated for comment, revised, and recirculated several times. This is often unnecessary if the assessment is carried out by the group. Step 4: Carry out the assessment After the assessment has been designed and the tools have been adapted, the manager must take steps to carry out the assessment. Before the assessment the manager should review the management checklists or discussion guidelines that they will use to ensure that they are comfortable with the format, procedures, and content. The group discussion. If group discussions are to be held, the manager should give participants as much notice as possible and provide them with a brief description of the purpose of the meeting and the kind of information they will be asked to provide. This allows staff to prepare by thinking about the topic beforehand and conferring among themselves. Also, this will afford participants time to gather information to illustrate their points of view, if they so desire. The ideal size for a group discussion is 5-10 people. Each group discussion should begin with a brief introduction and explanation of the purpose of the discussion. Also, the person who leads the discussion should explain the ground rules to the group before the session. This is especially important if the group members have held other kinds of meetings together. They should be reminded that this meeting has a different purpose and structure. The following ground rules will help the group stay on course: Ground rules for group discussion 1 Everyone's ideas and opinions are important. There are no right or wrong answers. Both positive and negative comments are welcome. Participants should feel free to disagree with one another so that all points of view are heard Don't wait to be called on; it's a group discussion. 1 Adapted from Debus, M., Handbook for excellence in focus group research. Academy for Educational Development, HEALTHCOM, Washington, DC, 1992. After the ground rules have been presented, the leader can use the discussion guidelines to lead the discussion on a given management topic. The leader of the group discussion can refer to the checklists if there is a need to probe the issue in more detail. The yes/no type questions from the checklists can be easily modified to encourage discussion. For example, if controversial issues are raised with a yes/no question, an adversarial or combative tone may develop, leading some group members to refrain from contributing. Rather than asking a group of yes/no questions, such as, "Do staff understand the programme mission?", try a more neutral wording, such as, "Tell me what you understand about the programme mission." or, "How well do you think the mission statement is understood?". Another alternative to start a constructive discussion is to use two questions, such as, "What aspects of the mission statement are understood well?" and, "What aspects of the mission statement are not understood well?" The group leader should keep track of the time and guide the group to new topics when it seems that enough information in a specific area has been shared. Before moving to a new topic or question, the leader should ask the group if anyone has any final comments to add. At the end of the discussion, the leader should thank the group for participating and give everyone in the group one last opportunity to comment on the overall topic. The leader should write a summary of the discussion, organised by topic, soon after the discussion so that key issues will not be forgotten. This information will be useful for identifying management problems, their causes, and possible solutions. Visits to the health facility. If field visits are required to meet with teams or individuals, arrangement for transportation will have to be made. The schedule of regular activities at the facility should be taken into account so that the assessment does not interrupt important activities. Yet, some managers may want to make unannounced visits so that staff do not make special preparations for the assessments. If observation is to be carried out, it is usually a good idea to explain what the observer will be doing and why, so as to avoid misunderstanding and anxiety. Experience has shown that most workers want to improve their performance and welcome constructive help and advice. But they will also resist co-operating when they suspect that gathered information will be used against them. Ideally, assessments should be carried out in a supportive, constructive manner to help staff improve programme management. After the assessment, the checklist should be reviewed to be sure that all questions are answered and that all answers are clear. Step 5: Tabulate and analyse the data The quantitative tabulation and analysis procedures suggested here are similar to those described in Module 6. Supervisors and managers can hand tabulate the data, often in minutes, depending on the number of observations. The information gathered during an assessment may be tabulated using a simple matrix. Data is transferred from observation checklists to the matrix, and can be summed up easily. If multiple observation checklists are used, the tabulation can be done on the checklist, eliminating the need to transfer the information to the tally sheet. Exhibit 4 on the following page shows how a tally sheet was used to record and tabulate 39 observations of the quality of the PHC drug supply system in ten health centres. The question numbers correspond to those of the Logistics management checklist in Appendix B. The results show that there are some items that are either done by everyone (e.g., No. 8: all health centres have a clear policy about who can authorise requests) or by no one (e.g., No. 9: no one seeks multiple cost estimates). Clearly, from this analysis, a number of items need attention. It can be easier to understand these numbers if they are displayed graphically, such as in Exhibit 5. In this graph the bars represent the number of health centres that responded "Yes" to an item. The longer the bar, the greater the number of positive responses, and vice-versa. The manager may wish to set a minimum standard, or threshold, for the overall programme. For example, that at least 6 centres respond positively to an item, otherwise, it is below standard and deserves immediate attention. Those programmes where no centre responds "Yes", deserve priority attention. Rapid data analysis plan. If a large number of observations are going to be made, it can be very helpful to develop an analysis plan. Calculations such as the following can be carried out as warranted. Total: Sum the number of observations for each variable (i.e., each management service task). Frequency distributions or counts: Add up the number of "yes" and "no" responses for each variable (task) and compute the percentage distribution: Take the number of each count ("Yes" and "No") and divide it by the total number of observations x 100. Threshold analysis: Set a minimal acceptable level for each item. Those which exceed that level are identified as problems. Exhibit 5: Drug supply: 10 health centres Scoring: In addition to analysing data by specific service tasks, a scoring system can be developed to assess overall quality. For example, each task that was carried out could receive a score of one point. The total "quality score" would be the total number of points for a set of tasks compared with the maximum possible score. If some tasks are much more important than others, they can be given added weight (e.g., 1.5 or 2 points). Breakdown by site: Compare one site or session with another. Totals, counts, and percentages can be calculated for each site. This will help managers to identify strengths and weaknesses at different sites. Graphs: Results can be plotted on a graph to give a manager or supervisor a summary of the data. This is a useful visual aid for presenting information. Exhibit 5 shows an example of such a graph. Step 6: Present and review preliminary results The results of a management assessment are likely to take the form of a list of problems, potential problems, or areas for improvement. The findings of the assessment should be summarised and reviewed with all those who participated in the assessment. Often, such information sharing leads to further insight into the nature of the problem. This information not only makes the assessment more accurate, it is invaluable for managers at the point when they attempt to develop plans for improvement. Here are some guidelines for presenting and reviewing findings with the team: Which activities were carried out well? Manager should begin by reviewing what staff members did well. This reinforces good performance and establishes a constructive rapport. Which activities need improvement? Areas that need improvement should then be reviewed, providing as much specific information as possible. Some problems will be associated with quantitative data, others will be described in more qualitative terms. Which problems can be easily corrected? Some problems are easy to correct, have obvious solutions, and require little extra effort to do correctly. Managers should act quickly on these by exploring with staff ways that corrections can be made. Experience has shown that those changes will be more acceptable and more likely to be implemented if they are suggested by the staff. The manager should encourage staff to take the initiative to make the corrections. In these cases further analysis (Step 7) will not be necessary. The team can take immediate action for improvement. Which problems should be treated as priorities? Management assessment may identify more problems than a manager or management team can realistically solve. Managers and their staffs should define the criteria they will use to decide where to take action. For example, they may choose to focus on problems that pose high risks to patients or staff, or problems that occur frequently because of the number of people affected. Which problems will be more difficult to correct? These problems may need to be analysed more formally by a problem-solving group or special study. The activities of the group would include in-depth problem analysis of the management process. Step 7: Analyse the management process in more detail Difficult problems require in-depth analysis, based on the experience and insight of all those involved in the process. This section presents 3 analytical techniques that can help teams work together to understand management problems and their causes. It is very important to find the root causes of the problem so that the action taken is a true solution rather than a superficial fix. Beginning with thorough analysis, the PHC team should be better able to develop a solution that is appropriate, effective, and acceptable to all those involved. This process might be done at the time of presentation of preliminary results, or during a separate meeting. Ask "why?" five times. One simple technique for understanding problems better is to search for the root cause by asking "why" five times. Too often we focus on the first cause of a problem, trying to solve it before asking about other causes. Thus, deeper problems go undetected. While there is nothing magic about the number five, repeatedly asking why will lead the team to find a number of causes, some that can be addressed, others that cannot. Based on this thorough analysis of the situation, they are better able to decide where and how to take action. For this technique to be effective curiosity is required. Participants must look at a familiar problem as if it were new, so that they may see it in all its complexity. It also requires openness; rather than accepting the familiar explanations, the team, and especially the manager, must be open to new ideas that will come from the group. A third important element is participation. All those who are involved in the process should be involved or represented in the analysis. Finally, listening during this process is very important. Participants will be more likely to share their thoughts if they are in a considerate, listening environment. Fishbone diagram. Another tool for the analysis of causes and the identification of root causes is a cause and effect diagram, commonly called a fishbone diagram. This method is useful because it organises information from a variety of sources, graphically representing the situation in a way that is easy to comprehend. Generally, the major categories of causes (such as human resources, equipment, facility and supplies, etc.) are listed on the major branches or "bones", and all the possible causes related to that category are listed there. The objective of the exercise is to look for the most likely root causes of the problem and to try to reach consensus about which causes of the problem should be the focus of a management improvement effort. An example of a fishbone diagram dealing with supervision is shown on the following page. Process flow chart. A process flow chart is another analytical tool that can provide a great deal of insight into the management process. The flow chart can be used to map out how a process is actually functioning. This helps the team to understand the process better and to find ways to streamline or improve it. Very often after making a flow chart of a "familiar" process, team members will comment that they hadn't really understood how the process was working until they charted it. The flow chart can also be used to map out the process as it should be. The following exhibit on page 33 is an example of a flow chart of the process for ordering drugs. Step 8: Take action to improve management Management assessment and analysis is not useful unless it is followed by action for improvement. Based on the findings of the assessment and subsequent analysis of causes, the team should be able to identify and implement ways to improve the quality of those tasks that are not done well. The PHC MAP manager's guide on problem-solving provides some guidelines and experiences that may be helpful. Also, the PRICOR Operations Research manuals describe procedures for designing and conducting studies to develop and test solutions to operational problems in PHC. <$FBlumenfeld, S., Operations research methods: a general approach to primary health care. PRICOR monograph series, methods paper 1. Bethesda, MD, Center for Human Services, 1991.> After action has been taken to solve a problem, it is important to confirm that it was implemented correctly and that it had the desired impact. At the time when a plan is made to take action, a plan to evaluate the effectiveness of that action should be developed. It may be a very simple check, carried out by the manager or a staff member, but it is an essential element in management improvement. If the problem has not been solved, further analysis will be needed to explore why, and a modified strategy for improvement should be developed. Once the problem is solved and the solution has become a standard part of programme operations, the PHC team can move on to another management problem in an effort to improve their programme continuously. Exhibit 6: Fishbone diagram Technical Issues Interpersonal Issues Logistic/ Geographic Issues Organisational/ Environmental Issues Supervision is not effective in improving programme performance Supervisors don't know programme norms Supervisors have no training in supervision Low morale Supervisors don't understand Lack of trust Lack of communication skills Transportation is not available Gas is not available Per diems are too low Staff unwilling to travel Terrain is rugged Supervisors and supervisees are dispersed Supervisory relations are not clear Supervisors have no job description No regular supervisory schedule Supervisors focus on administrative tasks rather than support/evaluation of programme performance Exhibit 7: Flowchart: Ordering drugs Appendix A: Management assessment discussion guidelines Planning, personnel, training, supervision, finances, logistics, information, and community organisation Planning "Health planning is the process of defining community health problems, identifying needs and resources, establishing priority goals, and setting out the administrative action needed to reach those goals." Introduction: Welcome the group and briefly give the purpose of the discussion. Ground rules: Explain the following ground rules to the group: Ground rules for group discussion Everyone's ideas and opinions are important. There are no right or wrong answers. Both positive and negative comments are welcome. Participants should feel free to disagree with one another so that all points of view are heard. Don't wait to be called on; it's a group discussion. Please speak one at a time. These discussion guidelines are provided to help you to lead a group discussion about your planning system. You may also refer to the detailed version of the management checklists (Appendix B) as a resource for the discussion. Discussion What works well in the way we do planning? Guidelines What aspects of the way we do planning do not work well? What is your mission? How well is it understood by staff, managers, donors, and community members? What are the programme objectives? What kinds of information do you need to plan your activities? Is this information available to you? How well are your job description and assignments understood by staff? Key questions Is there a written mission statement that describes the purpose and overall goals of the organisation? Does the programme plan directly address the overall programme mission? Are there specific objectives and programme(s) or activities for each goal? Are there clear referral policies for services not offered, secondary, and tertiary care? Do you have access to the basic information needed for programme planning? Is there an organisational chart or plan that clarifies the responsibilities of staff members and the relationships between them? Are there written technical norms for all services included in the programme? Does each staff member have a clear job description? Are work plans and staff assignments regularly reviewed and updated? Personnel management Personnel management includes activities that relate to recruiting, hiring and supporting staff, and defining the roles of each individual. It also refers to standard procedures related to compensation, benefits, and work schedules. Introduction: Welcome the group and briefly give the purpose of the discussion. Ground rules: Explain the following ground rules to the group: Ground rules for group discussion Everyone's ideas and opinions are important. There are no right or wrong answers. Both positive and negative comments are welcome. Participants should feel free to disagree with one another so that all points of view are heard. Don't wait to be called on; it's a group discussion. Please speak one at a time. These discussion guidelines are provided to help you to lead a group discussion about your personnel management system. You may also refer to the detailed version of the management checklists (Appendix B) as a resource for the discussion. Discussion What works well in the way we do personnel management? Guidelines What aspects of the way we apply personnel management do not work well? Are staff members satisfied with their jobs and work environments? What are the sources of satisfaction? What are the sources of dissatisfaction? What personnel procedures and policies are in place in your programme? Are these policies fair? Do they contribute to the effectiveness of the programme? What areas need to be addressed through new or improved policies? Do the job descriptions of staff members adequately reflect their actual day-to-day activities? How do supervisors work with staff members to plan activities and improve performance? What other things could supervisors do to support staff members in this way? Key questions Are there enough qualified staff members available? Do opportunities exist for promotion and professional development of staff? Does the system offer adequate job security to staff? Is there good communication between staff at different levels? Is communication between different departments good enough to co-ordinate and plan effectively? Is staff morale high? Does each staff member have a job description? Do job descriptions list all types of tasks that will be required of the staff member? Do staff members feel that their job descriptions are realistic? Do supervisors develop specific work plans with their staff at regular intervals? Do supervisors periodically review whether work assignments are completed? Training Training is the process of continually improving the knowledge, skills, and competencies of health workers. Introduction: Welcome the group and briefly give the purpose of the discussion. Ground rules: Explain the following ground rules to the group: Ground rules for group discussion Everyone's ideas and opinions are important. There are no right or wrong answers. Both positive and negative comments are welcome. Participants should feel free to disagree with one another so that all points of view are heard. Don't wait to be called on; it's a group discussion. Please speak one at a time. These discussion guidelines are provided to help you to lead a group discussion about your training system. You may also refer to the detailed version of the management checklists (Appendix B) as a resource for the discussion. Discussion What works well in the way we do training? Guidelines What aspects of the way we do training do not work well? What kind of information do you use to plan or focus training? Do you use information about health worker knowledge, attitudes, and practices? Do you use information about community needs? Is this information available to you? What training methods are used? Are they used correctly? Are they effective? Is the content of the training appropriate for local norms? What resources are available to support training? What resources are needed? Key questions Is programme specific information (from MIS or supervisor) about service quality used to plan or focus the training? Did the training include a pre-test of health worker knowledge and skills? Does training include a post-test of health worker knowledge and skills? Is there a follow-up assessment of training (about six months later) to assess the impact of the training? Are training topics explained clearly? Are training topics demonstrated? Do trainees receive written materials? Are health workers satisfied with the training they receive? Will training be followed up through the regular supervision system? Will trainee performance in new skill areas be monitored through MIS? Is the training content complete and accurate according to local norms? Are resources adequate? Supervision Supervision is the process of directing and supporting staff so that they may perform their duties effectively. Supervisors provide support, guidance, training, and assistance in the identification and solution of problems, so that service quality and effectiveness may continually improve. Introduction: Welcome the group and briefly give the purpose of the discussion. Ground rules: Explain the following ground rules to the group: Ground rules for group discussion Everyone's ideas and opinions are important. There are no right or wrong answers. Both positive and negative comments are welcome. Participants should feel free to disagree with one another so that all points of view are heard. Don't wait to be called on; it's a group discussion. Please speak one at a time. These discussion guidelines are provided to help you to lead a group discussion about your supervision system. You may also refer to the detailed version of the management checklists (Appendix B) as a resource for the discussion. Discussion What works well in the way we do supervision? Guidelines What aspects of the way we do supervision do not work well? Describe your supervisory system? Who does it? How? Do staff and supervisors feel that supervision contributes to programme improvements? What kinds of activities are carried out by supervisors during supervisory visits? How does the supervisor assess the quality of the technical services delivered? How does the supervisor work with supervisees to identify and solve problems? Do supervisors have a good rapport with staff? Why or why not? Do supervisees feel free to discuss problems with their supervisors? How are supervisory activities recorded? Are supervisory records used for follow- up, problem solving, or to help you in planning? Key questions Is there a plan or schedule for supervisory activities? Do supervisors use the following methods during supervision: Observation of service delivery? Asking the service provider about what problems she/he has been having? Review of records, supplies, or the conditions of the facility? Do supervisors assess technical service quality through observation of service delivery or competency-based testing? Do supervisors make comments aimed at improving technical service quality? Do they make comments aimed at improving counselling or health education? Do they demonstrate any technical skills to the supervisee? Do they establish a good rapport with the supervisee? Do supervisors help their supervisees to organise and plan their work? Are supervisory records kept? Financial management Financial management includes management of and accountability for programme finances, budgeting for planned activities, and, in some programmes, cost-recovery or fund-raising. Introduction: Welcome the group and briefly give the purpose of the discussion. Ground rules: Explain the following ground rules to the group: Ground rules for group discussion Everyone's ideas and opinions are important. There are no right or wrong answers. Both positive and negative comments are welcome. Participants should feel free to disagree with one another so that all points of view are heard. Don't wait to be called on; it's a group discussion. Please speak one at a time. These discussion guidelines are provided to help you to lead a group discussion about your financial management system. You may also refer to the detailed version of the management checklists (Appendix B) as a resource for the discussion. Discussion What works well in the way we do financial management? Guidelines What aspects of the way we do financial management do not work well? Are your financial records clear, easy to understand, and correct? How do you use the programme budget? How do budgeted and actual expenses compare? How are regular financial reports used and for what purposes? How do you take sustainability, equity, and growth into account in the financial planning process? Key questions Does the chart of accounts meet the current needs of the programme? Are accounting records kept up to date and balanced monthly? Are the accounts legible? Are the calculations correct? Are all transactions entered in the ledger? Is there a programme budget? Are there established procedures for review and approval of budgets? Are the budgets used as a tool for evaluating performance on a planned versus actual costs basis? Are regular financial reports prepared? Does the administrator evaluate results by making annual comparison of "budget" to "actual" financial performance? Is sustainability considered in the financial planning process? Is equity considered in the financial planning process? Is growth considered in the financial planning process? Logistics management Logistic systems deal with the procurement, storage, and tracking of supplies and equipment. Introduction: Welcome the group and briefly give the purpose of the discussion. Ground rules: Explain the following ground rules to the group: Ground rules for group discussion Everyone's ideas and opinions are important. There are no right or wrong answers. Both positive and negative comments are welcome. Participants should feel free to disagree with one another so that all points of view are heard. Don't wait to be called on; it's a group discussion. Please speak one at a time. These discussion guidelines are provided to help you to lead a group discussion about your logistics system. You may also refer to the detailed version of the management checklists (Appendix B) as a resource for the discussion. Discussion What works well in the way we do logistics? Guidelines What aspects of the way we do logistics do not work well? What procedures are in place for purchasing goods or services? Are these procedures followed? Are they effective? How are goods stored and monitored? What procedures are in place to determine what supplies are needed? Are these procedures followed? Are they effective? What is the procedure for assuring the maintenance of the cold chain? Are these procedures followed? Are they effective? What transportation is available for staff? Is it adequate? Key questions Does the institution have its own established checklists and procedures for the procurement of goods and services? Before placing an order for goods and services, does the administrator review the order to ensure that the item(s) are needed? Are supplies ordered regularly according to the defined procedures? When procedures are followed, are supplies received on time? Does the facility or organisation have established reception procedures? Does the institution have established procedures for inventory control? Is the inventory maintained according to the defined procedures? Is the storage area protected (from water, dust, etc.)? Are equipment and supplies labelled clearly and organised in an accessible way? Is there a PHYSICAL inventory taken to verify theoretical stock levels with actual PHYSICAL counts? Does the staff have access to transportation? Is the fuel available adequate for field visits and other off-site activities? Is there a thermometer or cold chain monitor in the refrigerator? Information management An information system is organised around key indicators that measure a programme's progress toward its goals. It is a systematic way of collecting, reporting, and using data at all programme levels. Introduction: Welcome the group and briefly give the purpose of the discussion. Ground rules: Explain the following ground rules to the group: Ground rules for group discussion Everyone's ideas and opinions are important. There are no right or wrong answers. Both positive and negative comments are welcome. Participants should feel free to disagree with one another so that all points of view are heard. Don't wait to be called on; it's a group discussion. Please speak one at a time. These discussion guidelines are provided to help you to lead a group discussion about your management information/monitoring system. You may also refer to the detailed version of the management checklists (Appendix B) as a resource for the discussion. Discussion What works well in the way we do information management? Guidelines What aspects of the way we do information management do not work well? What kind of information do you routinely collect? Is this information useful? How do you use it? What kind of information do you give to your supervisor or manager? Does the supervisor use the information collected? Are your information needs met by the current system? What kind of information would you like to have that is not collected at present? Of the information you collect now, what would you eliminate (discuss the types of information that you don't use)? Key questions Is there a list of indicators to be monitored at the district-level? Is there a list of indicators to be monitored at the health centre-level? Do these lists include indicators of Resource availability? Access? Utilisation? Coverage? Service quality? Outcome? Have information sources been identified for each indicator? Has the frequency of collection/compilation been established for each indicator? Has an analysis procedure for each indicator been established (including thresholds or standards)? Have mechanisms for interpreting and discussing results been established? Do managers prepare and transmit reports of monitoring results to the appropriate level in a timely manner? Do managers utilise information to identify areas for improvement of district-wide and local problems and strengths for specific interventions? Community organisation Community organisation is the involvement of the community in the design, planning, promotion, or delivery of health enhancing activities. Introduction: Welcome the group and briefly give the purpose of the discussion. Ground rules: Explain the following ground rules to the group: Ground rules for group discussion Everyone's ideas and opinions are important. There are no right or wrong answers. Both positive and negative comments are welcome. Participants should feel free to disagree with one another so that all points of view are heard. Don't wait to be called on; it's a group discussion. Please speak one at a time. These discussion guidelines are provided to help you to lead a group discussion about community organisation. You may also refer to the detailed version of the management checklists (Appendix B) as a resource for the discussion. Discussion What works well in the way we do community organisation? Guidelines What aspects of the way we do community organisation do not work well? Describe the communities served by the programme. What ethnic groups are represented? What are their health needs? Describe the role of women. What are the implications for health? What are the needs of socially disadvantaged groups? In what ways does the health committee or community organisation support the health centre and its activities? Is the community organisation strong and active? How do you know this? What kinds of activities are carried out by the community in support of the health activities? Do community members provide leadership in the health area? Key questions Is there a health committee or community organisation affiliated with the health centre? Does the community organisation meet regularly? Are meetings led by a member of the community? Are leaders chosen by the community? Are decisions made in a way that is satisfactory to group members? Are women involved in community organisation ? Does the community organisation have a good rapport with local officials? Are disadvantaged groups adequately represented? List under-represented groups Are the needs of socially or economically disadvantaged groups addressed by the health committee or community organisation? Has the group undertaken activities in the past year? Are the activities clearly visible in the community? Does the community help to implement activities? Does the community play a leadership role in health activities? Does the community effort receive adequate technical assistance from local, district, and outside agencies? Appendix B: Management assessment checklists Planning, personnel, training, supervision, finances, logistics,information, community organisation Planning "Health planning is the process of defining community health problems, identifying needs and resources, establishing priority goals, and setting out the administrative action needed to reach those goals." <$F On being in charge, WHO 1992.> This checklist is intended for use in the assessment of planning activities. Its objective is to help managers to enhance the quality of their programmes by identifying and resolving problems in the planning area. It can be adapted for use in both vertical programmes and integrated PHC efforts. The questions below can be answered through interviews, document review, observation of management activities, or a review of the health facility. Some questions ask for the respondent's opinion about the quality or adequacy of specific tasks; in such cases, the perspectives of managers, health service providers, and community members should be taken into account. Areas that are deemed inadequate can be further explored through focused discussions with key informants. With modification, the checklist can be used as a basis for an open-ended interview or group assessment. 1. Health facility 2. Service provider 3. Observer/Supervisor 4. Date Mission statement A mission statement describes the purposes and overall goals of an organisation. These questions will help you to determine whether the mission is clearly defined and understood by programme staff, the community, and donors. 5. YES NO Is there a written mission statement? 6. YES NO Does the mission statement include a clear primary health care strategy? 7. YES NO Is the mission statement understood by the health centre staff? 8. YES NO Is the mission statement understood by the community? 9. YES NO Is the mission statement understood by health centre management? 10. YES NO Is the mission statement understood by the board? 11. YES NO Is the mission statement understood by the donors? 12. YES NO Does the programme plan directly address the overall programme mission? Programme objectives Objectives are the specific results that are expected from a programme or activity. These questions will help you determine whether the objectives are defined adequately for the purpose of planning. 13. YES NO Is the geographic area to be served clearly defined? 14. YES NO Are the age groups to be served clearly defined? 15. YES NO Are high-risk groups to be served specified? 16. YES NO Are the services offered specified? 17. YES NO Are there explicit programme objectives? 18. YES NO Did the community play a role in setting goals and objectives? 19. YES NO Are the objectives reviewed and revised on a regular basis? 20. YES NO Do government programmes and/or donors understand and agree with the goals and objectives? 21. YES NO Are coverage objectives for each service stated and quantified? 22. YES NO Are coverage objectives understood by staff? 23. YES NO Are objectives for service quality stated and quantified? 24. YES NO Are service quality objectives understood by staff? 25. YES NO Are objectives for improvements in knowledge in specific PHC areas stated and quantified? 26. YES NO Are objectives for improvements in knowledge understood by staff? 27. YES NO Are objectives for behaviour change in specific PHC areas stated and quantified? 28. YES NO Are behaviour change objectives understood by staff? 29. YES NO Are objectives for health status improvement stated and quantified? 30. YES NO Are health status objectives understood by staff? 31. YES NO Does the programme include integration of PHC service delivery? 32. YES NO Are programme staff aware of other PHC services offered in the area? 33. YES NO Does the programme avoid duplication of activities through communication or collaboration with these services? 34. YES NO Are there clear referral policies for services not offered, both secondary, and tertiary? 35. YES NO Is there a clear policy about community involvement? 36. YES NO Is programme evaluation included in the plan? Information needed for programme planning The following questions will help you to assess whether the programme has access to the basic information needed for programme planning. 37. YES NO Do you know how many people overall are to be served through the programme? 38. YES NO Do you have a map of the service area? 39. YES NO Do you have a breakdown of the age and sex of the population? 40. YES NO Do you have current information about the health status of the population (birth rate, IMR, CMR, MMR, and causes of death)? 41. YES NO Do you have current information about the priority health needs of the population? 42. YES NO Do you have information about the economic status of the population? 43. YES NO Do you have information about the educational level of the population (literacy, what languages are spoken)? 44. YES NO Do you have information about the ethnic composition of the area? 45. YES NO Do you have information about key health practices for the ethnic groups served by your programme? 46. YES NO Do you know current levels of coverage for specific services? Programme planning The following questions will help you to assess whether the organisational structure, schedule, and planning activities are adequate. Organisational structure 47. YES NO Is there an organisational chart or plan that clarifies the responsibilities of staff members and the relationships between them? 48. YES NO Are there written technical norms for all services included in the programme? 49. YES NO Are service delivery strategies for each service made explicit i.e. through clinic sessions, household visits, mobile units? 50. YES NO Is there an explicit strategy for educational activities? Scheduling 51. YES NO Is there a regular schedule of clinical sessions at the health facility? 52. YES NO Is there a regular schedule for clinical outreach activities? 53. YES NO Is there a regular schedule for household visits? 54. YES NO Is there a regular schedule for educational sessions at the health centre? 55. YES NO Is there a regular schedule for community outreach education? Other management activities 56. YES NO Is there a plan for providing training? 57. YES NO Is there a supervision plan? 58. YES NO Is there a system for procurement, storage, and transport of supplies (logistics)? 59. YES NO Does the plan include a management information system that monitors service delivery and management activities? 60. YES NO Is there a financial monitoring system? 61. YES NO Is there a budget for the activities planned for the year? 62. YES NO Is funding adequate for the activities planned in the coming year? 63. YES NO Is there a long-term funding strategy? Work planning for individual staff members The following questions will help you determine if staff members clearly understand their responsibilities. 64. YES NO Does each staff member have a job description? 65. YES NO Do staff members regularly receive specific staff assignments in major task areas? 66. YES NO Do staff members understand how their assignments are related to programme mission and objectives? 67. YES NO Are work plans and staff assignments regularly reviewed and updated? Personnel management 1. Health facility 2. Service provider 3. Observer/Supervisor 4. Date Personnel essentials The following questions address some essential personnel management issues that reflect the quality of the personnel management system. 5. YES NO Are there enough qualified staff members available? 6. YES NO Do opportunities exist for promotion and professional development of staff members? 7. YES NO Does the system offer adequate job security to staff members? 8. YES NO Are salaries and benefits adequate to attract competent well-trained staff members? 9. YES NO Are salaries and benefits adequate to retain competent and well-trained staff members? 10. YES NO Do staff regularly receive their salary on the days when they expect it? 11. YES NO Is staff turnover at an acceptable level (one which does not hamper the implementation of planned activities)? 12. YES NO Do staff members meet or exceed performance expectation? 13. YES NO Are staff members well-informed about the overall programme plans and objectives? 14. YES NO Is there good communication between staff members at different levels? 15. YES NO Is communication between different departments good enough to co-ordinate and plan effectively? 16. YES NO Is there a good rapport between staff members and clients or community members? 17. YES NO Is staff morale high? 18. YES NO Do staff members like their jobs? Policies and procedures The questions below will help you to identify personnel policies and procedures that need attention. Do written policies exist for: 19. YES NO Hours of work? 20. YES NO Vacation and holidays? 21. YES NO Overtime? 22. YES NO Benefits? 23. YES NO Salaries and raises? Are clear procedures followed for: 24. YES NO Recruiting staff? 25. YES NO Selecting staff? 26. YES NO Hiring staff? 27. YES NO Promoting staff? 28. YES NO Disciplining staff? 29. YES NO Firing staff? 30. YES NO Filing a grievance? 31. YES NO Are personnel actions and reviews Documented? 32. YES NO Are records consulted and used for management? Job descriptions The following questions will help managers and staff members assess the adequacy of their job descriptions. 33. YES NO Does each staff member have a job description? 34. YES NO Do job descriptions list all types of tasks that will be required of the staff member? 35. YES NO Does each job description list the training, knowledge, and skills required to carry out the job? 36. YES NO Do job descriptions include checklists about what proportion of time should be spent on each type of task? 37. YES NO Do managers respect the boundaries established by job descriptions? Do they refrain from asking their subordinates to perform activities that are not part of their job descriptions? 38. YES NO Do managers offer opportunities for professional development to their supervisees? 39. YES NO Are job descriptions revised as roles change? 40. YES NO Do staff members feel that their job descriptions are clear and complete? 41. YES NO Do staff members feel that their job descriptions are realistic? 42. YES NO Are staff members actively involved in defining roles and responsibilities? Work planning and performance assessment In addition to having clear job descriptions, health workers should schedule activities or plan work on a periodic basis. This area overlaps, to some extent, with supervision. The questions below deal with the organisation, structure, regularity, and definition of the activity. Specifics about methods are included in the PHC MAP checklists on supervision. 43. YES NO Do all staff members have an immediate supervisor? 44. YES NO Do all staff members know who their immediate supervisor is? 45. YES NO Do supervisors develop specific work plans with their staff at regular intervals? 46. YES NO Do supervisors periodically review whether work assignments are completed? 47. YES NO Do supervisors assess performance quality on a regular basis? 48. YES NO Do supervisors help staff members to set, review, and revise priorities? 49. YES NO Do supervisors regularly review the training needs of staff members? 50. YES NO Do supervisors take constructive action to improve performance? 51. YES NO Are supervisors and staff members satisfied with the level of responsibility they have? 52. YES NO Are supervisors and staff members satisfied with the authority (decision-making ability) they have? Training 1. Health facility 2. Service provider 3. Observer/supervisor 4. Date Training plan The following questions will help you assess whether the training plan is adequate. 5. YES NO Is there a plan for training activities? 6. YES NO Is programme-specific information (from MIS or supervision) about service quality used to plan or focus the training? 7. YES NO Do health workers participate in at least one training or refresher course every year (or two years)? Training methods A variety of training methods must be employed for effective training. These questions will help you assess whether the training methods used are adequate. 8. YES NO Did the training include a pre-test of health worker knowledge and skills? 9. YES NO Did the training address gaps identified in the pre-test? 10. YES NO Does training include a post-test f health worker knowledge and skills? 11. YES NO Are measures taken to address the gaps identified in the post-test, if necessary measures might include refresher course, individual attention, or intensive supervision? 12. YES NO Is there a follow-up assessment of training (about six months later) to assess the impact of the training? 13. YES NO Are training topics clearly explained? 14. YES NO Are training topics demonstrated? 15. YES NO Do participants have an opportunity to put new knowledge and skills into practice during training? 16. YES NO Is adequate time given to discussion and questions from participants? 17. YES NO Do trainees receive written materials? 18. YES NO Do trainees find the written materials they receive useful? 19. YES NO Do trainees have an opportunity to evaluate the training? 20. YES NO Are health workers satisfied with the training they receive? 21. YES NO Does training include a variety of methods? 22. YES NO Will training be followed up through the regular supervision system? 23. YES NO Will trainee performance in new skill areas be monitored through MIS? Training content The following questions will help you assess the adequacy of the training content. 24. YES NO Is the training content complete according to local norms? 25. YES NO According to local norms, is the training content correct? 26. YES NO Is the material presented appropriate to the skills, educational levels, and abilities of the trainees? Are staff members adequately trained in the following areas: 27. Household visits 28. Antenatal care 29. Delivery 30. Postnatal care 31. Child spacing 32. Immunization 33. Growth monitoring 34. Oral rehydration therapy 35. Acute respiratory infection 36. Malaria 37. Tuberculosis 38. Curative care 39. Water and sanitation 40. Planning 41. Training 42. Community organisation 43. Logistics 44. Financial management 45. MIS 46. Personnel Management 47. YES NO Is training for health workers adequate? 48. YES NO Is training for community health workers adequate? 49. YES NO Is training for supervisors and managers adequate? 50. YES NO Is training/orientation of health committee members adequate? Resources Good training requires the availability of resources to support the training, such as space, audio-visual materials, books, etc. These questions will help you to assess the adequacy of the available resources for training. 51. YES NO Is there a space adequate for training activities available to staff? 52. YES NO Are there audio-visual aids (blackboard, flip charts, or slide projector)? 53. YES NO Are there reference materials available to trainers? 54. YES NO Are there reference materials available to trainees? Training manuals 55. YES NO Is there an adequate supply of training manuals? 56. YES NO According to local norms, is the material in the training manual complete? 57. YES NO According to local norms, is the material in the training manual correct? 58. YES NO Is the material in the training manual appropriate for the type of health worker being trained? Take home materials 59. YES NO Is there an adequate supply of take-home materials for trainees? 60. YES NO According to local norms, are the take-home materials complete? 61. YES NO According to local norms, are the take-home materials correct? 62. YES NO Are the take home materials appropriate for the type of health worker being trained? Supervision 1. Health facility 2. Service provider 3. Observer/Supervisor 4. Date Supervisory schedule The following questions will help you assess whether the supervisory schedule is adequate. 5. YES NO Is there a plan or schedule for supervisory activities? 6. YES NO Are there written checklists or protocols for supervision? 7. YES NO Do supervisees meet with their supervisors at least every (no. of months or weeks per local policy)? 8. YES NO Does the supervisor have staff meetings every months/weeks? 9. YES NO When supervisory activities are cancelled, are they re-scheduled? 10. YES NO Are supervision schedules (and schedule changes) communicated to health workers? Supervisory methods Good supervision encompasses a variety of methods. These questions will allow you to determine which methods the supervisor is using, and will help you to assess the quality of supervision. Do supervisors use the following methods during supervision? 11. YES NO Observation of service delivery? 12. YES NO Observation of counselling or an outreach health education session? 13. YES NO Asking the service provider about what problems she/he has been having? 14. YES NO Team approach to problem identification and solution? 15. YES NO Review of records, supplies, or the conditions of the facility? 16. YES NO Community-level assessment of the outcomes or impacts of interventions? Do the following areas receive adequate attention during supervision? 17. Household visits 27. Planning 18. Antenatal care 28. Training 19. Delivery 29. Community organisation 20. Postnatal care 30. Logistics management 21. Child spacing 31. Financial management 22. Immunization 32. MIS 23. Growth monitoring 33. Personnel management 24. Oral rehydration therapy 34. Tuberculosis 25. Acute respiratory infection 35. Water and sanitation 26. Malaria 36. Curative care 37. YES NO Are health workers given adequate support through supervision? 38. YES NO Are community health workers given adequate support through supervision? 39. YES NO Is the health committee or other community organisation given adequate support through the supervisory system? 40. YES NO Is the community participating in the supervisory system to the extent appropriate? 41. YES NO Are supervisors receiving adequate support from higher-level managers? Supervision of technical service quality These questions will help you assess whether the supervisors are supervising the quality of the services being provided. 42. YES NO Do supervisors assess technical service quality through observation of service delivery or competency-based testing? 43. YES NO Do supervisors make comments aimed at improving technical service quality? 44. YES NO Do they make comments aimed at improving counselling or health education? 45. YES NO Do they demonstrate any technical skills to the supervisee? 46. YES NO Do they verify that written protocols are available to the supervisee? Supervisory style These questions will help you assess the supervisory style of the supervisor. 47. YES NO Do supervisors allow the supervisee adequate time to talk about problems he or she has experienced? 48. YES NO Do they refrain from criticising the service provider in front of patients? 49. YES NO Do they establish a good rapport with the supervisee? 50. YES NO Do they establish a good rapport with clients or community members? 51. YES NO Do they praise good performance? Work planning and problem solving One role of the supervisor is to assist the supervisee plan his/her work and solve problems. These questions will help you assess the degree to which the supervisor fulfils this role. 52. YES NO Do supervisors help their supervisees to organise and plan their work? 53. YES NO Do supervisors help service providers to identify problems? 54. YES NO When problems are identified, do supervisors make recommendations or take action? 55. YES NO Do supervisees feel free to discuss problems with the supervisor? 56. YES NO Do supervisors make recommendations, respond, or take action on the problems or issues raised by the supervisee? Supervisory records Recording the supervisory visit is essential to ensure consistency and follow-up. The following questions will help you assess whether adequate records are being kept. 57. YES NO Are supervisory records kept? 58. YES NO Are identified problems recorded? 59. YES NO Are actions taken recorded? 60. YES NO Are records used for follow-up? 61. YES NO Are records used to plan other activities, such as training? 62. YES NO Are follow-up activities recorded? Financial management 1. Health facility 2. Service provider 3. Observer/Supervisor 4. Date General accounting The fiscal health of a programme/facility relies on well kept, accurate accounting books. These questions will help you determine if a basic accounting system is in place. 5. YES NO Is a chart of accounts used? 6. YES NO Does the chart of accounts meet the current needs of the programme? 7. YES NO Are accounting records kept up to date and balanced monthly? 8. YES NO Is someone responsible for daily transactions and entries? 9. YES NO Are the accounts legible? 10. YES NO Are the calculations correct? 11. YES NO Are all transactions entered n the ledger? Cash disbursements Disbursements are monies paid out of the programme/facility account. These questions will help you assess the adequacy of the disbursement procedures. 12. YES NO Are all disbursements made by cheque? 13. YES NO Are pre-numbered checks used so that missing cheques can be accounted for? 14. YES NO Do the signatories sign cheques only after they are properly completed and justified with supporting documentation? 15. YES NO Does the administrator approve and cancel the documentation in support of all disbursements? 16. YES NO Are all voided cheques retained and accounted for ? 17. YES NO Does the administrator review the bank reconciliation? 18. YES NO Are vouchers prepared for all disbursements? Accounts payable and purchases These questions will help you assess the adequacy of purchasing procedures. 19. YES NO Does someone other than the bookkeeper always do the purchasing? 20. YES NO Are suppliers' invoices regularly compared with recorded liabilities? 21. YES NO Are vouchers always prepared and approved before payment? 22. YES NO Are purchase orders always approved before they are sent to suppliers? Payroll These questions will help you assess whether adequate payroll procedures are used. 23. YES NO Are employees required to account for the hours they worked? 24. YES NO What do you receive as supporting documentation to calculate employee salaries? 25. YES NO Do employees sign a "pay slip" to indicate receipt of payment (if payment is not a cheque)? Petty cash Petty cash can be the most difficult monies to control. These questions will help you to assess whether the established procedures are being followed in the use of petty cash. 26. YES NO Is the petty cash box locked at all times? 27. YES NO Are arrangements made for access to the cash box during the custodian's absence (for lunch etc.)? 28. YES NO Is the fund checked by surprise counts made by another employee independent of the custodian? 29. YES NO Has a maximum figure for individual payments from the fund been established? 30. YES NO Are vouchers and supporting documents checked at the time of reimbursement by a responsible employee? 31. YES NO Does the custodian verify the amount of the fund? 32. YES NO Are vouchers voided so as to preclude their reuse ? 33. YES NO Are the amounts of the vouchers spelled out in words as well as written in numerals? 34. YES NO Are Petty Cash receipts pre-numbered? 35. YES NO When replenishing petty cash, does custodian fill out a written request and attach all supporting documents to this request? 36. YES NO Does person authorising replenishment verify that all disbursements have been properly justified before granting the request? Receipts These question will help you to assess whether receipts are adequately handled and documented. 37. YES NO Are funds received deposited in a separate bank account (not co-mingled with other accounts)? 38. YES NO Recorded on a cumulative basis? Financial planning and management A budget is an essential tool for programme management. These questions will help you assess whether the budget is being used properly in programme management. 38. YES NO Is there a programme budget? 39. YES NO Did the staff participate in the development of budgets? 40. YES NO Are there established procedures for review and approval of budgets? 41. YES NO Was the budget reviewed and approved by senior management and donors? 42. YES NO Does the budget reflect the goals and objectives of the programme? 43. YES NO Are the budgets used as a tool for evaluating performance on a planned versus actual costs asis? Does the current budget include adequate funding for: 44. YES NO Staff wages and benefits? 45. YES NO Supplies? 46. YES NO Equipment? 47. YES NO Transportation? 48. YES NO Other? 49. YES NO Are monthly financial reports prepared? 50. YES NO Does the administrator review monthly financial reports? 51. YES NO Do monthly reports show budgetary expenditures, encumbrances, and availability by category of expense? 52. YES NO Does the administrator prepare budgets and standards to aid in planning? 53. YES NO Does the administrator evaluate results by making annual comparison of "budget" to "actual" financial performance? 54. YES NO Does the administrator ensure that receipt and expenditure of funds is properly identified on a cumulative basis? Sources and adequacy of funding These questions will help you assess the sources of funding, reliability, and adequacy. 55. YES NO Does the programme receive funds from the community (fees or contributions)? 56. YES NO Does the programme receive funds from the government? 57. YES NO Does the programme receive funds from international donors? 58. YES NO Does the programme receive funds from other sources? 59. YES NO Is funding adequate for the next year of operation? 60. YES NO Is funding adequate for the next three years of operation? 61. YES NO Is the funding source or mix of sources reliable? 62. YES NO Is sustainability considered in the financial planning process? 63. YES NO Is equity considered in the financial planning process? 64. YES NO Is growth considered in the financial planning process? Logistics management 1. Health facility 2. Service provider 3. Observer/Supervisor 4. Date Procurement These questions will help you to assess the adequacy of the procurement system and its operations. 5. YES NO Does the institution have its own established checklists and procedures for the procurement of goods and services? 6. YES NO Is someone responsible for initiating procurement requests within the institution? 7. YES NO Is a standardised procurement request form used? 8. YES NO Is there a clear policy about who can authorise requests? 9. YES NO When items are purchased, are multiple cost estimates sought from potential suppliers? 10. YES NO Has the person who makes the decision received training in accordance with established policies for cost analysis and selection criteria? 11. YES NO Is verification obtained from the accounting department to ensure that sufficient funds are available in the budget prior to each procurement? 12. YES NO Before placing an order for goods or services, does the administrator review the order to ensure that the item(s) are needed? 13. YES NO For large procurements, is additional support and attention given to the procurement planning process employed? 14. YES NO Are supplies regularly ordered according to the defined procedures? 15. YES NO When procedures are followed, are supplies received on time? Reception of materials These questions will help to assess the adequacy of the reception of equipment and supplies. 16. YES NO Does the facility or organisation have established reception procedures? 17. YES NO Is someone responsible for receipt of materials? 18. YES NO Is there a standardised form for reporting reception of materials? 19. YES NO Is the reception report reviewed by manager(s)? 20. YES NO Are received goods inspected for defects? 21. YES NO Are received goods compared against purchase orders to be sure that the order has been filled correctly (quantity and items requested)? 22. YES NO Are equipment and supplies recorded in stock records immediately after completion of reception procedures? 23. YES NO Are reception procedures carried out? Inventory control These questions will help you to assess the adequacy of the inventory control system for equipment and supplies. 24. YES NO Does the institution have established procedures for inventory control? 25. YES NO Is the inventory maintained according to the defined procedures? 26. YES NO Is the size of the storage area adequate? 27. YES NO Is the storage area protected (from- water, dust, etc.)? 28. YES NO Are equipment and supplies labelled clearly and organised in an accessible way? 29. YES NO Is it possible to lock the storage area? Supplies 30. YES NO Is there a clerk responsible for tracking supplies? 31. YES NO Are stock records (cards or register) maintained for all items in stock? 32. YES NO Are records updated after each transaction to reflect accurately stock levels for a particular item at any given point in time? 33. YES NO Are there established procedures for reordering stock items? 34. YES NO Is a physical inventory taken to verify theoretical stock levels with actual physical counts? 35. YES NO Is such an inventory carried out at least every (weeks or months)? 36. YES NO Is there a supervisor on hand to observe the counting and ensure that the inventory is carried out properly? 37. YES NO Does a second person count supplies to ensure that the count was accurate? 38. YES NO Are expiration dates of medicines checked during the inventory? Equipment 39. YES NO Is there a permanent property record for all equipment? If yes, does the record include: 40. YES NO Date of acquisition? 41. YES NO Serial number (if any)? 42. YES NO Name and address of the vendor? 43. YES NO Amount paid for the property? 44. YES NO Location of the property? 45. YES NO Actual condition? 46. YES NO An inventory control number? 47. YES NO Are there established procedures for the assignment of equipment? 48. YES NO Does the organisation carry out an annual physical inventory of equipment? 49. YES NO If yes, was an inventory completed during the past year? 50. YES NO Are results of the inventory compared with the property records for verification? 51. YES NO At the time of the inventory is equipment reviewed to assure that it is in good condition and that the technology is up-to-date (where appropriate)? 52. YES NO Are there established procedures for lost, damaged, or stolen property? 53. YES NO Are there established procedures for the disposal of property? 54. YES NO Are there established procedures for the transfer of property within the organisation? Stock issue These questions will help you to assess the adequacy of the system for issuing supplies. 55. YES NO Is there a standard request form to be filled out before items can be released from inventory? 56. YES NO Is there an authorisation procedure in place? 57. YES NO Are disbursements of supplies recorded in inventory records at the time of transaction? 58. YES NO Is a receipt prepared at the time the material leaves the storeroom? 59. YES NO Does the recipient of the materials sign for them? 60. YES NO Is health centre staff satisfied with the performance of suppliers? 61. YES NO Is health centre staff satisfied with the performance of the personnel responsible for procurement, inventory management, and disbursement? Transportation If the programme includes community outreach or community based service delivery, or if satellite health posts are part of the programme, an appropriate transportation system should be in place. These questions will help you to assess the management of transportation resources. 62. YES NO Does the staff have access to transportation? 63. YES NO Is the fuel available adequate for field visits and other off-site activities? 64. YES NO Is there a schedule or appointment book kept for health centre vehicles? 65. YES NO Is there a maintenance schedule for vehicles? 66. YES NO Is there a mechanism for acquiring spare parts? Information management 1. Health facility 2. Service provider 3. Observer/Supervisor 4. Date Planning the information system Health service indicators monitor the performance of a system or programme. An information system is a group of these indicators that reveal the status of the programme. These questions will help you determine if a basic information system is in place. 5. YES NO Is there a list of indicators to be monitored at district-level? 6. YES NO Is there a list of indicators to be monitored at health centre-level? Do these lists include indicators of: 7. YES NO Resource availability? 8. YES NO Access? 9. YES NO Utilisation? 10. YES NO Coverage? 11. YES NO Service quality? 12. YES NO Outcome? 13. YES NO Have information sources been identified for eachindicator? 14. YES NO Has the frequency of collection/compilation been established for each indicator? 15. YES NO Has an analysis procedure for each indicator been established (including thresholds or standards)? 16. YES NO Have mechanisms for interpreting and discussing results been established? Record keeping - first level In order for the information system to work, information collected at the most peripheral level must be complete and correct. These questions will help you assess the completeness and precision of the information being collected at the first level. Direct service delivery 17. YES NO Is data on the size of the target population available? <$FThis could be collected from census data or from a population-based information system.> 18. YES NO Do health workers maintain records on services delivered to specific individuals (with information on age, weight, diagnosis (classification), treatment, referral)? 19. YES NO Do health workers maintain records on the number of health education sessions, outreach clinics, vaccination sessions, etc.? 20. YES NO Do health workers maintain records on high-risk cases for follow-up? 21. YES NO Do health workers maintain records on the number of deaths, the number of cases of immunizable diseases, the number of malnourished children? Support services 22. YES NO Do health -workers maintain records on stock inventories? 23. YES NO Do health workers maintain records on the integrity of the cold chain? 24. YES NO Do supervisors maintain supervision records with information on performance of specific health workers? Compilation, tabulation and analysis of information - first level <$FThe first level refers to health centres/health posts or other level of primary data collection and compilation.>. While information is generally collected with the express purpose of being sent to the central level, it can also be very useful to the health worker. These questions will help you identify whether the health worker is using the information at the first level. 25. YES NO Do health workers routinely compile/tabulate information from service records for monitoring? 26. YES NO Do health workers calculate the value of indicators on resource availability for specific interventions? 27. YES NO Do health workers calculate the value of indicators on utilisation of specific interventions? 28. YES NO Do health workers calculate the value of indicators on high-risk follow-up for specific interventions? 29. YES NO Do health workers calculate the value of indicators on coverage with specific interventions? 30. YES NO Do health workers calculate the value of indicators on quality of worker performance for specific interventions? Use of information for management decision-making These questions will help you identify how the health worker is using the information collected/tabulated/analysed. 31. YES NO Do local-level health managers utilise information to identify programme problems