June 29, 2018
Direct Observation – An Underutilized Formative Research Technique in Global Health Program Design and Implementation
Observation, unlike other formative techniques, can uncover issues critical to making a health intervention succeed or, sometimes, reveal reasons why it is likely to fail
Formative research is often considered the domain of behavior change interventions. It’s commonly equated with focus groups and interviews. In a recent article published in Global Health: Science and Practice, Steven Harvey, PhD, an assistant professor of International Health at the Johns Hopkins Bloomberg School of Public Health, argues that direct observation uncovers insights that other techniques are unlikely to find and that it should be employed more often in global public health program design and implementation that depend on
- habitual tasks that people find difficult to articulate
- effects of the built environment
- mechanical or clinical skills
- service delivery processes
In four case studies, Harvey shows how direct observation found information no other data collection technique would have likely found. This information, in turn, proved essential in either correcting course or explaining why an intervention ultimately failed.
Habitual tasks that people find difficult to articulate
Researchers tested the feasibility of corralling free-range chickens raised near homes to minimize contact between children and birds. Contact with poultry feces, present where children routinely played, was thought to be a major contributor to the high prevalence of Campylobacter-associated diarrheal disease in the study site. Campylobacter jejuni is found almost universally in the intestinal tracts of poultry.
In initial interviews, participants stated that they kept their birds corralled “all the time.” However, direct observation found birds loose between 13% and 33% of visits. Through further investigation, researchers found that “all the time” really meant “all the time except at certain (presumably obvious) specific times.” Participants presumed that researchers knew that the birds needed time to exercise outside the corral. Had they relied solely on interviews (reported behavior), they might never have known that birds were sometimes loose or might never have thought to ask why.
Several other direct observations, including children playing with birds in the corrals, contributed to a conclusion that the intervention was unlikely to succeed, a conclusion confirmed by subsequent research demonstrating that corralling, instead of decreasing risk of Campylobacter-associated diarrhea in children, actually doubled it.
Effects of the Built Environment
In an insecticide-treated bed net study in the Peruvian Amazon, observers took detailed notes on sleeping spaces in participating households. These notes revealed that many people slept directly on cane flooring rather than on a bed. The flooring had gaps between the cane staves. Since many houses were built on stilts, this meant mosquitoes could enter the sleeping space from below. A net alone could not provide adequate protection in this setting. The structure of a typical bed in the study setting—no mattress and gaps between the wooden or bamboo slats that allowed mosquitoes to bite from underneath—might never have occurred to public health practitioners, most of whom presumably sleep in beds with mattresses. Even had it occurred to them, they would not have been able to collect systematic data on bed configurations without observation.
To evaluate the ability of skilled birth attendants (SBAs) on 4 crucial delivery skills, researchers developed tests on anatomical models. Assessing skills on actual patients is difficult because an observer qualified to evaluate clinical competency would need to stop observing and intervene before allowing an insufficiently skilled provider to endanger a patient's life or well-being. Moreover, even common obstetric complications are relatively rare. This makes it impossible to assess the skill of more than a handful of providers using actual patients.
Across the seven countries the tests were administered, most SBAs could not demonstrate the correct hand positions for carrying out the manual removal of a retained placenta. Although bimanual uterine compression is a relatively simple procedure requiring no instruments or equipment, virtually no SBA was familiar with it. This and other observations led to additional practical training for SBAs and to development of specific training modules.
Service delivery processes
To test the feasibility of community health workers (CHWs) administering rapid diagnostic tests (RDTs) for malaria, a study team identified 16 discrete steps necessary to correctly prepare and interpret the test; performing them in the wrong order (e.g., opening the sterile lancet before cleaning the finger with an alcohol swab) or the wrong way (depositing the blood drop where the buffer solution is supposed to go) could compromise test accuracy or patient or health worker safety. The observation checklist enabled the team to determine the proportion of health workers who completed all steps correctly, identify specific steps where health workers had problems, and modify training to address the problems observed.
As a stand-alone method, observation can measure phenomena not measurable by any other method. In combination with interviews or focus groups, it can suggest questions to be posed through these other methods. It can also triangulate findings from other methods, reveal potential differences between reported and observed behavior, and thus help assess social desirability bias. Given these benefits, observation—either alone or in combination with other methods—is something both investigators and program managers should consider when undertaking formative research.
Observe Before You Leap: Why Observation Provides Critical Insights for Formative Research and Intervention Design That You’ll Never Get From Focus Groups, Interviews, or KAP Surveys was written by Steven Harvey. Global Health: Science and Practice May 2018, GHSP-D-17-00328; https://doi.org/10.9745/GHSP-D-17-00328