Gavi, long above the fray and untouched by vituperations over aid effectiveness, has come under scrutiny in a paper from the Center for Global Development (CGD). The paper found Gavi “failed to increase vaccination rates for diseases covered by cheap, existing vaccines” and did little better with new, more expensive vaccines. The study came with important caveats but, together with findings of other researchers, jeopardizes not just Gavi’s gold standard status but challenges its—indirect—claim to have saved millions of lives.
Gavi is the poster child of global health. As Bill Gates said recently: “Every time people look at aid money and say what’s the most impactful thing they can find in that whole space, Gavi gets ranked at the very top.” Gavi began as the Bill & Melinda Gates Childhood Immunization Program, and the “work that the foundation has done in vaccines through Gavi,” continued Gates, “has been the most important thing we’ve done.” As a result, “we’ve saved millions of lives,” seven million, according to Gavi, since it began in 2000.
The CGD’s Justin Sandefur agrees that “vaccines are saving millions of lives—I have no doubt about that.” But Sandefur and colleagues examined, “the causal effect of giving Gavi funding in terms of increasing number of lives saved…” He notes that even Gavi does not claim causality but to have helped developing countries save millions of lives. “If you read very carefully, it doesn’t make a clear, causal attribution to Gavi funding doing that,” said Sandefur. But, “I don’t think that would be a lay person’s interpretation of what’s on the website necessarily.” Readers—and perhaps funders—infer causality. Bill Gates does.
But it is actually a hypothesis that, over the last 15 years and with a budget of $7 billion, Gavi increased the number of children vaccinated, thereby saving millions of lives. “Certainly we can’t corroborate the full headline numbers,” said Sandefur. For the classical vaccines, DPT (Diphtheria, Pertussis, Tetanus) and Hepatitis B vaccines, “we’re not finding any impact,” said Sandefur. CGD found immunization rates were a function of national income, not Gavi support. The CGD study compared countries near the eligibility threshold for Gavi, national income of $1,000 per person. If Gavi raised vaccination rates, there should be a jump right at the eligibility cut-off, because of the sudden jump in available funding.
But there is no jump for the older, inexpensive vaccines, DPT and Hepatitis B. Coverage for those Gavi-supported vaccines looked the same as for measles, which Gavi did not fund.
Gavi spokesperson, Rob Kelly, countered that “The paper misinterprets the statement that Gavi has immunized an additional 440 million children to mean that those children would not have been immunized at all in the absence of Gavi support.” Kelly acknowledged, that "many of these children would have been reached in the absence of Gavi support, but with fewer vaccines.”
Rolling out newer, more expensive vaccines has been a major emphasis for Gavi, both to reduce inequities (it used to take decades for new vaccines to reach the developing world) and to further reduce vaccine-preventable deaths from diarrhea, for example. According to Kelly, “The data makes it clear that the very large majority of Gavi supported vaccines were not in country programs prior to Gavi support.”
However, CGD found only “small and statistically insignificant effects for the three high-priced vaccines promoted by Gavi,” Haemophilus influenza type B, pneumococcal disease, and rotavirus. According to Sandefur, this is because “…the rotavirus and pneumo vaccines simply haven’t had that wide an adoption in the lowest income countries—yet.” Sandefur said the absence of Gavi’s impact on vaccination for the older, cheaper vaccines is “probably the main thing people are going to take away from the paper,” rather than the “non-robust, positive results on the newer vaccines.”
Fungibility: how even free didn't work
The countries receiving Gavi support seem to spend the money elsewhere: “subsidized vaccines appear to have displaced domestic vaccination spending in countries near the threshold,” concluded the CGD study. The policy implications, said Sandefur, are that “delivering cheap, already existing vaccines to these middle-income countries with millions and millions of poor people is probably not going to accomplish much.” Such countries include India, China and Ghana, for example and “account for a huge share of the global poor.” They also account for a huge share of the lives Gavi claims to save.
The fungibility of aid and vaccine support in particular are not new issues. A Gavi-funded study raised the concern of whether Gavi’s Immunization Support Services (ISS) program displaced national government vaccine spending. Results of the 2007 final report were “inconclusive.” But it said ISS was “the easiest source of funding to meet [national immunization program] priorities…” Consequently, the report recommended more effort to ensure that ISS “does not become the funding source of first resort.” The analysis found worrying signs. Without ISS, spending on activities most likely to raise vaccination rates fell in 20 of 27 countries examined, by a median 4%. Although the changes were not statistically significant, it was “possible that ISS funding has displaced other sources of funding,” a result confirmed by the more recent CGD study.
"Near" the cutoff: The slippery slope to no impact
What about much poorer countries not near the threshold? Sandefur said he is “agnostic” about the impact of Gavi on much lower income states like Niger and Burkina Faso. “Our results are focused on countries near the threshold.” However, for the CGD analysis, “near the threshold” actually means income levels down to $500 a year per person. Thus the CGD findings of no impact technically extend to Gavi-supported, low-income countries like Chad, Somalia and North Korea.
What about still poorer countries? Sandefur cautioned that, “Philosophically, there's also an argument to be made that the causal effect we estimate here applies strictly to a hypothetical country right on the threshold, just teetering on the edge.” This would limit the comparison to Gavi-supported Ukraine, on one side of the cut-off, and the Philippines on the other. However, there's no strict rule about how far to extrapolate these causal effects. “The more you extrapolate, the more damning our conclusions,” said Sandefur. “I would go beyond Ukraine, but stop short of the whole world.” Sandefur noted, however, that methodological objections are not raised to desired results, only negative ones: “if we had found great, positive effects, everybody would be eager to extrapolate to all countries.”
Sandefur acknowledged there is a basis for extrapolating the findings of no impact from Gavi even down to very poor countries. “If there was a big switch in the impact, you would expect to see some big jump in the vaccination rates between lower income and middle income and what I’m saying is you don’t really see that. There’s not some sudden regime shift when you move from low income to middle income countries.” In addition, vaccination rates before and after Gavi started in 2000 can be compared to “see if you see any jumps at that point,” after Gavi kicks in. “And you don’t,” said Sandefur.
A major methodological problem for measuring Gavi’s effect in poor countries is that there is no control group. According to Sandefur: “All very poor countries either got Gavi aid, or were eligible to get Gavi aid and didn’t for some reason that probably implies they're a bad control group…” Absence of a control group has led to “fairly wild extrapolations from what's happening in non-poor, non-Gavi countries,” explained Sandefur.
A 2008 paper from the Institute for Health Metrics and Evaluation (IHME) estimated 7.4 million additional children were vaccinated since Gavi came into being. But Sandefur questioned “whether there are really strong grounds for causal inference or attribution to Gavi on the basis of those regressions, to be blunt.” And indeed, the IHME paper does not assert causality, instead reporting that “we have not addressed whether the increases in DTP3 coverage that have occurred in GAVI ISS recipient countries would have occurred in the absence of GAVI’s support.”
Gavi support did lead to an increase in reported vaccination coverage. Countries received rewards for increasing vaccination rates and reported rates duly rose, as high as 80–100% in countries like Niger and Mali where surveys put coverage much lower, closer to 40–60%. Whether a positive impact of Gavi would remain absent over-reporting was “an unanswered question,” according to IHME.
CGD confirmed IHME’s 2008 results. “There is clearly evidence that there was manipulation of the DPT data by Gavi-eligible, ISS-eligible countries,” said Sandefur: “So progress there seemed to have been exaggerated.” Yet, even with the benefit of immunization over-reporting, CGD still found no impact from Gavi.
An undesirable conclusion
Gavi was started because it seemed to many that the urgency had gone out of vaccination efforts led by the World Health Organization and UNICEF. But IHME’s 2008 study found that perception was mistaken: “Globally, survey-based coverage shows that the yearly increase is fairly constant before and after the establishment of Gavi…” Few would question whether Gavi was a good idea. But the intrepid Gates-funded researchers at IHME wondered: “Would Gavi have been implemented if there was less of a perception that improvements in DTP3 immunization coverage during the 1990s were stagnating?” Melinda Gates continues to believe in the now-disproven stagnation hypothesis. Speaking last month of the Gates Foundation’s entry into vaccination, she said: “We started to study it. We started to learn how we had this incredible vaccine system that had kind of crumbled over time.”
Melinda explained that “we” meant more than just the Gates Foundation. “When I say ‘we,’ I don’t want people to think we’re taking credit for all this,” pointing to the importance of partners. However, the Gates Foundation annual letter laid indirect claim to reducing child mortality: “The last time we cut the child death rate in half, it took 25 years. We will do it again in 15 years.” It’s a story everyone wants to believe. Gavi is a “game-changer,” gushed Melinda Gates’ interviewer. Yet quite possibly, even the very poorest countries needed no help and “we,” however defined, contributed nothing to saving the lives of children, only the perception of saving millions of lives.
Gavi subsumed the Expanded Program on Immunization (EPI), officially begun in 1974 and run by UNICEF and WHO. EPI hoisted vaccination rates from as low as 5% all the way to a reported 80%, a goal declared as achieved in 1990. Today, with Gavi in charge, WHO recently said immunization “stalled in recent years”—the needle stuck at the familiar 80% level.
Gavi recently completed a new $7.5 billion round of financing and promises to save six million more lives.