To stem a growing polio crisis, health officials are accelerating the development of a new oral vaccine with plans for emergency approval and deployment in regions with active polio transmission as early as June 2020. The new vaccine, called nOPV2, might conclusively end the outbreaks, caused by the live virus in the vaccine reverting to a virulent form. But expedited approval means skipping the real-world testing of large clinical trials...

Read the rest at The Scientist

{Polio Eradication, Part 2}

By Robert Fortner & Alex Park

Authors’ note: 

In late 2017/early 2018, Alex Park and I co-authored a three-part series on polio for De Correspondent, originally appearing in Dutch:

Na 30 jaar en 15 miljard dollar is polio bijna helemaal uitgeroeid. Hoe?

Als je een ziekte bijna uitroeit. En daarmee een nieuwe variant de wereld in helpt

Waarom de miljarden van Bill en Melinda Gates malaria niet uit zullen roeien

De Correspondent initially planned to publish English versions of the articles but, during the course of our series, announced plans for an English-only web publication, The Correspondent. That publication has been struggling to get off the ground ever since. 

Fortunately, we were able to find a home for the third piece in the series, at Undark under the title “The Enduring Appeal (and Folly) of Disease Eradication”. But parts one and two never found an English-language publisher.

Back in 2017 when we were writing, eradication of wild poliovirus was quite close but, to our astonishment, modeling showed a disconcerting 1 in 20 chance of huge outbreaks of 50,000 cases or more in the decades after eradication. These theoretical epidemics didn’t spell the return of wild polio-- the dreaded, crippling virus which has been the constant target of the largest medical campaign in history, but of vaccine-derived poliovirus (VDPV)-- another disease with virtually the same symptoms, and one whose sole cause is the oral vaccine meant to end polio in the first place. 

______

As a global alliance built to eradicate wild polio approaches the finish line, one of its biggest legacies may also be the greatest threat to its progress: vaccine-derived poliovirus, or VDPV. By some estimates, post-eradication VDPV outbreaks could be larger than any polio outbreaks to date. But how will the world celebrate the achievement when VDPV outbreaks are almost certain to occur into the future? 

At the Bill & Melinda Gates foundation in Seattle, this year’s World Polio Day event followed what has become a standardized ritual. Before an audience of hundreds, a series of speakers praised the work of volunteers and donors, reiterating the point that the end of polio was near. “We are this close to ending polio,” Dean Rohrs, Vice President of Rotary International, told the crowd of several hundred people, holding up a thumb and finger mere centimeters apart. As several speakers emphasized, only 12 cases of wild polio had been reported so far that year, anywhere in the world. 

But wild polio is not the only polio left.

When the World Health Assembly adopted global polio eradication in 1988, the disease was limited to three naturally occurring virus types. That changed in 2000, when scientists realized the weakened virus in the Sabin oral polio vaccine (OPV) used in most developing countries could revert to a pathological state, creating a new virus which can spread and paralyze just like polio: vaccine-derived poliovirus, or VDPV. 

Reversion in the body of a vaccine recipient is surprisingly common, but transmission to another person is extraordinarily rare: There have been relatively few outbreaks of VDPV this century, although billions of doses of oral vaccine have been delivered over the course of the campaign. When VDPV does manage to catch hold, however, the “attack rate” is about the same as it is for wild polio: one case of paralysis for roughly every 200 people infected. 

VDPV only spreads in areas where immunity rates are low, often due to war, such as eastern Syria and the Democratic Republic of the Congo. But as the global polio eradication campaign winds down, susceptibility to VDPV is rising, leaving literally millions of children unprotected against paralysis from VDPV.

At the Seattle event, none of the speakers even mentioned VDPV. However, global health experts generally believe that VDPV will outlast wild polio, meaning the anticipated “end of polio” will be nothing of the sort. At the moment the world celebrates the end, the talk around VDPV “will be a complicated communication exercise,” according to Michel Zaffran, director of the polio program at the World Health Organization. “We know that we actually might continue to get outbreaks after the certification of the wild poliovirus eradication.” And those outbreaks could be massive, perhaps larger than any polio outbreak in history.

{Getting ready for a party}

With the end of wild polio in sight, some of the eradication campaign’s key stakeholders are eager to declare victory and withdraw. 

By some projections, the last case could be detected as early as next year. If no cases are detected for another three years, WHO will officially certify the end of wild polio. Although the Global Polio Eradication Initiative (GPEI) targets both wild polio and VDPV, after certification, much of the apparatus it commands to vaccinate the world’s people against any form of polio will shut down practically the day after wild polio is declared vanquished. In April 2017, the Polio Oversight Board formally decided to “sunset” GPEI at certification instead of waiting some period of time after certification.

What would an outbreak of VDPV a few years from now look like? Let’s assume for a moment that the last case of wild polio is detected in 2019, and that VDPV lingers on in a few corners of the world. Based on current risk assessments and mathematical models, it is possible to imagine a future not long after when a VDPV outbreak catches the world off guard. 

{The outbreak of the future}

The following scenario begins in January 2018, in Syria.

As the Syrian war civil war draws to a close healthcare workers are continuing to fight an outbreak of VDPV which began almost a year ago. In Aleppo, a fourteen-year-old named Mohammed is one of countless people trying to put his life back together. 

Mohammed comes from a large family which was divided during the war. The most recent split occurred last May, shortly after his sister gave birth to a child, Asif. From the beginning, Asif was a sickly child, and with healthcare broken in Aleppo, the family chose to send him to live with grandparents in Damascus. Now that his sister and nephew are settled in the capital, Mohammed decides to visit them.

Before leaving, Mohammed receives a dose of oral polio vaccine, one of hundreds of thousands healthcare workers distribute around Syria in response to the VDPV outbreak. Mohammed thinks nothing of it. But the vaccine’s virus mutates and turns to VDPV. Showing no symptoms of the disease, and completely unaware that he has it, Mohammed heads west.

In Damascus, Mohammed holds Asif nephew, and accidentally coughs on him, infecting him with VDPV. But Asif’s constant sickness is due to Primary Immunodeficiency, a rare condition, although one more common in the Middle East and North Africa. Asif’s immune system never clears the infection. 

Five years later, Syria is a changed country. With the war now ended, healthcare is slowly rebuilding in Aleppo. In June 2023, Asif and his mother travel back to the city to live in the family home. There he distributes VDPV into the open sewers around his family’s house every time he defecates. 

Among Syrians, VDPV is low on a long list of issues facing the country. But the mood is optimistic as the flow of refugees has reversed, with more Syrians now entering the country than leaving it for the first time in years. The work of rebuilding is also attracting an international group of contractors. 

One of those people is a 26-year-old electrician from Romania named Vasile. Vasile grew up in a small village in northern Romania, where it was common for children not to receive all WHO’s recommended vaccinations. (In 2016, Romanian authorities estimated 89 percent of the population had received the recommended third dose of polio vaccine, though some areas had a much lower vaccine rate.) When he was a child, Vasile received one dose of polio vaccine, but not the recommended second or third, and he never developed immunity.

Last year, Vasile’s company was hired to rebuild a municipal building in Aleppo, Syria’s most devastated city and now the site of an international reconstruction effort. In January, he flies from Bucharest to Aleppo to observe some work on the site. Since the war destroyed the city’s water pumps, drinking water is scarce. Six kilometers away, a group of returnees have launched a makeshift bottling plant in a bombed out hotel. Unfortunately, unbeknownst to them, sewage, some of which was laced with VDPV, has contaminated the water supply.

In late February, Vasile buys water from the vendors and drinks it. He’s quickly infected with VDPV, but, like most people who catch the disease, he doesn’t show any symptoms. 

Weeks later, Vasile returns to his hometown in northern Romania. Vasile’s arrival after months abroad is an occasion, and family members from other parts of the country come to visit. Vasile kisses his wife, and coughs on his neighbor’s children, spreading the disease to each of them. In a matter of weeks, the VDPV Vasile brought into Romania infects more than twenty other people, but the outbreak remains undetected. (According to one analysis, VDPV could spread undetected in Europe for up to a year.)

In May, a person infected with VDPV in Bucharest travels to Italy. Another, infected in Cluj-Napoca, travels to Ukraine, where polio vaccination rates have been inconsistent but generally low for more than a decade. By August, Italian authorities announce a handful of paralysis cases, the first sign that the disease has entered Europe. In Ukraine, VDPV is spreading much faster, undetected. From this huge pool of infected people, small clusters of paralysis begin popping up, eventually turning into thousands of cases across Europe while international travelers have spread VDPV across the world. WHO declares an emergency and announces plans to re-introduce the oral polio vaccine--the only way to halt the outbreak. What had been a thirty-year eradication campaign starts over again.

{How great is the risk?}

This scenario is not far-fetched: In recent years, scientists have attempted to quantify the risk of a VDPV outbreak after wild polio is declared eradicated. One such effort, by the Orlando, Florida research group Kid Risk, put the risk of an outbreak of 50,000 or more cases of paralysis at around 5.7 percent. The same paper said there could be as many as 800,000 cases of paralysis. Kid Risk’s research on polio has been funded by both the US Centers for Disease Control and Prevention (CDC) and the Bill & Melinda Gates Foundation. 

Experts disagree on what factors could contribute to an outbreak and their likelihood. The most likely trigger, according to Kid Risk, comes from individuals with immune deficiencies, like Asif in our story. One patient in the UK has been excreting VDPV for 28 years, although in countries with weak health systems, immune deficiency usually leads to death at a young age. 

After eradication of wild poliovirus, use of the oral vaccine will stop. Many newborn children will no longer be protected from polio. Years after the global celebration of polio eradication, an immunodeficient carrier of VDPV could infect one of the many millions of non-immune people, starting an outbreak.

The Gates Foundation said that predicted outbreaks of over 50,000 cases “were based on a few assumptions in the study that illustrate potential risks, but do not represent the current plans for polio eradication and global immunization.”

However, the 5.7 percent chance of a massive outbreak might be a low estimate. According to Tapani Hovi, “there are other excreters,” who are relatively healthy. Hovi, an infectious disease expert at Finland’s National Institute for Health, and other researchers have found VDPV-positive sewage samples in Finland and Slovakia over extended periods. But records of nearby hospitals found no patients with an immune deficiency. Genetic sequencing of the Finnish samples suggested one person might have been excreting VDPV for ten years. Said Hovi: “How healthy they are and how frequent they are, I have no idea. But they exist and should be taken in account in risk assessment.”

Research continues on whether immunodeficiency caused by HIV/AIDS could result in a chronic VDPV infection. A recent study found that "mildly symptomatic” HIV-infected children are able to clear the oral polio vaccine virus. Scientists cautioned that the group studied was small (just 57 children) and received life-saving antiretroviral therapy. But there are some two million children in the world living with HIV and, in some regions, as few as 20 percent receive treatment that keeps the immune system functioning. The possibility that untreated children suffering from AIDS could develop a chronic VDPV infection cannot be ruled out. As one group of researchers put it: “At present, no evidence exists whether secondary immunodeficient groups, such as HIV infected patients, could act as a long-term reservoir of poliovirus, but it is possible."

A smaller threat than chronic VDPV infection, according to Kid Risk, is polio escaping a laboratory. But lab breaches are not unheard of. In an incident this year in the Netherlands, two lab workers were exposed. One developed an infection and, confined at home, excreted virus-contaminated faeces into the local sewer system. WHO later complained that the environment around the breach was not monitored long enough.

Risk modeling for disease outbreaks is an imperfect science, and not everyone agrees on the exact range of probabilities. Stephen Cochi, a senior polio researcher at the CDC, said “We cannot take 5.7 percent as any kind of a precise number,” because it’s based on a model and numerous assumptions. Some model inputs, like the number of immunodeficient people infected with VDPV in the world, are uncertain estimates but are assumed to be 100 percent accurate. 

But with the health of millions of people at stake, even highly improbable events have to be taken seriously. Researchers believed there was only a 9 percent chance that wild polio had not been eliminated from Nigeria in 2016. At the time, it hadn’t been seen for three years. Yet as the paper with that estimate went to press, new cases were found from transmission that had gone undetected for five years. “[T]his may call into question the presented results,” the scientists wrote. They stood by their model and pointed to “a probable surveillance failure.” 

{The end of polio eradication}

Even with the current polio eradication alliance intact, fighting a large-scale outbreak would be difficult. But as the campaign closes in on wild polio, it’s also preparing to close up shop. 

Three years after the last case of wild polio is detected, the Global Certification Commission (GCC) tasked with certifying its eradication will cease to exist. GPEI will also dissolve upon certification of wild polio eradication.

“I worry about the ongoing commitment that will be necessary to surveillance after the interruption of wild poliovirus,” said David Salisbury, who heads the GCC. “We still will have to have surveillance to ensure any vaccine-derived viruses are being detected.”

Whether there will be any certification process for VDPV eradication has not yet been decided. There might be an informal certification process, or none at all, according to CDC’s Cochi. “It’s too early to tell exactly what form the monitoring of vaccine-derived polioviruses… will take and how formalized any certification process will be,” he said.

The campaign’s rush to close down comes in part from the donors’ fatigue, says Jon Abramson, chair of WHO’s Strategic Advisory Group of Experts (SAGE). Now that the campaign has missed three deadlines, he says, and donors are “already losing interest in polio eradication.”

“There is exhaustion in a lot of the donor community,” he says, including at Rotary, the donor that started it all.

“Our commitment is the certification of the eradication of the wild poliovirus,” Rotary’s Pandak said. Though she recognizes that VDPV spreads and causes paralysis just like wild polio, rather than committing to its eradication, she only said Rotary is “open to conversations and discussions” about the campaign’s future.

{“Every last child”?}

Compounding the risk of a major epidemic is the fact that so many people are vulnerable to the most common form of VDPV. 

Last year, the type 2 component was dropped from the oral vaccine. Type 2 wild polio had not been reported since 2015, and the vaccine component was also the most common source of VDPV. 

But, type 2 VDPV still exists in the world. And without type 2 in the vaccine, millions of children born since the change aren’t protected. Exactly how many, WHO doesn’t know for sure. Roland Sutter, Coordinator of WHO’s Polio Research, Policy and Containment team, guesses that “probably less than 10 percent of the global birth cohort,” or “less than 13 million” children are vulnerable. 

For decades, the polio eradication campaign slogan has been to immunize and protect “every last child.” But the millions of these type 2-vulnerable children are a stunning betrayal of the program’s own values. 

Asked about the eradication program allowing such a large population of kids to be susceptible to type 2 paralysis, Rotary’s Carol Pandak replied: “That is a tough question. I don’t really have an answer for that, to be honest.”

Asked by email if citizens in donor nations would accept such a risk to their children if so many were left unvaccinated, the Gates Foundation sent us a lengthy comment on why the campaign used OPV in developing countries. A favorite slogan of the foundation is: “All lives have equal value.” 

But parents in wealthy countries would never tolerate gambling with the health of their children. The parents of unprotected children in poor countries likely don’t even know their children have been hazarded in the quest to achieve eradication.

When type 2 VDPV outbreaks do occur, absent a new vaccine, healthcare workers will have to rely on the existing type 2 oral vaccine. But using large amounts of type 2 vaccine, as health care workers are currently doing to fight the real VDPV outbreak in Syria in our (otherwise) fictional scenario, may have as much as a 50 percent risk of seeding future outbreaks, according to researchers at the Institute for Disease Modeling. 

{A new vaccine? Now?}

One way to ameliorate the VDPV risk would be to use a vaccine that could immunize people without the risk of it becoming pathological. 

After the first known VDPV outbreak, on the Caribbean island of Hispaniola in 2000, a few scientists proposed doing exactly this, but it was ten years and several outbreaks before development began in earnest, with support from the Gates Foundation. Asked why it waited so long, foundation spokesperson, Rachel Lonsdale said “the field’s understanding of end-game realities evolved.”

If it works, the vaccine would function like the current oral vaccine, but the virus inside would be unable to mutate. Trials are taking place now in Belgium, but researchers are still waiting to see whether taking away the virus’ ability to mutate will also reduce the strength of the immune response it elicits. 

{The noble lie}

Through its investments in new vaccine development and support for researchers like Kid Risk attempting to model the risk of future outbreaks, the Gates Foundation appears to be preparing for a future with VDPV. But the assumptions they convey suggest an expectation that outbreaks will be modest in size and easy to control. “We would treat vaccine-derived polio in small, localised campaigns,” the head of the Gates Foundation’s polio program, Jay Wenger told The Independent earlier this year. “It’s obviously a complication, but we see it as a doable." 

It’s commendable that the Gates Foundation intends to sustain its efforts against VDPV after wild polio is gone. If VDPV outbreaks are limited to small outbreaks, the world may be able to fight them without the billion dollar per-year campaign. But based on current models, there’s no reason to assume small epidemics are the only scenario the world should plan for. Much of the planning is being undertaken by the Transition Independent Monitoring Board headed by Liam Donaldson. A spokesperson for Donaldson said he was too busy for an interview. According to his office, he was too busy to answer questions we emailed over a month ago.

There appear to be too many constraints for a clean solution, either to stop VDPV now or to deal with it in the future. And it all comes back to the question: How do you celebrate polio eradication if polio isn’t gone? If VDPV is swept under the carpet, why would donors pay for a quiet campaign to clean up and truly eradicate polio when the eradication moment has already passed and the world’s attention has moved on? 

If the campaign acknowledges VDPV-- a disease whose eradication is far from certain-- then the eradication moment the world has been waiting thirty years for will lose its special quality, and the idea of eradication itself as a tool of public health will lose its lustre. Instead of a huge achievement for humanity, it will be yet another missed deadline as the finish line recedes beyond the horizon. 

Consequently, what is likely to happen is that the campaign will have its eradication moment. Although the public danger of massive polio outbreaks will be higher after the celebration, VDPV will be relegated to the fine print, a critical difference, missed by the public until, perhaps, an outbreak of the disease makes international news. 

The risk may be dumped on the very countries polio eradication was supposed to help, where already millions of kids could be paralyzed by polio.

{Polio Eradication, Part 1}

By Alex Park & Robert Fortner

Authors’ note: 

In late 2017/early 2018, Alex Park and I co-authored a three-part series on polio for De Correspondent, originally appearing in Dutch:

Na 30 jaar en 15 miljard dollar is polio bijna helemaal uitgeroeid. Hoe?

Als je een ziekte bijna uitroeit. En daarmee een nieuwe variant de wereld in helpt

Waarom de miljarden van Bill en Melinda Gates malaria niet uit zullen roeien

De Correspondent initially planned to publish English versions of the articles but, during the course of our series, announced plans for an English-only web publication, The Correspondent. That publication has been struggling to get off the ground ever since. 

Fortunately, we were able to find a home for the third piece in the series, at Undark under the title “The Enduring Appeal (and Folly) of Disease Eradication”. But parts one and two never found an English-language publisher.

Back in 2017 when we were writing, eradication of wild poliovirus was quite close but, to our astonishment, modeling showed a disconcerting 1 in 20 chance of huge outbreaks of 50,000 cases or more in the decades after eradication. These theoretical epidemics didn’t spell the return of wild polio-- the dreaded, crippling virus which has been the constant target of the largest medical campaign in history, but of vaccine-derived poliovirus (VDPV)-- another disease with virtually the same symptoms, and one whose sole cause is the oral vaccine meant to end polio in the first place. 

Today, VDPV is expanding quickly in Africa, where it threatens to triggerbring potentially very large outbreaks of paralysis. One of us, Robert, is working on an article on the subject. But for now, there’s still too little discussion of this threat in popular media. So we are bringing the pieces forward in English for the first time. Part 1 is below, Part 2 is here. 

After nearly 30 years and $22 billion, a massive, dedicated campaign has nearly eradicated wild polio. But as the campaign prepares to celebrate the end of one terrible disease, cases of an equally terrible one of its own creation-- vaccine-derived poliovirus, or VDPV, are rising. VDPV outbreaks could be massive and require restarting the eradication campaign after it ends. So why do we hear almost nothing about it? 

In 1980, at a gathering of some of the world’s leading scientists, the person most responsible for  the world’s first ever successful campaign to eradicate a disease was trying to prevent the idea of eradication from going mainstream. 

DA Henderson, an American epidemiologist, had led the first campaign to successfully eradicate a disease. Smallpox killed an estimated 300,000 people in the twentieth century alone, and its official eradication in 1980 is still hailed as one of the great achievements in human health. As Henderson told the group, smallpox eradication had been a unique opportunity. The world had lept at a chance to wipe out the disease because unique circumstances meant it was possible. But, he added, “we have not demonstrated feasibility … with any other disease.” 

There was another problem. 

Henderson pointed out that the world’s health ministries widely agreed that the world should throw the weight of its healthcare resources into improving basic health infrastructure, to cope flexibly with multiple health problems. In a 1978 meeting in the Soviet Union, delegates from 134 nations had endorsed exactly this position under the slogan “Health for All,” with a goal of extending primary healthcare to all people by the year 2000. “A single-disease eradication program runs against this tide,” he said. 

Henderson’s warnings were largely ignored. At the end of the meeting, the assembled scientists nominated three diseases for eradication-- measles, yaws, and polio. Yaws is a common but rarely fatal disease in some tropical countries. WHO nearly eradicated it in the 1950s and 1960s. More recently, WHO has described yaws as a “forgotten disease.” By contrast, measles remains a major child killer, causing almost 90,000 deaths last year. Yet back  in 1988, WHO declared a goal of eradicating polio worldwide, formally launching a campaign which, nearly thirty years later, remains one of the most ambitious, most controversial, and best-funded health campaigns in history. 

… 

For the last two months, we’ve spoken with some of the people leading the polio eradication campaign about the campaign’s end stage. 

Our interest in this story came from different starting points. For the last several years, Robert, a former employee of Microsoft, has been writing a book about Bill Gates. In 2010, Robert wrote a story for Scientific American on how super-human efforts in India, where polio was worst, had turned the corner. At the time, it seemed like one of precious few, transcendently positive developments in the world. But polio eradication was never India’s idea; they have had many, much bigger public health problems. And, as global eradication drew closer, WHO declared polio an emergency-- even though the threat of spread was lower than ever in human history. 

Then a real emergency broke: Ebola. In 2014, I was covering the West African Ebola epidemic from the Washington, DC office of the US news site Mother Jones. After I wrote a story about the funding problems at WHO, Robert shared some of his blog posts, and we started talking more, eventually publishing a story on WHO, the US government, and the Gates Foundation during the epidemic for HuffPost. 

We found that while the Gates Foundation cared a great deal about polio eradication, in the early days of the outbreak, it had ignored and downplayed Ebola’s threat. But WHO had lacked the capacity to respond from the very beginning. In the middle of the crisis, WHO extended the polio emergency before finally declaring one for Ebola. The distorted priorities could be seen in the very make-up WHO. At the time, 38 percent of WHO staff in Africa were supported through polio funds. Polio eradication had become so big that WHO had been largely reorganized around it. How did that happen?

{What is polio, and how does it spread?}

One indisputable fact is that polio is a terrible disease, and one the world would be better off without. Polio attacks cells in the spinal cord that control the muscles. Affected people, usually children, lose the ability to move muscles, limbs, even their entire bodies. In the worst cases, when the virus affects the muscles which control breathing, polio can kill. 

In wealthy countries, where polio has been eliminated except for lab specimens, the virus has historically spread through saliva. But polio can also spread through human waste. This is especially a problem in poor countries, such as Pakistan and Afghanistan, where the virus still exists, and where sanitation is often poor and sewage water can blend with sources of drinking water. 

Polio has no cure. Though most people who contract it never show symptoms, for the minority who do, polio’s ravages can be mitigated but not reversed. 

But if the health benefits are one motivation for polio eradication, another, perhaps even greater one is the right to say “we did it.” Reading the many reports on polio eradication and talking to its proponents, the desire to reach the end seemed at times to outweigh the desire to end a cause of paralysis in children. 

Carol Pandak, director of the PolioPlus program at Rotary International-- historically one of the leading funders of polio eradication-- told us, “We made a promise to the world’s children, and Rotary keeps its promises.” 

“It’s a superhuman achievement of the globe uniting against a common enemy,” said Walter Orenstein, an early advocate of the cause.

Had the original goal of the eradication campaign been lost after thirty years? Or had polio eradication always been about inspiring people, as much-- or more-- than it was about improving the health of the world? Before charting out the future of the eradication campaign (which we’ll discuss in Part II of this series), we needed to look at the origins of the campaign.

{What is the polio eradication campaign?}

In 1988, the year the global eradication campaign officially began, there were more than 350,000 cases of naturally occurring, or “wild” polio in 125 countries. So far this year, 15 cases of wild polio in two countries-- Pakistan and Afghanistan-- have been reported, a drop of 99.99 percent. 

This massive reduction is the work of the eradication campaign. Today, what people typically call the eradication campaign is a formal entity called the Global Polio Eradication Initiative, or GPEI. Though its main office is housed at WHO headquarters in Geneva, WHO is, officially one partner among many. Others include the US government’s Centers for Disease Control and Prevention and nonprofit groups, such as the Bill and Melinda Gates Foundation and the volunteer group Rotary International. Though many governments support it, the Gates Foundation is currently the single largest contributor to the campaign. 

In principle, the campaign’s job is simple: to inoculate the vast majority of the world’s children against polio, year after year, until the disease disappears. In most wealthy countries, like the Netherlands, health care workers use an injectable vaccine. In most poor countries, the available option is an oral vaccine. Dropped on a child’s tongue a few times over a period of months, the vaccine stimulates the immune system in the digestive tract, virtually guaranteeing the person will be immune to polio for life. 

{What has eradication cost so far?}

But while vaccines are common component of public health regimens, the polio eradication campaign is not like any other public health effort in the world today. 

“Eradication is a very unforgiving goal,” says Orenstein. “If I told you that I have a public health campaign that reduced disease incidence by 99.99 percent plus, you’d say ‘wow, what a program.’ But with eradication, one infection is one infection too many.” 

Even polio’s double-digit figures are a nagging reminder that the campaign’s work is not yet done. Until the last case of polio is reported and isolated, the only way to prevent a resurgence is to vaccinate the vast majority of children everywhere, from Amsterdam to Tokyo to Zurich, from Armenia to Zambia, in cities, towns, and bedouin camps, war zones and refugee settlements. If vaccine rates drop before polio is gone forever, the risk remains that it could surge again. 

This means every year until polio is officially eradicated, the campaign must buy and ship hundreds of millions of doses of vaccine, particularly to poor countries which cannot afford it on their own. Since vaccine is perishable, the campaign pays for refrigerators, coolers, and generators to carry it from factories to the field, often in rural areas in remote villages. Since polio cannot be diagnosed in these hard to reach places, the campaign also maintains a network of labs in cities around the world which can test for it. And all over the world, in cities, towns, and villages, the campaign marshals an army of workers to carry vaccine to anywhere and everywhere there are children. 

After thirty years, sustaining this level of commitment has proven enormously complex and expensive. To date, the campaign has cost an estimated $15 billion. The campaign has also set-- and missed-- three deadlines-- in 2000, 2005, and 2012-- and is fast approaching a fourth, in 2018. At a current cost of around $1.25 billion per year, it ranks with HIV/AIDS and malaria as one of the most costly disease-specific campaigns in history. 

But was polio also one of the most destructive diseases? Not really. In 1990, two years after the campaign began but before it had substantially cut down infection rates in most of the poor countries where polio remained, the World Bank estimated polio constituted 0.24 percent of the burden of all diseases on the world economy. This is an imperfect measure, but it points to a fact that was widely understood at the time: polio was not one of the greatest disease threats in the world. By comparison, measles, often called the greatest killer of children in history, caused four million deaths in 1990, representing a more than ten-fold greater burden. Yet WHO had chosen to dedicate massive resources to its eradication anyway. Why?

{How was polio chosen?}

When we spoke with campaign leaders about its history, one name that came up repeatedly was Albert Sabin, the Polish-American inventor of the oral polio vaccine. 

The historical record reveals some interesting details about how he promoted polio eradication as a goal. Remember that, after the 1980 meeting in the United States, attending scientists largely agreed that eradication of some disease was a worthwhile goal, but they ended the conference with a list of three candidates, of which, polio was just one. 

Sabin had a preference, however, and it was polio. A few months after attending that meeting, he traveled the country to address thousands of members of Rotary International, a volunteer organization known for a well-connected membership dedicated to public service. Rotary members volunteered their time in their communities to help alleviate poverty. But in his address, Sabin said that, before it could eliminate poverty, the world first had to address the health problems of poor children, like polio. And as he explained it, Rotary members could lead this effort. 

Together, Rotary and Sabin came up with a number for global polio eradication: a mere $120 million.

In 1985, Rotary adopted global eradication as a goal. Soon after, the organization sponsored campaigns that used oral vaccine to either dramatically reduced or eliminated polio in the Philippines, Bolivia, and the Gambia. With these successes, healthcare leaders came to a realization: Polio was useful for attracting outside funders-- an attractive cause with a clear objective that could bring in money for the less easily defined mission of building healthcare.

“You can go to donors and say, ‘help us do this thing which will take a really long time,’ or you can go to them and say ‘help us eradicate this disease,’ and they like that other option more” says Svea Closser, an anthropologist at Middlebury College in the United States, who has studied polio eradication. 

Latin America was the next target. There, Rotary and local healthcare leaders appealed to governments and private donors, using the donations to eliminate polio and increase vaccinations for a number of diseases, including tetanus, pertussis and measles. Infection rates for widespread diseases plummeted while polio disappeared. 

Though early critics of polio eradication had worried that focusing on a single disease would take away support from everything else, the Latin American case suggested the opposite was true. Polio eradication was the rising tide that would lift the health of the entire region. 

{How did the campaign become global?}

By the late 1980s, WHO could no longer avoid joining the campaign. In 1987, several eradication leaders published a paper in WHO’s in-house journal calling for a global campaign. Polio eradication would be “a banner” which other health projects could rally behind, the group wrote. But these benefits would be ancillary. The priority would be eradicating polio once and for all.

“Global poliomyelitis eradication eradication… is inevitable,” they wrote. “The only question is whether we will accomplish it or pass on the needed action to our successors.” One year later, in 1988, the World Health Assembly declared a goal of eradicating polio worldwide by the year 2000. As healthcare workers had done in the Americas, the new campaign would use the oral vaccine.

In the Americas, polio eradication had been a common cause for an array of people wanting to lift the health of millions of people. But in becoming a global cause, eradication separated itself from any cause but its own. 

{What happened?}

Speaking with some of the people involved in polio eradication today, it was clear that many of them were tired of fighting polio. It’s hardly surprising: Many of them have been waiting for the end for decades.

But fatigue is not a new condition for the people working in and paying for polio eradication. Closser, the anthropologist, who attended meetings with leaders of WHO and other campaign partners in Pakistan a few years later, said the desire to finish the job narrowed the focus to polio alone. “I went to a lot of meetings where people said, ‘routine immunization is good, but we have to eradicate polio now or it won’t get done,’” she told us. “To get polio eradicated [they believed], they had had to push forward with this single-minded focus, and if they got too involved in routine immunization, it would slow them down and they would never eradicate polio.”

The campaign might have died around this time if not for the intervention of the richest man in the world. 

{How did Bill Gates get involved in polio eradication?}

In fall 2007, Steve Landry, the vaccine program manager at the Bill and Melinda Gates Foundation, asked Linda Venczel, a foundation program officer, to give a presentation on polio eradication to Bill Gates. Gates, whose aunt had been a polio victim and whose father was a Rotary member, was in the process of leaving Microsoft, the company he had founded, to become more involved in his namesake foundation.

Venczel had made similar presentations before, having spent three years as the deputy director of the polio eradication branch of the US CDC. What was different this time was the amount of money she had to budget for. 

“I wasn’t used to the kinds of budgets that we could throw at a problem as we could at the foundation,” Venczel said with a laugh. “When I asked, ‘how much money,’ I was told ‘you can start at $400 million.’ That was an incredible amount that could really be game changing.”

Over two hours, Venczel stood before the organization’s executive leadership, describing in detail the various options for eradicating polio within the allotted budget. Throughout the meeting, Gates, whose aunt had been crippled by polio when he was a child, peppered Venczel with technical questions. “One thing about Bill Gates is he’s very astute,” Venczel said. “He’s very quick to understand the science of everything, so he truly is a technical partner in thinking through things.” 

Then Gates, once a math major at Harvard, asked a big question. If he funded it, what percent chance did polio eradication have of succeeding? It was impossible to know for sure, so Venczel offered two other numbers. About 30 percent of the factors that would determine success-- like war, natural disaster, or political turmoil in the remaining countries-- were outside anyone’s control, she said. The other 70 percent depended on the campaign itself, and the money available to support it. Up to a point, with more money, the chances of success would increase. 

If he wasn’t already, Gates was persuaded. Before the meeting adjourned, he committed $700 million to polio eradication, nearly a third of a billion dollars over the original baseline.

We don’t know why, exactly, Gates chose to focus so much on polio. When we asked the Foundation, they directed us to several speeches Gates had made, where he made broad statements about uplifting humanity. But like some of the early proponents of polio eradication, Gates has promoted the idea of eradication itself as an idea. Even before he committed resources to polio, Gates had declared a desire to eradicate malaria. Since then, he has connected polio eradication to malaria more specifically. “Polio, we hope to get done by 2018,” he said in a 2014 interview. “The credibility, the energy from that will allow us to take the new tools we’ll have then and go after a malaria plan.”

Just as polio eradication was launched through the dedication of a very small group of people invested in the idea of eradication itself, under Gates’ leadership, that tradition continues.

{What’s next?}

With Bill Gates now serving as its de facto leader, polio eradication is closer to success than ever. But while the campaign’s leaders prepare to celebrate the end of wild polio, another problem is slowly threatening to undermine its progress.

The reason is that the oral vaccine, unlike the injectable vaccine used in wealthy countries, uses a weakened but still-living form of the poliovirus which retains a penchant for mutating back toward paralyzing virulence. The resulting infection is called vaccine-derived poliovirus, or VDPV. 

Since 2000, scientists have known that VDPV can spread through saliva and drinking water, and cause paralysis in children, just like wild polio. Technically, these cases are called Circulating VDPV, or cVPDV And while wild polio appears to be in terminal decline, VDPV cases are rising: In 2016, there were three cases of VDPV in the world. So far this year, there have been eighty. Scientists largely agree that VDPV will continue to exist after wild polio is gone.  

VDPV would not be a serious threat if vaccination rates remained high throughout the world. But with the eradication campaign ready to declare victory, many fear that VDPV infections could surge. By one estimate, cases of VDPV-associated paralysis could number in the thousands after wild polio is officially eradicated.

So why don’t we hear anything about this disease of the campaign’s own creation? 

Part 2: The long shadow of oral polio vaccine? The looming threat of outbreaks of 50,000 cases after “eradication”

https://undark.org/article/disease-eradication-malaria/

Co-written with Alex Park

In late August 2014, Tom Frieden, then director of the Centers for Disease Control and Prevention, traveled to West Africa to assess the raging Ebola crisis.

In the five months before Frieden’s visit, Ebola had spread from a village in Guinea, across borders and into cities in Liberia and Sierra Leone. Médecins Sans Frontières, the first international responder on the scene, had run out of staff to treat the rising numbers of sick people and had deemed the outbreak “out of control” back in June.

But when Frieden arrived in West Africa, the World Health Organization, the United Nations agency charged with coordinating the global response to disease outbreaks, had only just declared Ebola to be an international public health emergency. Although WHO had announced a $100 million Ebola action plan the week prior to that declaration, many major donors were still sitting on the sidelines....

Read the rest at Huffington Post

HER2 and Herceptin represent the biggest success story in breast cancer of the last two decades. All might be as advertised, but growing evidence raises key questions: Does HER2 drive breast cancer? Is it prognostic? Is the definition of HER2-positive clinically validated? Are HER2 assays reproducible? Can re-testing HER2 status change trial outcomes? Does HER2 predict benefit from Herceptin? Does Herceptin’s mechanism of action depend on HER2? Has Herceptin increased five-year breast cancer survival at the population level? Would Herceptin be approved by the FDA today? In the most recent trials of Herceptin and T-DM1, can the spectacular success of adding pertuzumab to Herceptin be squared with the failure of pertuzumab to make any difference when given with Herceptin-based T-DM1?

Examination of these questions (ten in total) casts doubt on the validity and clinical utility of the HER2 subtype and current treatment guidelines for Herceptin in breast cancer.

1.   Does HER2 drive breast cancer?

Summary: Evidence conflicts regarding whether HER2 drives cancer in humans; the decades-old experiments should be repeated to resolve the question definitively. Today, HER2 is the sole “Class I” oncogene that works by overexpression or amplification rather than mutation; no other type of cancer with a targeted therapy is driven by overexpression or amplification of the target. GRB7 is frequently co-amplified with HER2 and might be required to drive cancer, or GRB7 might contribute to oncogenesis independently of HER2. Conceivably, neither HER2, GRB7 nor any of the genes on the amplicon they share drive oncogenesis.

Robert Weinberg’s lab discovered that mutation of neu in rats drives cancer. But the analogous gene in humans, HER2*, is rarely mutated in breast cancer. Instead HER2 is either amplified or its protein product overexpressed. However, Weinberg’s lab found that amplifying neu 100-fold and increasing expression 10-fold did not transform mouse cells from normal to cancerous. Subsequently, the lab of Stuart Aaronson performed two similar experiments in the same mouse NIH/3T3 cell line but amplifying HER2 rather than neu. The first experiment confirmed Weinberg’s finding. But a second test used a different promoter that ratcheted HER2 expression five to ten times higher than in the first experiment, resulting in transformed cells, according to the researchers. Three decades later, however, Weinberg seems unpersuaded: the “report of Aaronson… may or may not have been independently replicated over the past 30 years since it appeared,” Weinberg wrote in email.

The experiments should be repeated. (I was not able to determine if HER2 has been shown to be transforming in human cell lines.)

HER2 is the only “Class I” oncogene that drives by overexpression and/or amplification

A 2004 census of amplified and overexpressed oncogenes found that just six met the most stringent “Class I” criteria. But by 2010, this fell to three: EGFR, AR and HER2. Now in 2016, HER2 stands alone as a Class I gene.

Class I genes must meet the lesser criteria of Class II and III genes and also require “that a drug that targets the encoded protein is used to treat patients for which efficacy must have been shown in clinical trials.” For EGFR in colo-rectal cancer, the targeted agent is Erbitux. However, although the FDA label for Erbitux lists EGFR/ERBB1 expression as an indication, the “consensus is that ERBB1 expression not required for therapeutic success,” according to Bert Vogelstein at Johns Hopkins University. In prostate cancer, amplification of the androgen receptor gene (AR) does not initiate prostate cancer but results from treatment. That leaves HER2 in a class by itself.

I asked Mike Stratton, a co-author of the 2010 census paper and director of the Singer Institute, if HER2 was indeed the sole Class I gene. Stratton replied: “I think that what you say is correct.”

A fundamental principle of targeted therapy is to attack tumor cells without harming normal cells. Consequently, targets are usually mutations. Herceptin’s ostensible target, however, is unmutated HER2. There is no driving oncogene for any human cancer where a targeted therapy aims for an unmutated target—except HER2 breast cancer.

GRB7

In cancer cells, amplified regions are quite common. The near absence of Class I genes, according to the 2010 census paper, “reflects the difficulty encountered in identifying the true cancer gene on amplicons that often include several candidate genes.” GRB7 resides on the same amplicon as HER2, and one research group found that both genes are co-amplified in 15% of invasive breast cancers. The same group suggested HER2 was not transforming by itself but required GRB7 co-amplification, creating the possibility that a “combination of multiple genes, which do not have independent transforming activity, causes transformation.” That is, HER2 might not be transforming. An analysis of the Cancer Genome Atlas concurred that “GRB7 may be necessary for cancer cells harboring this amplicon, as previously suggested.” Betsy Ramsey’s lab also studied GRB7 and HER2. In email, Ramsey summarized: “Certainly GRB7 seems to be a player with or perhaps without HER2.”

2.   Is HER2 prognostic in breast cancer?

Summary: Individual studies come to disparate conclusions about HER2 as prognostic in breast cancer. Reviews of the literature have been tagged with Expressions of Concern, leaving belief that HER2 is prognostic unsupported.

Women testing HER2-positive opt for more radical surgery than patients found to be HER2-negative. According to an analysis of more than 113,000 women, “mastectomy rates were higher in women with HER2-positive tumors than in those with HER2-negative tumors.” The reason is not known although the negative prognosis associated with HER2 breast cancer, long considered as “aggressive,” might be leading doctors and patients to pursue more aggressive treatment. But the evidence no longer supports HER2 as prognostic.

The HER2 prognostic literature begins with a 1987 paper from Slamon et al. which considered 86 node-positive patients from an unrelated clinical trial. (This might have made the cases non-random. In addition, the authors did not discuss whether the 86 were all from the same arm, raising the possibility of treatment confounding the analysis.) Slamon and colleagues found a statistically significant association between degree of HER2 amplification and recurrence and, to a lesser degree, survival: greater HER2 amplification increased the risk of recurrence and shortened survival. But when simply separating cases into amplified (n=52) vs. non-amplified (n=34), no difference in recurrence or survival emerged. To show such a difference and that HER2 was prognostic, the researchers dropped patients with a middling HER2 copy number of 2-4 (n=23) from the analysis. Based on the few remaining patients (n=11), with gene copy numbers of five or more, the authors reported a statistically significant difference in disease free survival (p=0.015) although still no difference in survival (p=0.06). Thus was born HER2’s reputation as an aggressive form of breast cancer.

More than one hundred studies followed Slamon et al., with an array of disparate results that might be expected given the methodology of the founding paper. However, the contradictory mess resolved into iron scientific consensus: “Initial conflicting reports regarding the prognostic relevance of HER2 were resolved with improved methodologies,” wrote Mark Moasser “and the overwhelming data now confirms this initial [Slamon et al.] landmark genetic-biologic finding.” The overwhelming data, according to Moasser, was “nicely reviewed” in a paper by Jeffrey Ross and colleagues.

Ross served as first author on three reviews of the HER2 prognostic literature, published in 1999, 2003, and 2009. However, in March of this year, all three reviews received an Expression of Concern from The Oncologist. (The EOC was prompted by my re-analysis of the Ross et al. reviews.)

The 2009 review included 107 papers. No search or inclusion criteria were specified, creating the possibility of selection bias. More important than how the 107 papers were chosen, however, the review contains errors on 30. More than one in four (28%) of the papers reviewed are mis-reported or should not have been included. For example, a paper from Battifora et al. is misreported: Ross et al. categorized it as “yes” under multivariate analysis of prognostic factors when the paper clearly did not find that HER2 was independently prognostic: 

“This analysis identified independent prognostic factors of DFS and OS when all variables were considered together. Independent predictors of DFS included stage of disease, histology, and nuclear grade. Nuclear grade and stage were the only significant predictors of OS.”

There are 10 errors of this particularly blatant sort among the 107 papers reviewed in Ross et al. 2009. A separate group of seven of the 107 papers conducted no multivariate analysis of whether HER2 was prognostic, but Ross et al. reported that those studies did and that each of the seven found HER2 independently prognostic. Six of the seven did not conduct a multivariate analysis of any kind; one of the seven did but all patients were HER2-positive.

An additional set of 11 papers should have been excluded. Nine of these correlated HER2 with different biomarkers, not clinical outcomes. Among these, one paper included 3,655 patients, by far the largest study in the review. Together, these 11 papers contributed 7,511 (19%) of the 39,730 patients in the review. Of these extraneous patients, 7,213 (96%) were adduced in support of HER2 being independently prognostic.

I contacted Jeffrey Ross regarding these errors. Concerning those in his 2003 paper, Ross acknowledged “scattered errors.” Ross disputed none of the 30 errors I identified.

There do not appear to be other literature reviews showing HER2 to be prognostic in breast cancer, leaving that belief unsupported.

3.   Is the definition of HER2-positive clinically validated?

Summary: There is no gold standard assay for definitively identifying HER2-positive tumors. The preferred assay and cutoff values have changed and changed back over time. But the modified standards are arbitrary, not clinically validated. The changes may reduce interobserver disagreement but do not increase accuracy in identifying true HER2-positives.

Changing assays: IHC vs. FISH

“There is no gold standard at present,” according to the most recent guidelines for HER2 testing, published in 2013. Perhaps for that reason, the preferred method for determining HER2 status has changed over time. Early research had shown that “amplification added little predictive value to the expression data.” Instead, the assay of choice, immunohistochemical staining (IHC), measured HER2 overexpression. The senior author of the paper, Dennis Slamon, subsequently led the first Phase III trial of Herceptin, for metastatic breast cancer. That study used immunohistochemical analysis and led to the first FDA approval of Herceptin in 1998.

However, not long after, re-analysis by the trialists showed that amplification measured by fluorescence in situ hybridization (FISH) predicted response better than IHC:

“FISH assays have higher sensitivity and higher accuracy and more frequently correctly identify altered HER-2/neu status (amplification/overexpression) in previously molecularly characterized specimens than did the FDA-approved immunohistochemistry assays interpreted manually.”

Of note, two of the authors, Michael Press and Dennis Slamon, also co-wrote the paper nine years before which found that “amplification added little predictive value to the expression data.” Now it was the opposite. Slamon-led researchers ultimately dismissed IHC with prejudice in 2005: “We do not consider immunohistochemistry screening for entry to clinical trials or for selection to Herceptin immunotherapy to be an acceptable strategy.”

The FDA, not long after approving Herceptin and IHC, voiced its displeasure about HER2 testing and the “many unanswered questions regarding HER2 detection systems…” The agency threatened to change the Herceptin label because of the “considerable confusion and misunderstanding on the part of the oncology community,” which was “significant enough to warrant general precautionary comment in the trastuzumab [Herceptin] label…”

By 2006, HER2 testing was broken, “a disorganized practice” with a “high rate of inaccuracy,” according to guidelines published that year by the American Society of Clinical Oncology (ASCO) and the College of American Pathologists (CAP). Both organizations had previously recommended HER2 testing but without seeing a need to specify how to do it.

Instead of choosing sides in the IHC vs. FISH debate, the new guidelines put the two assays together. No single kind of test sufficed to identify all HER2-positive tumors. An equivocal result from one assay would trigger another test using the other assay. The UK used this “two tier” system, and the ASCO/CAP 2006 guidelines proposed the same approach for the United States.

Although the UK guidelines detailed testing procedures, they did not cite clinical evidence in their support. Instead, the UK authors “expected that emerging data on accuracy of prediction of response to HER2 targeted treatments will influence the choice of testing method.” The US guidelines also adduced no evidence in support of the two tier system, acknowledging that, “current data are insufficient to define whether these patients represent true- or false-positives.”

Some of the ASCO/CAP panelists had wanted to scrap IHC entirely: “A minority view expressed within the panel was that IHC is not a sufficiently accurate assay to determine HER2 status and that FISH should be preferentially used.” Instead, between 80 and 90% of primary HER2 testing in the United States is done with IHC, and only 10 to 20% uses FISH, according to figures from 2008. A 2010 paper reported that 80% of HER2 assessments in the United States used the IHC-based HercepTest, manufactured by Dako. FISH costs more and requires more expensive equipment, a possible barrier for some hospitals and conceivably part of why FISH is not mandatory.

Herceptin trials were no help in shaping the ASCO/CAP guidelines: “the large prospective randomized clinical trials of trastuzumab were not prospectively designed to answer these questions.” Instead, retrospective “correlative studies” would have to suffice. Also, instead of seeking a true gold standard, any new assays would be measured by how well they reproduced the results of the old assays. According to the guidelines:

“Although a new HER2 assay ideally should have its clinical utility validated using specimens from prospective therapeutic trials that tested the effects of anti-HER2 therapy, the Update Committee recognizes that the rarity of these valuable specimens requires that new HER2 assays be approved on the basis of concordance studies comparing them with other established HER2 tests.”

In addition, the guidelines aimed for concordance rather than accurate diagnosis, with concordance substituting for accuracy. Inaccurate diagnosis is not possible to detect because there is no gold standard. By contrast, discordance can be measured, causes embarrassment and raises concerns at the FDA. As the guidelines recognized, however, interobserver agreement is not the same thing as accurate diagnosis: “concordance of assays does not assure accuracy (i.e., how close the measured values are to a supposed true value…).” Early UK guidelines strove to reduce “interobserver variation in the assessment of staining” by standardizing scoring “against known positive, negative, and borderline cases.” I asked Ian Ellis, corresponding author of the long-ago UK guidelines paper, whether the cases were clinically validated or if the staining patterns were selected for the purpose of minimizing interobserver disagreement. Ellis did not reply to multiple inquiries.

Cutoffs

In 2006, widespread HER2 assay discordance prompted ASCO/CAP to announce more stringent cutoffs than the FDA because “the original US Food and Drug Administration-approved interpretation guidelines provide insufficient specificity.” The FDA had approved an IHC staining threshold of 10% of cells. But the panelists believed this resulted in an unacceptably large number of false positives. The guidelines recommended a higher cutoff of 30%, not based on published evidence but anecdote, “the cumulative experience of panel members that usually a high percentage of the cells will be positive if it is a true IHC 3+.”

The guidelines referred to “published reports using cutoff values higher than 10%,” but the footnote pointed to a single study in France which achieved 95% concordance between IHC and FISH by using a vastly higher staining cutoff of 60%. Why the revised US guidelines recommended a 30% cutoff is not clear.

The cutoff for FISH HER2/CEP 17 ratio was also raised, from 2.0 to >2.2. In addition to changing the definition of IHC3+, IHC 2+ patients were no longer considered HER2-positive as they had been in the original, FDA approval-winning trial of Herceptin.

But then seven years after raising the IHC thresholds, the ASCO/CAP guidelines rolled them back. The 2013 guidelines committee “decided to revert to the previously used IHC criterion of more than 10% cells staining.” Re-examination of one trial with 2,904 patients found that the higher threshold only excluded 107 or 3.7%. This seemed to contradict the “cumulative experience” of the 2006 panel that “usually a high percentage of the cells will be positive if it is a true IHC 3+.” Now it seemed to be remarkably rare for IHC 3+ to have more than 10% staining.

The re-examination found that patients not meeting the higher ASCO/CAP guidelines showed no difference (p=.55) in disease free survival whether they received Herceptin or not. But the guidelines were switched back nonetheless.

The threshold for IHC had been raised in 2006 out of concern for false positives. But a then-ongoing study assuaged this worry. The 2013 guidelines said that study found “less than 6% of patients initially considered eligible were not subsequently centrally confirmed as being HER2-positive.” The US, at least, appeared to have its house in order.

The 6% figure came from a central laboratory at the Mayo Clinic that re-examined about one thousand locally-tested patients. However, central vs. local testing in Europe of more than 8,000 patients yielded discordant results in 15% of cases. And when the two US and European central labs later compared results, examining only samples known to be false negatives, they differed on IHC scores for 6 of 25 cases (24%) while FISH scores differed for 3 of 25 (12%). Moreover, the Mayo Clinic systematically assigned higher IHC and FISH scores to a set of 23 cases previously judged as equivocal in local testing. Of the 23, the Mayo Clinic found 15 to be HER2-positive while versus 11 according to the European lab. In other words, the Mayo Clinic might have generated a high percentage of false positives, the concern that had led to raising the IHC staining threshold to 30%, which the 2013 guidelines rolled back.

Which central lab was right? Did the high US concordance rate mean US labs were correctly identifying true HER2-positive patients and the European lab was wrong? “It is not possible to know,” according to the ring study paper, “which central laboratory determination of HER2 status… was biologically correct in terms of distinguishing patients who do or do not benefit from HER2-targeted… therapies.”

4.   Are HER2 assays reproducible?

Summary: Neither IHC, FISH, local or central testing generate reliably reproducible HER2 results. Re-examination of the clinical trials leading to FDA approval of Herceptin found discordant HER2 status on as many 26% of patients. Technical shortcomings of both IHC and FISH assays contribute to reproducibility problems. Each laboratory may be using its own cutoff criteria in judging these semi-quantitative assays. The type of assay, its manufacturer and who performs the test can decisively influence the HER2 status of any given patient.

The major trials leading to FDA approvals of Herceptin in breast cancer have been re-examined, producing substantially different results for patients’ HER2 status. As mentioned, the breakthrough trial leading to the first FDA approval of Herceptin depended on IHC. But re-examination found amplification measured by FISH predicted response to Herceptin more accurately; i.e., FISH did not reproduce IHC.

Central testing frequently contradicts local results, a problem which affected both US trials that supported FDA approval of Herceptin in the adjuvant setting. Central retesting of patients in NCCTG N9831 failed to reproduce a local HER2-positive result in as many as 26% of re-examined cases. Similarly, retrospective analysis of NSABP B-31 found 18% of tumors were HER2-positive according to local testing but HER2-negative by central testing using both IHC and FISH.

Trial MA.31 compared Herceptin and lapatinib in metastatic breast cancer in 652 patients found HER2-positive by local testing. However, central re-testing found 115 patients (18%) were not HER2-positive, although they had been enrolled in the trial and treated with an anti-HER2 therapy.

False negatives are also a problem. A 2014 study of 552 locally HER2-negative patients found that 4% were HER2-positive by central testing.

Non-reproducibility also impacts central laboratory testing. As mentioned, a US and European lab each examined the same set of 23 equivocal HER2 cases. The US lab found 15 (65%) HER2-positive while the European lab found only 11 (48%) HER2-positive.

Contributing to reproducibility problems are shortcomings of the tests themselves. In 2006, the ASCO/CAP HER2 testing panel had considered throwing out IHC, the original technology for identifying HER2-positive breast cancer. Skepticism, even condemnation, of IHC continues to this day. “The IHC assay is lousy,” according to Bert Vogelstein at Johns Hopkins University. “No IHC assay is great, many are inaccurate,” he added. FISH, according to Vogelstein, “is not that great either, but it’s the best that the pathologists have.”

For both IHC and FISH, the handling of samples before the test can affect test results. Also, the assays are only semi-quantitative. As the FDA observed in 2001, it “views both IHC and FISH as semi-quantitative if performed under ideal circumstances.” In addition, “Both methods require subjective interpretation.”

According to David Rimm, a HER2 testing expert at Yale Medical School, each lab has its own cutoffs, which he considered a “dirty secret.” (In reply to my initial inquiries about issues with HER2 testing, Rimm replied: “You are about to uncover a landmine.”) The College of American Pathologists (CAP) sends HER2 testing facilities samples to measure and encourage adherence to common, cross-laboratory criteria for HER2-positives and negatives. But according to Rimm, “the College doesn’t send them too many hard ones,” possibly to avoid generating discordant, non-reproducible results.

An additional complicating and underappreciated problem is that tumors are sometimes heterogeneous. A biopsy from one part of a tumor can test HER2-positive while a sample from a different part of the tumor tests negative. Researchers reporting such a case wrote: “We do not know the frequency with which a disparity of this degree occurs, but it is not even mentioned in reviews on this subject or consensus guidelines published previously. We therefore assume that it must be a rare phenomenon or one clearly underappreciated.”

According to Kornelia Polyak of the Dana Farber Cancer Institute, the phenomenon is not rare: “This is a pretty serious problem as we see that ~30-40% of HER2+ tumors have high heterogeneity for HER2 itself within the tumor.”

Not only do different labs sometimes disagree about assay results for a given specimen, in addition, where the tissue sample comes from in the tumor can determine whether a patient is deemed HER2-positive or HER2-negative. (Also tumor cells can interconvert between HER2-positive and HER2-negative states. See question 6.)

5.   Can non-reproducible HER2 assays alter trial outcomes?

Summary: Central and local testing can produce conflicting HER2 assessments. In at least one trial, central retesting changed the outcome of the study. There are implications not only for bioethics but a forthcoming meta-analysis that attempts to measure Herceptin’s effects across trials, some with multiple, conflicting HER2 assessments.

The outcome of trial MA.31 depended on whether local or central HER2 determinations were used. By central testing, Herceptin extended life more than lapatinib; by local testing, there was no difference. Recall that in MA.31, local testing identified 652 patients as HER2-positive but central re-testing later found 115 (18%) weren’t HER2-positive.

This makes trial interpretation difficult or impossible, while the timing of the tests created an ethical conundrum. The central re-testing occurred during MA.31 according to Karen Gelmon, the study’s corresponding author. Regarding the unusual timing, Gelmon said: “the thought was to make it easier for the patients and doctors to use local HER2 for randomization to avoid a delay in starting treatment…” However, in speeding patients into the possible benefits of an untested treatment regimen, the design and conduct of the trial resulted in centrally HER2-negative patients being treated with anti-HER2 therapies.

Gelmon said “the central results are considered definitive.” But most patients, including those that were centrally HER2-negative, appear to have completed the study. Regarding this ethical conundrum, Gelmon said: “If the central confirmation showed negative results it was up to the treating physician to decide how to treat, which is always how it is, and they could continue the Herceptin if they thought the local testing was valid.” Gelmon did not reply when asked how many patients stopped receiving anti-HER2 treatment after being found HER2-negative by central testing. It is not clear if patients were informed of the equivocal test results. Anti-HER2 therapies are of course not approved for use in HER2-negative patients because the benefits, if any, are outweighed by toxicities and other side effects.

Although this ethical dilemma arose in a clinical trial, it conceivably impacts every breast cancer patient. It is unclear whether to believe local testing, central testing or neither. Gelmon doubled-down on central testing: “yes – central or validated testing is what should be recommended.” However, the ASCO/CAP guidelines recommend only that the testing laboratory be accredited. Gelmon simultaneously regards central testing as definitive but supports optionally ignoring it. Edith Perez, prompted by FISH-negative cases who later turned out HER2-positive by IHC, recommended that “in the case of negative results, it’s advisable to repeat the test you started with or to run a different test,” perhaps making it sound like testing should be continued until the result is positive. Perez led the NCCTG N9831 study.

It is unclear that a coherent testing algorithm is obtainable from the recommendations and practices coming from these clinical trials.

CTSU

The Clinical Trials Service Unit (CTSU) at Oxford is conducting a meta-analysis of Herceptin in early breast cancer. But how will the study define “HER2-positive?” For trials with two sets of assessments, the meta-analysis will have to choose between them (and ignore one set) or present results for both local and central testing. However, not all trials re-tested HER2 status, further complicating the aggregation of Herceptin’s effects across trials.

In addition, the type of assay used might be important and worth reporting. “Scientists who actually do these assays (rather than see the reports of the results) know that neither of these assays (FISH or IHC) are particularly reliable on clinical samples,” said Bert Vogelstein. Given the actual complexity and uncertainties in HER2 assessments, CTSU could (and should) examine their accuracy, computing the likelihood of an assessment being correct and/or linking assessment accuracy estimates to the confidence interval around Herceptin’s clinical benefit. Since there is no way to identify “true” HER2-positives, perhaps the best that can be done is to calculate the likelihood of concordance or discordance if a sample were subjected to re-testing.

I asked CTSU’s Richard Gray: “Will your study use the initial or retested results for those trials? How will your meta-analysis deal with the mixture of protocols for determining HER2 status across trials?” Gray replied indirectly: “One prime aim of the meta-analysis will be to investigate whether there is benefit in HER2 receptor equivocal patients, and we’ll collect results of all available local and central assays to look at this.”

However, arguably the real problem is how to perform a meta-analysis of randomized clinical trials where a main variable, HER2, was not controlled.  Kornelia Polyak believes Herceptin works, but observed: “If you pick variable patients by definition you will have variable responses leading to confusion.”

6.   Is HER2 a valid biomarker that predicts benefit from Herceptin in breast cancer?

Summary: The published literature no longer supports the validity of HER2 as a biomarker for Herceptin in breast cancer. Both US trials leading to FDA approval of Herceptin in early breast cancer later found HER2 did not predict benefit from Herceptin. In small subgroups, HER2-negative patients appeared to benefit more than HER2-positive patients, and Herceptin is now being tested in HER2-negative patients. Alternative biomarkers for Herceptin have been proposed but none accepted.

The FDA approved Herceptin in 2006 for early breast cancer based mainly on two US trials: NCCTG N9831 and NSABP B-31. Both trials later announced some patients had been misdiagnosed as HER2-positive, making it possible to examine clinical outcomes of patients negative for HER2 by central testing who had been treated with Herceptin.

In 2007, one year after helping win FDA approval of Herceptin, B-31 trialists reported neither FISH nor IHC predicted response to Herceptin: “No statistical interaction was found between DFS benefit from trastuzumab and levels of protein (p=0.26) or HER2 gene copy number (p=0.60).” However, although the B-31 trialists wrote of “no statistical interaction,” it appears that HER2-negative patients benefited more than HER2-positive patients. The subgroups were small but nonetheless the researchers reported significant values for each of them, turning the HER2 world on its head.

HER2-negative patients benefited more from Herceptin than HER2-positive patients in NSABP B-31. (Adapted from Paik et al., 2007)

In 2013, B-31 trialists sought a new biomarker, reiterating that “HER2 itself failed to show predictive interaction with trastuzumab…”

The second trial key to FDA approval, NCCTG N9831, corroborated B-31’s finding that FISH did not predict response to Herceptin. A 2010 re-analysis of N9831, again by the original investigators, found “Trastuzumab benefit seemed independent of HER2/centromere 17 ratio and chromosome 17 copy number,” i.e. independent of FISH.

The two trials which had established HER2 by IHC and/or FISH as the biomarker for Herceptin subsequently disestablished both. On this basis alone, HER2 would seem to no longer be a valid biomarker for predicting response to Herceptin. Logically, HER2 status no longer stands as a valid indicator for treatment with Herceptin. We have known this since 2010.

FISH and IHC might have found redemption in Hera, the trial that led to approval of Herceptin in the adjuvant setting in Europe and also supported FDA approval in the US. Post-approval, Hera trialists examined the relationship between degree of HER2 amplification by FISH and benefit from Herceptin. But they chose not to examine 41 patients with a FISH ratio under two, i.e. HER2-negative by FISH. “We deemed it inappropriate to analyze this small group,” wrote the investigators. Consequently, they could not say how FISH-negative patients responded to Herceptin and whether or not IHC by itself predicted response.

Without looking at the FISH-negative group, researchers continued to posit “a strong threshold effect whereby any degree of amplification above the cutoff ratio of 2.0 is of equal clinical significance.” However, B-31 previously and N9831 found no threshold among a combined 330 FISH-negative patients. The Hera team simply looked away.

The Hera trialists also examined IHC staining intensity and clinical outcomes. This time, IHC-negative patients were not included, which prevented analysis of whether FISH predicted response to Herceptin. Hera investigator Mitch Dowsett explained, in June 2014: “Because of our policy on recruiting only centrally confirmed HER2-positive cases to Hera we were not in a position to do this.” However, apprised that Hera enrolled at least 299 centrally confirmed, HER2-positive patients who were IHC-negative, Dowsett revised his explanation. “I think ‘policy’ is overstating things. We could and maybe should have looked at this group in more detail previously.” But, “prompted by a UK pathologist,” rather than the failure of IHC to predict response in B-31 and N9831, Dowsett said the Hera trialists would examine the IHC-negative, FISH-positive subgroup.

In August 2015, more than a year later, I asked Dowsett how the project was going. “The work was conducted and a manuscript created,” he replied. But then the primary investigator, Bharat Jasani, “left for [a] job in Kazakhstan,” said Dowsett, stalling the investigation. I emailed Jasani and asked: “Were you examining IHC-negative, FISH-positive cases from Hera before leaving for Kazakhstan?” Jasani seemed to contradict Dowsett: “The simple answer is no and I would like to confirm once again that I have not examined at any time any IHC-negative, FISH-positive cases from Hera.”

Analyses of key subgroups in the Hera trial appear to have been avoided. As it stands, every re-test of any assay used to assess HER2 in the FDA approval-winning trials in early breast cancer found that that assay did not predict benefit from Herceptin, or that being HER2-negative predicted greater benefit.

Similar to the Hera trials evasions, HER2 testing experts also avoided addressing HER2’s validity as a biomarker when I raised the issue to them in 2014. I emailed John Bartlett, at the Ontario Institute for Cancer Research, asking: “what established HER2 as a biomarker and what data informed the cutoff point for positive vs. negative?” Bartlett previously co-authored HER2 testing guidelines. His assistant replied: “John says he should be able to answer it via email.” However, Bartlett eventually wanted to speak by phone. When I requested email, the assistant wrote back: “Unfortunately Dr. Bartlett is unable to answer this question.”

The lead author of HER2 testing guidelines, Antonio Wolff, wrote me that FISH “Absolutely yes” predicts response to Herceptin even though N9831 and B-31 showed it did not. “I do fear that the dots you are connecting don’t quite tell a story,” Wolff said. Rather than explain, he wrote: “I think I will stop here.” He requested to speak by telephone but would not allow recording it: “Recording our conversation will not be ok and you do not have my permission.” He added: “My goal was to walk you through your questions informally as an expert source.” Arguably, Wolff declined to go on record to explain why FISH remained valid.

Reliably identifying HER2-positive patients might be impossible. According to Daniel Haber, at Massachusetts General Hospital, “Whether there are ‘true HER2’ tumors or not is up for discussion.” Instead of HER2 status predicting response to Herceptin, response to Herceptin determines who is HER2-positive. Said Haber: “the real definition is probably whether they [patients] respond to HER2 therapy or not…” But if so, HER2 is not a valid biomarker for Herceptin, and Herceptin has no valid biomarker, and prescribing Herceptin for HER2-positive patients makes no medical sense.

7.   Does Herceptin’s mechanism of action depend on HER2?

Summary: Recent research suggests Herceptin does not block HER2 signaling, once considered its mechanism of action. No new mechanism of action has been clearly established. Some researchers believe Herceptin might work in HER2 0 patients, i.e. independently of HER2 status.

Herceptin does not block HER2 signaling

 “[T]he talking points, the posters, the advertisements, are all about ‘HER2 blockade.’  It makes a good story, much simpler to understand, very pretty pictures, and nicely amenable to commercialization. Unfortunately it's not true.”

So wrote Mark Moasser, at the University of California at San Francisco, in email. More formally, in a published paper, Moasser wrote that Herceptin “was developed on the basis of 1980s understanding of HER2, and it is now clear that it does not actually inhibit HER2 signaling functions very well.” Tyrosine kinase inhibitors like lapatinib do block HER2 signaling but the clinical benefits of lapatinib are scant. Dual anti-HER2 therapy in which lapatinib is added to Herceptin showed no survival benefit in either the adjuvant or neoadjuvant settings as tested in the ALTTO and NeoALTTO trials. A lapatinib-only arm in ALTTO was closed early due to futility.

Additionally, there appears to be no consensus whether degree of HER2 positivity increases response to Herceptin, with greater amplification or overexpression leading to more pronounced clinical benefit. Also unexplained is how Herceptin might work in patients positive for HER2 by amplification but negative for overexpression. Krop and Burstein further fragment the HER2 edifice: in wondering “qui bono” or who benefits from Herceptin, they posit that “the mechanisms may differ in early- and late-stage breast cancer.”

Also, tumor cells appear to convert back and forth between HER2-positive and HER2-negative. Thus HER2 expression “identifies dynamic functional states,” according to Jordan et al.  Interconversion may make tumors heterogeneous for any HER2 signal and might partly explain difficulties linking HER2 expression to any tumor phenotype.

NSABP B-47

HER2 may have nothing to do with Herceptin’s mechanism of action: “We don’t know that trastuzumab would not work in the adjuvant setting for HER2 0 patients,” according to Lou Fehrenbacher.  Fehrenbacher is leading a trial, NSABP B-47, which tests Herceptin in HER2-negative patients. The trialists considered enrolling HER2 0 patients, but according to Fehrenbacher, this was deemed “too adventurous.” It might have undermined the entire HER2/Herceptin story. Instead of asking the question: “do Herceptin’s effects have anything to do with HER2,” B-47 answers the question: “Should HER2 low patients also be treated with Herceptin?” a stepwise distancing from current orthodoxy rather than quick, complete abandonment. B-47, which only includes HER2 1+ or 2+ patients, might also result in a large increase in patients treated with Herceptin. According to Fehrenbacher:

“[T]he number of women with 1+ and 2+ non HER2-positive tumors in the US, is 4x the number with HER2-positive. So if the trial is successful the number of women benefiting from trastuzumab will rise to a level 500% of the current number.”

By contrast, a trial design including HER2 0 patients might have shown no relationship between Herceptin and HER2 or perhaps an inverse relationship like the re-analysis of B-31.

B-47 represents an opportunity to test both whether Herceptin works in HER2 0 patients and whether the degree of HER2 positivity predicts greater benefit from Herceptin. B-47 should add an arm of HER2 0 patients allocated to Herceptin or placebo. In addition, a partial arm of 3+ patients should be added, all receiving Herceptin, to allow comparison of the drug’s effect across the range of HER2 positivity, from 0 to 3+.

ADCC

There is no agreed upon alternative mechanism of action for Herceptin. A leading but unproven candidate is antibody dependent cellular cytotoxicity (ADCC). The current FDA label says “Herceptin is a mediator of antibody-dependent cellular cytotoxicity,” but only based on in vitro evidence. For clinical evidence, Mark Moasser pointed to “A recent landmark study… that showed response/resistance to trastuzumab is powerfully predicted by the immunological signature.” This re-examination of NCCTG N9831 found “that immune function genes are strongly linked to clinical outcome.” The authors proposed a complicated signature comprised “of any nine or more of 14 immune function genes at or above the 0.40 quantile for the population.”

But a critique by NSABP B-31 trialists found that randomly selecting any 14 genes at any expression level resulted in an interaction probability of less than 0.01 in 92% of 10,000 runs conducted using data from 731 patients in B-31. Consequently, “the conclusion that immune-related genes are driving the observation may not be valid because this criterion can be eliminated without effect on model performance.”

ADCC remains a hypothesis. “As far as I’m concerned, the jury is still out whether Herceptin works by ADCC, through other indirect mechanisms, or through interrupting some signaling pathway,” according to Bert Vogelstein. “If it does involve ADCC, it would have to discriminate between low and high amounts of cell surface ERBB2 protein.” It is “not so obvious” how Herceptin might do this given the widely varying levels of ERBB2 protein on cancer cells even in HER2-positive tumors.

In 2010, Edith Perez recommended against Herceptin for HER2-negative patients. One reason: “It doesn’t make any biological sense,” according to Perez. If Herceptin’s mechanism of action is independent of HER2, seemingly it would not make biological sense to recommend Herceptin for HER2-positive cases.

Ultimately, however, Mark Moasser is not worried by there no longer being an agreed upon mechanism of action for Herceptin: “At the end of the day, it doesn’t really matter what the mechanism is, as long as it works.”

8.   Would the FDA approve Herceptin today?

Summary: Herceptin’s approval in the metastatic setting benefited from a new FDA fast track. The single phase III trial providing the basis for approval underwent extensive mid-trial modifications—adding different treatment arms and unlike patients—practices that are no longer permitted. Avastin later faced a different FDA process which led to revocation of its approval.

In early breast cancer, had Herceptin’s FDA application been based on the re-analyses of NCCTG N9831 and NSABP B-31, it presumably would have been rejected. The initially impressive results presented to the FDA likely required the heavy modifications made to them, including merging N9831 and B-31 together while dropping one arm which showed no survival benefit for Herceptin. It is unlikely such changes would be allowed today. The trials were enabled and shaped by new NCI policies that allowed cooperative groups like NCCTG and NSABP to conduct phase III trials in support of FDA approval while permitting those groups to accept funding from pharmaceutical companies.

Herceptin in metastatic breast cancer

Fast track

Genentech’s Herceptin first won FDA approval in 1998 for metastatic breast cancer. With the process taking just five months, Herceptin benefitted from being the second drug to come off a new FDA fast track. Public perception at the time was that an approval logjam was blocking life-saving cancer drugs from reaching patients. But going faster required relaxing standards. On the fast track, “potential” effectiveness was to be considered with the standard only that “potential effectiveness of the treatment should outweigh its toxicities.” These educated guesses would not necessarily have to be checked later: “A post-approval study will not necessarily be required in the exact population for which the approval was granted.”

Trial H0648g provided the basis of Herceptin’s FDA application. But according to the FDA review, “multiple major changes in the protocol were enacted during the conduct of the study.” The biggest mid-course change added entirely new arms to the trial after enrollment of only about 100 patients. The original design tested Adriamycin and cyclophosphamide (AC) against AC + Herceptin (H). The new arms tested a taxane (T) against T + H.

The new arms were then pooled with the original arms—to the chagrin of the FDA. It considered patients in the AC and T arms as “clinically distinct.” The taxane cohort represented a “different prognostic group” than the AC patients and “baseline characteristics differed markedly between paclitaxel and AC patients regardless of assignment to Herceptin therapy or not.” However, the FDA acquiesced on pooling.

Remarkably, as arms were added, the double-blind with placebo design was dropped and the trial became open label. “Patients and investigators object to the placebo,” said the FDA, again accepting a fait accompli.

The trial found adding Herceptin to AC made no difference in overall survival. Similarly, in the new taxane arms, adding Herceptin did not increase survival. But the pooling of the AC and taxane arms, which the FDA had frowned upon, produced a statistically significant overall survival benefit, albeit with a confidence interval touching 1.0.) But absent the large, mid-course alterations to the trial, Herceptin would have shown no survival benefit.

Avastin

Avastin, also from Roche/Genentech, lost its FDA approval for treating breast cancer after post-approval trials failed to demonstrate a survival benefit. Genentech proposed Avastin for treatment of metastatic HER2-negative breast cancer. As with Herceptin earlier, an accelerated FDA application for Avastin relied on an open label trial, E2100. Although the FDA initially approved Avastin, the review scolded the drug sponsor: “Genentech did not meet with FDA to reach agreement on the design of Study E2100 prior to study initiation.” The FDA found a host of problems with trial E2100 including the open label design and loss of patients to follow-up:

“[T]he effect on PFS by an independent group, masked to treatment assignment, was not implemented during the conduct of the trial. Retrospective analyses by an endpoint review team masked to treatment assignment to independently confirm the E2100 results was marred by substantial loss to follow-up prior to the independent review team’s confirmation of disease progression."

In addition, the lack of independent review led to investigator bias—toward Avastin. According to the FDA, “the discordance rates are slightly different for the two study arms, with the difference favoring the PAC/Bev [Paclitaxel/Avastin] arm over the PAC arm in ECOG investigator-determined assessment of PFS.” The FDA also looked at missing and data and found that a worst case analysis resulted in “elimination of the treatment effect altogether.”

The FDA examined financial ties to the sponsor and found five of the sixteen members on the data monitoring committee members received payments greater than $25,000 from Genentech. A sixth reported compensation that “could be affected by the study outcome.” In addition, “Eight out of 26 investigators (30%) who provided financial disclosure in the E2100 study administration body and data monitoring committee reported financial conflict of interest for receiving payment from pharmaceutical companies.” One of the study co-chairs “failed to reply to the Financial Disclosure requests.”

For Herceptin, by contrast, the FDA did not examine financial ties of trial investigators. But when the study was published in the New England Journal of Medicine, nine of the 12 authors reported relationships with Genentech. The FDA allowed arms to be added mid-trial for Herceptin whereas for Avastin, simply starting a trial without it being OK’d by the FDA drew censure.

Subsequent testing of Avastin in a double-blind, placebo-controlled design required by the FDA found no overall survival benefit, and the FDA revoked its approval of Avastin for breast cancer. For Herceptin to show a survival benefit in the metastatic setting had required pooling of arms the FDA regarded as distinct in an open label design.

Herceptin’s approval hurdles were lower; it might not have met later, higher standards.

Re-analyses re-visited

In early breast cancer, the FDA approved Herceptin in 2006 based on a joint analysis of NCCTG N9831 and NSABP B-31. But by 2007, B-31 trialists reported that HER2 didn’t predict response to Herceptin, whether measured by IHC or FISH: “No statistical interaction was found between DFS benefit from trastuzumab and levels of protein (p=0.26) or HER2 gene copy number (p=0.60).” And although the authors reported no statistical interaction, HER2-negative patients appeared to benefit more than HER2-positive cases. Corroborating B-31’s results, N9831 trialists reported in 2010 that FISH did not predict response to Herceptin: “Trastuzumab benefit seemed independent of HER2/centromere 17 ratio and chromosome 17 copy number…”

Had these results been presented to the FDA when considering the application for Herceptin in early breast cancer, the application presumably would have been rejected.

“Joint” trial

What the FDA saw in the Herceptin application was a single successful trial, which was actually made from two studies merged together, with one arm discarded. The unplanned changes were made while the trials were in progress.

In N9831, Arm B tested sequential Herceptin in roughly one thousand women and ultimately showed no overall survival benefit from Herceptin: five-year survival for arm B was 89.3% versus 88.4% in the control arm. Arm B was dropped when N9831 was joined to NSABP B-31.

Although the FDA went along with merging the trials, the oncology community was divided. In 2006, one specialist noted that “In terms of combining the data from the two trials, some oncologists were initially questioning whether that was legitimate.” Sandra Swain, who was at NCI when the trials were joined, answered it was “clearly legitimate.” She asserted that the trials were combined because they were going well: “No one had any idea that we’d have the benefit that we do.” However, joining trials increases statistical power, enabling detection of weaker effects while, obviously, dropping a low or non-performing group of patients might have enhanced the perceived effects of Herceptin in the remaining arms.

An FDA spokesperson offered conflicting answers in 2014 regarding whether the individual trials would have met their endpoints, initially saying: “The FDA cannot speculate on if the trials would or would not have met their original endpoints.” But subsequently the spokesperson speculated that the trials would have been “likely to demonstrate efficacy as individual trials…”

No results for B-31 have been published. A number of researchers suggested in a letter, “Trastuzumab: possible publication bias,” published by the Lancet in 2008, that the results of the individual trials should be published separately. Asked in 2014 for efficacy data, NSABP’s Soon Paik declined, saying only that “they are essentially the same as what is in the combined analysis.”

A meta-analysis of Herceptin being conducted by the Clinical Trials Services Unit (CTSU) at Oxford will include all N9831 patients, including Arm B. According to CTSU’s Richard Gray: “We will analyse the combined concurrent and sequential trastuzumab arms versus no trastuzumab from the 3-way randomisation periods of N9831,” as well as the concurrent and sequential arms separately.

Cooperative Groups

The N9831 and B-31 trials were conducted by cooperative groups, the North Central Cancer Treatment Group (NCCTG) and the National Surgical Adjuvant Breast and Bowel Project (NSABP) respectively. Originally, cooperative groups were funded by NCI. However, in 2000, NCI allowed cooperative groups to accept industry funding. And two years before, the FDA said it would accept trials performed by cooperative groups to support applications for FDA approval. Previously, cooperative groups mostly conducted phase II trials. Arguably, these decisions transformed a public and publically-funded system into one dominated by pharmaceutical companies. B-31 and N9831 were started around the time of the new NCI and FDA policies, in July 1999 and April 2000.

In 2002, the FDA sought to tighten a number of clinical trials policies, but they were opposed by the cooperative groups. NSABP, joined by NCCTG, challenged the FDA reforms in a letter signed by John Bryant, the statistician for the joint N9831/B-31 trial. The FDA had sought to treat the cooperative groups as a sponsor, perhaps because they had begun receiving industry funding. Also, the FDA wanted more blinding of study teams and greater independence of statisticians preparing reports. But the NSABP letter answered that “it will not be practical to arrange for statisticians independent of the Cooperative Groups to prepare and present interim reports…” There were too many trials and “simply not enough qualified personnel available to do so.” The cooperative groups claimed these and other proposed changes would have a “substantial negative impact” on clinical trials including even patient safety.

Concern about industry funding of previously trustworthy cooperative groups surfaced at a 2009 NCI workshop on “Multi-Center Phase III Clinical Trials and NCI Cooperative Groups.” As one participant said:

“If we do not have a robust independent review of these trials, the criticism will be raised quite quickly that these trials are being done by industry and that public dollars should not pay for them. What will protect these trials is that they have a very robust independent review, not just a cooperative group–only review.”

The FDA audited none of the US Herceptin trials. According to the FDA medical review of the joint N9831/B-31 trial, “A DSI [Division of Scientific Investigations] inspection was not performed for this application; given the large number of sites and small percentage of patients enrolled at any individual site, no single study or limited number of sites would have substantial impact on the study results.” If multiple sites and widely distributed patients protect against improprieties, then perhaps no phase III trial would ever need to be audited.

The FDA was unable to confirm that the cooperative groups audited their Herceptin trials: “Because of the nature of the conduct and reporting of the clinical site audits, it cannot be determined whether a specific study was audited during the clinical site inspection…” A statement by the sponsor about audits provided “no information on the actual results of site audits,” according the FDA review of Herceptin.

(I suggested to Richard Gray that the CTSU Herceptin meta-analysis could attempt to reproduce the results of the individual studies as one kind of check on the un-audited trials.)

The possibility of investigator bias was not examined. The FDA “did not request confirmation of the events by an independent endpoint assessment panel that was masked to treatment assignment.” The FDA reported “approximately 4% of the population in the ITT efficacy dataset had missing information with respect to surgical type, nodal status, hormone receptor status, tumor size, histological grade and histologic type.” However, the FDA did not examine whether the gaps could have influenced trial endpoints, whereas in the case of Avastin, a worst case analysis found that missing data eliminated the reported treatment effect.

Irregularities

In 2014, the FDA modified the Herceptin label to state for the first time that the drug increases overall survival in early breast cancer. However, the benefit was found in an “efficacy evaluable” population rather than the gold standard, intention to treat population (ITT). In an April 2014 conference call, the FDA asserted that the ITT and efficacy evaluable populations were identical and that the sponsor, Roche/Genentech, requested that the label read “efficacy evaluable.” Why a pharmaceutical company would request a lower grade of evidence for the lifesaving benefits of its drug is not clear.

Also, in the joint trial, disease free survival falls while overall survival climbs. Perhaps only Provenge demonstrates a similar pattern among cancer drugs. Provenge does not enjoy the same reputation for efficacy as Herceptin.

“It is what [it] is,” N9831 statistician Vera Suman wrote in email. 

Joint N9831/B-31 trial results over time (Source: Vera Suman personal communication, 18 October 2013)

Also, in the final report on the joint study, years of median follow-up took an unusually large, 4.5-year leap in the space of approximately one calendar year. Rebecca Gelman, statistician at the Dana Farber Cancer Institute, brought this to my attention in 2013:

“As a side comment, this all leads me to wonder about the ‘8.4 years of follow-up’ in the 2012 SABC abstract, since it is so much longer than the 2011 JCO paper. Either someone did a big update of survival in 2012 (by calling all the patients or by checking the National Death Index), or else the SABC abstract was reporting OS at a time past the median survival).”

Another statistician described the leap in follow-up as “impossible,” saying that median follow-up usually goes up about one year for every calendar year. The FDA said the difference might be explained by the data lock dates for the two papers. However, the agency didn’t provide dates that would allow verifying their explanation.

9.   Can trial estimates of survival increases be squared with population-level survival figures?

Summary: Some medical researchers have suggested that therapies containing Herceptin may cure breast cancer. A Genentech-funded study estimated Herceptin saved 156,413 total life years in the United States from 1999 to 2013 for metastatic breast cancer alone. However, NCI reports only a 1.1% increase in five-year survival over a similar period. Estimates of Herceptin’s life-extending benefits should be compared to population-level figures.

HER2 prevalence at the population level is only 15%, according to NCI, well below early estimates of 25-30%.

Impact of Herceptin on five-year survival at the population level

At the 2012 SABC, presenting joint N9831/B-31 results, co-primary investigator Edith Perez advanced the idea that Herceptin cures breast cancer: “We believe that the data support the concept that many patients who present with HER2-positive breast cancer may be cured with combination strategies.” Herceptin had come a long way. Dennis Slamon, a main progenitor of Herceptin, originally believed that, by itself, Herceptin was only cytostatic, halting tumor growth which “resumed on termination of antibody therapy, indicating a cytostatic effect.”

According to a Genentech-funded study, Herceptin has saved 156,413 total life years in the United States from 1999 to 2013 for metastatic breast cancer alone. But it is unclear if population level statistics corroborate Herceptin’s curative powers. It is not known whether five-year survival has increased as much as it would need to in order to match the Genentech-funded estimate of years of life added. “We did not try to triangulate our results to the overall population,” said corresponding author of the study, Mark Danese.

In the overall population, according to NCI’s Jenny Haliski, “we are seeing a small increase in survival since 1998,” the year of Herceptin’s first FDA approval. Haliski is NCI Media Branch Chief. “Part of this increase can be attributed to improvements in treatment,” said Haliski. However, clinical trials are conducted in “ideal situations and usually include younger patients without comorbidity,” according to Haliski. “Thus, treatment efficacy in a clinical trial is usually higher than treatment effectiveness at the population level.”

It ought to be possible and instructive to decompose the 1.1% increase in five-year survival from 1999 to 2012 to determine the contribution from Herceptin. As I have suggested to Richard Gray, CTSU’s meta-study could and perhaps should try to square its estimate of Herceptin’s benefits with population-based survival figures.

Herceptin won FDA approval for metastatic breast cancer in 1998 and early breast cancer in 2006. (Chart source: National Cancer Institute, SEER Cancer Statistics Review 1975-2013, Table 4.13, all ages, all races)

HER2 prevalence

Early estimates for HER2 breast cancer prevalence of 25-30% (e.g. Slamon et al.) have not been reproduced at the population level. NCI puts prevalence at only 14.9% based on SEER reporting.

10. Can the Cleopatra and Marianne trials be reconciled?

Summary: The Cleopatra trial, which added pertuzumab to Herceptin and a taxane, produced the largest survival increases of any of clinical trial of Herceptin, nearly 16 months. But the Marianne trial seems to contradict Cleopatra. Marianne tested a version of Herceptin, T-DM1. Adding pertuzumab provided no more clinical benefit than T-DM1 alone. The phase II NeoSphere trial of pertuzumab and Herceptin also did not produce the remarkable results of Cleopatra.

Adding pertuzumab to Herceptin and a taxane in the Cleopatra trial yielded a remarkably large increase in survival, nearly 16 months longer than the standard of care, Herceptin + taxane. However, the Marianne trial seems to contradict Cleopatra: an arm testing pertuzumab with the Herceptin-based T-DM1 did no better than Herceptin + taxane. As a notice on the ASCO website said: “the addition of pertuzumab to T-DM1 provided no efficacy benefit.” T-DM1 conjugates the cytotoxic emtansine to the Herceptin antibody.

Similarly, in the neoadjuvant setting, adding pertuzumab to Herceptin showed no benefit in the NeoSphere trial which reported that “progression-free survival and disease-free survival at 5-year follow-up show large and overlapping CIs.” Pertuzumab by itself showed very little single agent activity in a phase II trial, so the benefit of combination with Herceptin is presumably synergistic. Why would it not also be synergistic with T-DM1?

Paul Ellis, who led the Marianne trial, pointed to a “number of possibilities and probably a mix of a number of issues” that explained why pertuzumab showed no benefit. In Cleopatra, said Ellis, “patients have Taxol/ Taxotere as a backup” if they do not respond to Herceptin. However, T-DM1 by itself performed just as well as Herceptin plus a taxane. No “backup” needed, and the question is why including pertuzumab added nothing in Marianne.

Ellis also observed that the “Herceptin dose per week [was] higher than T-DM1.” Yet the dose of T-DM1 was apparently high enough to perform as well as H + T. And there does not appear to be support for another trial with a different dose. Said Ellis, T-DM1 “will now never see the light of day” in early breast cancer.

Also figuring in Ellis’ possibilities were “slightly different patient populations.” However, the differences would need to be extreme rather than slight: no response at all to pertuzumab in Marianne and incredible life-extending responses among Cleopatra patients.

That leaves the idea that “maybe [T-DM1] binds differently and alters configuration in a different way” than Herceptin. However, Ellis acknowledged this directly contradicted expectation: “Every senior clinician I know in his area expected Marianne to be positive!” In addition, prior to Marianne, one research group reported “T-DM1 plus pertuzumab resulted in synergistic inhibition of cell proliferation and induction of apoptotic cell death” while another found “Trastuzumab-DM1 (T-DM1) retains all the mechanisms of action of trastuzumab.” Commented Ellis: “I think this study [Marianne] has forced them to go back into the lab and try and understand it better.” According to Ellis, “even the guy at Genentech who invented both Pertuzumab and T-DM1 can’t really understand why” pertuzumab did nothing in Marianne.

In other words, it appears that conjugating emtansine to Herceptin completely cancels synergy with pertuzumab, although both drugs were designed by the same person. Alternately, Marianne disconfirms the results of Cleopatra.

I also asked Allan Lipton about the Cleopatra-Marianne dissonance. Lipton replied: “I do not think I am the right person to answer your Cleopatra questions.” But Lipton has co-authored several papers on alternative assays for determining HER2 status and investigated HER2:HER3 dimerization and pertuzumab. I replied to Lipton: “I wonder if you aren’t the ideal person to answer such questions.” He demurred: “I don't think I have any answers for you on these observations from clinical trials.”

Although pertuzumab is frequently described as completing the blockade of HER2 and HER3, according to Mark Moasser, “pertuzumab doesn’t interfere with dimerization when HER2 is overexpressed.” HER2 overexpression has been thought of as the sine qua non of HER2-positive breast cancer. Moasser emphasized that it is “very true” that pertuzumab doesn’t block HER2 signaling when HER2 is overexpressed. Instead, “trastuzumab and pertuzumab work through immunologic mechanisms in HER2-positive cancers, and two antibodies provides double the tumor cell coverage and better immunologic targeting by the immune system.” He added: “This is not universally accepted by everyone but at this point the data is pretty clear to me and many others.”

Moasser attributes the disappointing performance of pertuzumab + T-DM1 in Marianne to the absence of a taxane: “I would say it's because taxol (or taxotere) is so effective, it’s not a shortcoming of T-DM1.” Paul Ellis advanced a similar argument. However, T-DM1 by itself performed as well as Herceptin and a taxane. In fact, progression free survival with T-DM1 alone was higher, 14.1 months vs. 13.7 months although not significantly. But adding pertuzumab to T-DM1 did nothing.

According to Moasser:

“Chemos have a 12-hour high concentration exposure and cause a lot of tumor cell kill in a short time leading to release of many cellular antigens, etc.  T-DM1 provides continuous exposure and there is incremental tumor cell killing day-by-day rather than mass killing on one day. That may be less immunogenic than the chemo method.”

However, emtansine provided enough immunologic kick for T-DM1 to equal the clinical benefits of Herceptin and a taxane. Thus Moasser’s explanation for the futility of pertuzumab seems to require that pertuzumab has different immunological prerequisites than T-DM1.

In the trial which won Herceptin initial FDA approval, adding a taxane to Herceptin delayed disease progression by 3.9 months, while in Cleopatra, further adding pertuzumab to the regimen added nearly 16 months. This quite massive effect is unexplained. Said Moasser: “I don’t claim to know all the nuances of how chemo and immunology interact with each other, and frankly nobody really does, the field is still in its infancy.” The pharmacologists, however, have somehow hit a home run with Herceptin and pertuzumab although swinging as if with eyes closed.

With EGFR inhibitors in lung cancers or BRAF inhibitors in melanomas, the mechanisms of action are clear as are the clinical results. However, said Bert Vogelstein, “we do not know how or why Herceptin works,” and the conflicting results of Cleopatra and Marianne show “that all conclusions or predictions are on thin ice,” according to Vogelstein.

Conclusion

The HER2 and Herceptin story used to be simple and compelling: we knew who it worked for and why. Now we don’t, despite nearly two decades of learning. The current balance of scientific evidence arguably no longer supports the idea of a HER2 subtype in breast cancer.

There is conflicting evidence whether HER2 is even transforming and whether it drives breast cancer. Also, the Ross et al. literature reviews supporting the prognostic role of HER2 are especially dubious. (Those papers should be corrected or retracted.) At present, the view that HER2 is prognostic is unsupported.

Although medical diagnostics have gray areas, the reproducibility of HER2 testing appears to be in a range where it perhaps should not be considered scientific. Different pre-analytic conditions, different assays, different subjective assessment criteria, tumor heterogeneity and the lack of any gold standard lead to conflicting results which are resolved arbitrarily. That the Hera trialists evade or perhaps even dissimulate regarding investigations of particular subgroups that could help validate or further discredit FISH and IHC might point to a widening disparity between appearance and reality. The main Herceptin orthodoxy has broken down completely: Herceptin does not block HER2 signaling and its mechanism of action might have little or nothing to do with HER2.

Nonetheless, Krop and Burstein contend: “Beyond a doubt, trastuzumab works.” Yet absent questionable modifications to key trials, Herceptin might not have won FDA approval. Avastin, which lost FDA approval, also works for some breast cancer patients, but there is no biomarker to predict response. The published literature demonstrates that HER2 does not predict response to Herceptin, leaving Herceptin without a valid biomarker. To paraphrase Daniel Haber: “HER2-positive” just means “responds to Herceptin.” Even HER2-negative patients can benefit, perhaps even more than HER2-positive patients. Seemingly, either all breast cancer patients should get Herceptin or none should, the latter option representing the FDA’s decision for Avastin.

At present, the standard of care is for all breast cancer patients to be tested for HER2. The tests suffer very considerable reproducibility problems. In addition, based on the re-analysis of clinical trials leading to FDA approval, HER2 doesn’t predict response to Herceptin. We don’t know who should get Herceptin but current guidelines pretend otherwise with HER2 tests that are too much like divining rods.

The clinical benefits of Herceptin might be smaller than thought. The modifications of the trials leading to FDA approval might have artificially pumped up the drug’s benefits. But in addition, at the population level, five-year survival has only increased about 1.1% since the introduction of Herceptin. Converting that modest rise into median number of months of increased survival per Herceptin patient might be instructive—perhaps corrective—of strong claims regarding the curative powers of Herceptin-containing treatment regimens.

The Cleopatra trial reported the largest increases in survival of any Herceptin trial ever. The addition of pertuzumab to Herceptin and a taxane pushed median survival up by an incredible 16 months, whereas adding the supposed workhorse of the two, Herceptin, to a taxane produced only a 4-month rise. Furthermore, in the Marianne trial, adding pertuzumab to the Herceptin-based T-DM1 did no better than T-DM1 alone, adding zero months of survival instead of 16. Worryingly, researchers who might be able to explain the seemingly contradictory results are silent. Somewhat as with conflicting HER2 assessments, researchers and physicians can just choose what to believe.

A kind of HER2 fundamentalism has taken hold as foundational truths have broken down: “clinicians should rely on established markers of HER2 expression for selecting patients,” suggested Krop and Burstein. But those very same biomarkers are what have been dis-established. Also, “established” does not mean valid, rather physicians are counseled to use the old knowledge from when the HER2/Herceptin story was compelling and coherent.

Like efforts to keep the earth at the center of the solar system, complicated epicycles have been devised to hold on to HER2 orthodoxies. A simpler explanation might better fit the contradictory evidence: while HER2 overexpression and amplification are real phenomena, there might not be a clinically meaningful HER2 breast cancer subtype.

Summary of Recommendations

• Reconduct the experiments addressing whether HER2 is transforming in mouse cell lines

• Add arms to B-47. Include HER 0 patients, receiving either Herceptin or a placebo, and a partial arm of HER2 3+ patients, all receiving Herceptin

• Decompose the 1.1% increase in five-year breast cancer survival from 1999 to 2012 to determine the contribution from Herceptin

The Herceptin meta-analysis being conducted by the Clinical Trials Services Unit at Oxford should:

• Attempt to reproduce findings of the individual studies, including the joint N9831/B-31 trial that led to FDA approval of Herceptin in early breast cancer
  

• Estimate the likelihood of assessed HER2 status being correct, if that is possible

• Allow confidence intervals around Herceptin’s clinical benefits to reflect estimated HER2 test accuracy

• Check estimates of Herceptin’s contribution to overall survival against population-based survival figures

A map from the Malaria Atlas Project, modified and superimposed on a photograph of Maarten Vanden Eynde’s “IKEA Vase”

The Malaria Atlas Project (MAP) found that human interventions this century averted fully 663 million cases of the disease. “Malaria in Africa,” according to MAP, “has halved since the turn of the millennium.”

MAP’s interactive application visually depicts human triumph over disease, malaria driven back, year after year. But is the triumph real or a special effect? More broadly, is malariology accurately representing reality or is it giving malaria a makeover?

Both the visual aspects and the science of MAP invite scrutiny and raise questions. What the maps show sometimes diverges from what the data actually say, for example. And MAP's data sometimes contradict the World Malaria Report when they ought to be nearly the same. It is doubtful that MAP accounted for age shifting while it is certain that MAP did not model the impact of an epidemic of insecticide resistance on the effectiveness of insecticide-treated bed nets. Both decisions might lead to an overestimate of human progress against malaria. Indeed, a different set of choices might show malaria is now resurgent rather than falling.

Images and science are being tweaked elsewhere in the malaria world. A paper in the Lancet on insecticide resistance presents a map that may have been improperly manipulated. In a separate study of insecticide resistance, a senior author "muted" the finding that resistance substantially reduced the protective benefit of bed nets. In addition, estimates from malaria researchers of the economic benefits of malaria have jumped implausibly from $0 in 2010 to $4 trillion today. Malariaologists are also going as far as saying that artemisinin-resistant malaria is spreading in Southeast Asia and threatens a leap to Africa when current published evidence does not support this contention. 

A Lancet ombudsman fended off criticism of one publication saying: “the paper conveys information that suffices for the message,” a philosophy that mis-informs too much malaria research.

These dissimulations may be well-intentioned, but they are not science.

Malaria Atlas Project (MAP)

Modelers make choices that shape the model. A few shards from an IKEA coffee mug became an amphora (pictured above) by the hands of artist Maarten Vanden Eynde. Similarly, the actual shape of malaria’s burden is ambiguous. Shards of malaria incidence data are so scarce that the World Health Organization (WHO) said it can't tell if cases are rising or falling in 32 of the 45 countries in the Africa region.

The MAP visualization mostly displays modeled estimates, not data. Importantly, MAP relies not on reports of malaria cases (which tend to be few and dubious) but parasite prevalence surveys. These surveys test for malaria parasites in blood samples taken from people in numerous different locales over time. MAP combines geo-located survey information with many other factors, like satellite weather data, all processed by minutely engineered statistical methods. Along with the visualization, MAP and other malaria researchers (Bhatt et al.) produced a numerical summary, published in Nature last September: “The effect of malaria control on Plasmodium falciparum in Africa between 2000 and 2015.”

But for countries like Madagascar, the map and the numbers used in the paper disagree. MAP displays malaria cases estimated from parasite prevalence surveys while, behind the scenes, the aggregate statistics for malaria cases in Bhatt et al. are based on country data.

Maps of Madagascar that should show about the same amount of malaria, based on the bar graph (top), but do not

According to the bar graph, average malaria incidence in Madagascar was roughly 69 cases per thousand people in both 2005 and 2014. The map for 2014, however, clearly shows far less malaria than in 2005. Peter Gething, corresponding author for Bhatt et al., confirmed the disparity, saying it is “entirely correct that there is a mis-match between the time series and the map.”

More than mis-matched, the map and data tell contradictory stories. Visually, malaria in Madagascar appears to be getting crushed. But the data—considered by the researchers as more reliable—say malaria has been rising since 2008, approaching the same level seen in 2000. Malaria in Madagascar looks much better than scientists believe it really is.

Gething, who leads the MAP effort, explained that for some countries “it makes far more sense to base estimates of cases on the country-reported data.” He added: “The list of countries for which the second approach was used is listed in the paper if you are interested.” But the list is not in the paper. Gething did not reply to multiple email requests for the list. However, after the editors of Nature intervened, the MAP tool was changed to list the 11 countries handled like Madagascar where the country data “may not correspond to the parasite-rate derived maps.”

The text in red is a post-publication clarification for MAP (Source: Malaria Atlas Project)

Gething said the 11 countries had smaller malaria burdens and better health systems, leading to more reliable reporting. “Many were unambiguous but inevitably some were borderline situations where arguments could be made for either approach.” But the position of the borderline might have decided the conclusions of the paper. Based on WHO-published country data, reported malaria cases appear to be rising in up to 28 African countries. If some or all of those 28 countries had been chosen, Bhatt et al. might have found a malaria resurgence. Gething would not further detail the criteria used to select the 11 countries.

In concussion research, the NFL stands accused of cherry picking data to produce a milder picture of head trauma. As one critic put it, in excluding unflattering data, “You’re not doing science here; you are putting forth some idea that you already have,” like Maarten Vanden Eynde choosing to make an amphora from fragments of a coffee mug.

Of concern, MAP used country reporting for Gambia, Mauritania and Senegal, three countries which WHO categorized as not having assessable country data. Also puzzling, Senegal has a relative abundance of parasite prevalence surveys. (See figure 2 in the Bhatt et al. supplement.) Senegal even contributed to the much more rarefied data used to transform parasite prevalence into malaria case estimates.

There are presumably good reasons to use country data for Senegal, but Gething would not say what they were. Also, it is not clear if MAP used all the available parasite prevalence surveys or, NFL-style, an unspecified subset. Again, Gething would not say. (He answered 2 of 13 emails which I sent over a three-month period.)

More concerning, what Gething described as “official country-reported data” used by MAP differs radically at times from similar data published by WHO in the 2015 World Malaria Report (WMR). Gething said “some adjustments for known under-reporting or missing data” were applied to the country data. But for Rwanda in 2014, MAP and the WMR present very different pictures of what is happening, although both are based on some form of national reporting.

WHO shows malaria surging in Rwanda (top graph, orange line) whereas MAP (bottom graph) shows malaria tailing off in recent years. (Note: The time axis for the MAP chart runs from 2000 to 2015, one more year than the WHO chart.)

A press account corroborates a sizable malaria resurgence in Rwanda: “Malaria cases in Rwanda rose at 68.6% last year [2014] to reach 1,598,076, against 947,689 cases last year; According to figures released by the Rwandan Ministry of Health.”

It’s not just Rwanda. For 2014, of the 11 nations for which MAP used country reporting, MAP figures undershoot WHO confirmed cases in five: Botswana, Namibia, South Africa, Swaziland and Rwanda.

(Swaziland, as a side note, is not mapped but shows as gray for all years, indicating either intermittent malaria transmission or none. Intentional or not, a gray Swaziland slyly promotes the strategy of “shrinking the malaria map.”)

MAP does not use country data for Burundi. MAP’s survey-based algorithms, however, produce estimates that directly contradict WHO-reported country data.

Malaria incidence in Burundi: Rising sharply according to WHO-reported country data (top, orange line) but falling steadily to its lowest point this century according to the Malaria Atlas Project (bottom).

“We are not sure why the estimates exceed the reported number of cases,” said WHO’s Richard Cibulskis who is also a co-author of Bhatt et al. Cibulskis was uncertain “whether this reflects some double counting of cases or the estimates are just off.” Double counting can be excluded, unless it also afflicts previous versions of the WMR which show much the same chart for Burundi. WHO has not corrected the 2015 edition, so the MAP estimates are “just off.” While data and estimates must be expected to differ in a modeling exercise, the degree of divergence in Burundi might raise proportional concern regarding the model’s validity.

Gaussian process models and reconstruction paste

Maarten Vanden Eynde’s amphora mostly took its shape from reconstruction paste, with just a few pieces of the original blue coffee mug. Similarly, the malaria map for Chad is almost all model. Over the 2000-2015 period, MAP had only a single 2004 study of 960 people.

Red arrow points to the single data fragment, during the 2000-2015 period, to map malaria for all of Chad. (Adapted from Bhatt et al. supplement, Figure 2.)

MAP fills in this data void with exquisite math, computing power and data borrowed from elsewhere to find a steady decline of malaria in Chad, from a peak in 2006 to a low in 2015.

But an amphora is not a coffee mug and malaria in Chad is differently shaped in the eyes of other academics. According to Foster et al., “616,722 malaria cases were reported in 2012, an increase of over 200,000 cases since 2006.” The World Malaria Report also sees malaria in Chad very differently. (Graph not shown.)

Richard Cibulskis suggested that greater use of rapid diagnostic technology possibly increased detection of cases, although “this does not necessarily reflect a true increase in malaria incidence, just an increase in diagnostic effort.”

But Cibulskis acknowledged there were true increases: “Some countries such as Uganda have experienced a resurgence in cases.” To distinguish signal from noise, researchers consider malaria hospital admissions, deaths, diagnostic practices and test positivity rates. I asked Cibulskis if, after taking those factors into account, “can an increase in cases be ruled out for any of the [28] countries which are showing increasing confirmed cases?” In other words, can the possibility that malaria is actually on the rise across most of Africa be excluded? Cibulskis did not reply.

Paying it forward: shifting malaria to older age groups

Malaria interventions frequently target very young children who lack immune protection which develops over time—and by becoming infected with malaria. Averting malaria in the very young reduces cases and overall malaria transmission, but it also prevents acquisition of immunity. As children get older, even where malaria transmission has been pushed down, some will become clinically ill with malaria because of reduced immunity. Overall, cases are greatly diminished but some are “shifted” to older age groups.

Bhatt et al. report parasite prevalence estimates for a cohort aged from 2 to 10. But it is unclear if they adjusted their estimates for the age shifting effects of malaria interventions. If not, their estimates might overstate progress against malaria by leaving the effects of age shifting off the books.

Best scientific practice seems to require accounting for age shifting. According to Briët & Penny (2013): “Many malariological studies limit themselves to examining malaria in children under ten or under five years of age...” However, “analyses for the whole population are preferred as the analyses for children under five do not capture the shifts of morbidity and mortality to older age groups...” I asked Melissa Penny, a co-author of Bhatt et al., whether that paper did as she recommended.  Penny deputized Peter Pemberton-Ross to answer my question, but he didn’t. He said the software used “certainly includes the possibility for age-shifting through its immunity submodel.” He also said that inferring incidence data from prevalence data “may implicitly assume some age-shifting.” But Pemberton-Ross would not say, yes or no, if the Bhatt et al. estimate of 663 million cases averted accounted for age shifting. Peter Gething did not reply to my inquiry about age shifting.

Bed Nets

Spatial only Gaussian Markov random field

Bed nets were “by far the largest contributor,” to averting those estimated 663 million cases, blocking 68% or 450 million malaria episodes, according to Bhatt et al. However, although the MAP interactive application shows the distribution of insecticide-treated nets (ITNs) changing in both space and time, the Bhatt et al. paper used a spatial only model for bed nets. The paper’s supplement states that a spatial only model for bed nets “was preferred over the spatio-temporal model.” Researchers made do with “national means estimated previously” for nets, published in the 2013 World Malaria Report.

Conceivably this creates a mismatch between the maps of bed nets shown by the interactive application and the data used to estimate cases averted, perhaps like the mismatch  of map and data for Madagascar. But a spatial only model for nets might mean a mismatch for all countries.

According to Pemberton-Ross, the spatial only model is just “a technical issue… This choice will have affected the results, but not necessarily by making them less accurate.”

The issue might be fundamental rather than technical, but Peter Gething did not reply when I asked if the conclusion that nets averted 450 million cases of malaria since 2000 rested on a comparatively crude, spatial only model. I also asked Gething whether the interactive mapping tool was displaying spatio-temporal bed net data when, behind the scenes in the Bhatt et al. paper, calculations for cases averted were actually based on a static, spatial only model. Gething did not reply.

I raised these issues to Nature. The editors were responsive to the Madagascar map discrepancy, and appear to have occasioned MAP’s listing of the 11 countries treated like Madagascar “in the interests of transparency,” said Rebecca Walton, Nature’s Senior Press Manager. But the other issues were met with pro forma dismissal: “The paper was rigorously peer reviewed as part of our usual editorial procedures.”

Insecticide resistance—and intransigence

Although bed nets are thought to have stopped 450 million cases of malaria, Bhatt et al. urged that maintaining their effectiveness in the face of insecticide resistance “should form a cornerstone” of future control strategies. But this grave threat to bed nets is entirely absent from the model, as if it’s solely a future concern. (Peer reviewers presumably agreed.)

However, the very distribution of hundreds of millions of nets sparked a proportionally vast rise of resistant mosquitoes, “a worsening situation that needs urgent action to maintain malaria control,” as the subtitle of a recent paper put it. Only a single class of insecticide, pyrethroids, is used to treat nets. Unsurprisingly, mosquitos have developed multiple genetic escape mechanisms, very much as they did when faced with DDT, the primary weapon used in earlier, mid-20^th century efforts to eradicate malaria.

Mosquito resistance to DDT increased gradually, ultimately rendering it ineffective and leading to the failure of eradication efforts. With pyrethroids, we seem to be watching a brutal remake of the DDT story. But researchers raise more questions than they answer about pyrethroid resistance: "Is it a problem? How do you know?" asked David Smith, a member of MAP and co-author of Bhatt et al.

In addition to doubting if insecticide resistance is a problem, Smith suggested that dispelling such doubts is nearly impossible: “The experimental unit is the population,” he contended, and “we would need to start collecting data from across the continent,” meaning Africa. A second study, also at continent scale, would be needed to measure the “attributable effect of resistance.” Even more remarkably, Smith said he “would expect the effect size of ITNs to go up overall,” if these two massive studies were somehow completed.

Smith is not alone in denial and casuistry. In Strode et al., researchers set out to investigate “the evidence that resistance is attenuating the effect of ITNs.” But instead, the scientists declared “ITNs are more effective than [untreated nets] regardless of resistance,” which is tautological. Until 100% of mosquitoes are 100% resistant to pyrethroids, an insecticide-treated net will always be more effective than an untreated one.

“Agreed with respect to the tautology,” acknowledged first author Clare Strode. “The ability of ITNs to kill insecticide resistant mosquitoes was significantly reduced when faced with resistance mosquitoes,” but, this message was “muted” in the paper according to Strode. “I originally included a much stronger statement of fact that ITNs kill fewer resistant mosquitoes than susceptible counterparts,” she continued, “but the statistician and Cochrane expert recommended a less bold statement.”

The Cochrane expert, Paul Garner, did not dispute Strode’s account or explain why he recommended a less bold statement. (Also at Garner’s suggestion, the Strode et al. review excluded 914 studies without explaining why.) In 2004, Garner was involved in the Cochrane Review of bed nets that provided much of the basis for the massive scale-up that intervention. Muting the conclusions of Strode et al. might serve to protect the conclusions of the earlier review and the subsequent, massive, bed net intervention that seems to have gone awry.

Many malariaologists demand proof that pyrethroid resistance reduces the impact of nets, a stance akin to the tobacco lobby’s denial that cigarettes cause cancer. According to Clare Strode, “I cannot see how increasing resistance would NOT impact ITN efficacy.” It’s undeniable: “There is no biological basis to argue otherwise,” said Strode.

Nick Hamon agrees: “Yes, resistance compromises efficacy. That is no longer in question.” Hamon runs IVCC, a consortium tasked with developing new insecticides.

Nonetheless recent peer-reviewed papers still ignore resistance. A paper in the Lancet estimating how much malaria might be reduced by further expanding interventions “assumed no loss of effect due to drug or insecticide resistance.” In fact, the authors almost doubled the killing effect of nets in their model, adapted from Menach et al. (2007). Senior and corresponding author, Azra Ghani, did not reply to emails asking how to reconcile today’s stronger insecticide resistance with an assumption of greater killing power than nine years ago.

Insecticide resistance has been modeled. Brady et al. found that resistance cuts the effectiveness of nets by half or as much as two thirds, depending on how swiftly resistance develops. (See Figure 3D.)

In 2013, Penny & Briët determined that introducing nets in high transmission areas with insecticide resistance only reduced transmission by 75% instead of 90%. Penny went on to co-author Bhatt et al., but that paper ignored insecticide resistance even though one third of the population of Africa lived in high transmission settings in 2000.

I am not aware of any papers estimating increased malaria cases and/or deaths resulting from insecticide resistance. IVCC’s Nick Hamon made recourse to “two respected, independent malaria scientists” for a crude estimate of the number of deaths that would be averted if there were a new insecticide to replace failing pyrethroids. According to Hamon: “One scientist gave me a range of 141,000 – 228,000 and another 125,000.” Adding even part of 125,000 to WHO’s estimated 395,000 malaria deaths in 2015 would make for a very sizeable increase in mortality.

Hamon cautioned that “these are ‘back of envelope calculations’ and should be treated as such,” adding that “nobody wanted to be quoted, and for good reasons.” He would not say what the good reasons were. Nonetheless, among themselves, malaria experts countenance disturbingly large increases in malaria deaths resulting from insecticide resistance.

Another mostly insider conversation is the effect of resistance on the infectivity of mosquitos. Against hope and expectation, early indications are that the genetics of resistance also increase mosquito susceptibility to infection by malaria parasites. (See Ndiaith et al. and Alout et al.) Conceivably, not only has the scale-up of bed nets sparked a massive wave of resistant mosquitos, those mosquitos are also more likely to become infected with malaria and thus are potentially more likely to transmit the disease.

Averting the appearance of a malaria disaster: Possible image manipulation in Hemingway et al.

Another paper this year in the Lancet, “Averting a malaria disaster,” drew attention to mounting insecticide resistance and the need to develop new chemicals to replace those that are failing.

However, the map (Figure 2B) accompanying the paper might have understated the extent of the resistance problem. According to the caption, Figure 2B was “reproduced” from an online tool called IR Mapper. However, Figure 2B includes a number of green dots, indicating no resistance, that are not found on IR Mapper, in Sudan, for example. Some yellow dots in IR Mapper, showing possible insecticide resistance in Angola, appear as green dots in Figure 2B, indicating no resistance.

I found at least seven such discrepancies between Figure 2B and IR Mapper. In addition, Figure 2B does not display any yellow dots. IR Mapper has been displaying red, green and yellow dots since its inception in 2012, according to Duncan Kobia Athinya of Vestergaard Frandsen, which oversees IR Mapper. Said Athinya: “I cannot speak as to why the Lancet figure does not feature possible resistance (yellow) points, but IR Mapper has followed WHO criteria since its launch in 2012.”

Also unexplained is the green dot in Sudan in Figure 2B. Said Vestergaard’s Melinda Hadi: “I still do not have an answer regarding the green susceptible point in Sudan.” Figure 2B was created in October of 2014 but not published until April of 2016. Studies, and thus dots, have been added subsequently. Also, some might have been removed. But regarding the green dot in Sudan, Hadi said:  “I can confirm a publication was not removed from IR Mapper.”

Perhaps explaining these and other discrepancies, according to Hadi: “the maps in the Lancet article were reproduced. The IR Mapper database was provided to the Liverpool School.” Of possible relevance, IR Mapper includes a “View own data” facility that allows users to create a map from a database.

Hadi explained that yellow dots were left out: “Data points that were classified as possible resistance (90-97% mortality) were not presented in Figure 2.” In addition, Figure 2B “only included data from peer-reviewed publications, so you will note other data points available on the platform (e.g., PMI data) were excluded.” PMI, the President’s Malaria Initiative, collected data in 18 countries.

Asked about issues regarding Figure 2B, first author Janet Hemingway said: “the figure is a screen shot downloaded back in 2014…” Hemingway is Director of the Liverpool School of Tropical Medicine. The discrepancies resulted from the passing of time, according to Hemingway: “Lancet have sat on the paper for almost a year since submission and acceptance so I guess it is possible over this period that IR mapper has been updated for historical data, but we made no alterations to the download.”

Hemingway declined to answer any more questions:  “I have no intention of responding further on this, as there is no further explanation…”

However, the passing of time did not explain the discrepancies such as the missing yellow dots and the presence of a green dot in Sudan and half a dozen other anomalies that surfaced in a non-exhaustive analysis.

I raised these issues with Figure 2B to the attention of Lancet editor-in-chief, Richard Horton. Horton did not reply.

Prompted by the intervention of the Committee on Publication Ethics (COPE), Lancet editor Zoë Mullan relayed an explanation from Hemingway (who had earlier declined to answer questions). Hemingway said: “The single green dot in Sudan he refers to, I suspect is a point that was subsequently corrected if GPS co-ordinates had been incorrectly allocated for example.” Hemingway did not say where the dot could now be found.

Regarding the absence of yellow dots, Mullan explained, perhaps implausibly: “IR Mapper was clearly not showing any yellow dots on the day the authors downloaded the screenshot that became their figure.” Besides the unfortunate timing, Mullan’s explanation would also seem to require that the authors, who are experts on insecticide resistance, didn’t notice the absence of yellow dots that indicate possible resistance, nor did peer reviewers.

In addition to being an editor at the Lancet, Mullan is a trustee of COPE, perhaps creating a conflict of interest in responding to COPE-initiated inquiries. (Mullan’s role at COPE was not disclosed to me; I happened upon it later by chance.)

I also asked Richard Horton directly: “Is Figure 2B a screen capture or reproduced from the IR Mapper database? Why does Figure 2B not show any yellow dots?”

However, Horton only replied that the editors “feel confident that the data reported” in the paper “are accurate and reliable.” He vouched, to some degree, for the data but not for the accuracy of Figure 2B. He did not address the absence of yellow dots.

COPE declined to press the Lancet further. Wrote COPE’s Iratxe Puebla: “Given that the issues relate to a specific figure, we do not feel this falls within COPE’s remit to evaluate...” Furthermore, “COPE considers it beyond its remit to comment on… how facts are presented in individual publications.”

Richard Horton recently criticized COPE for not intervening sufficiently in a controversy regarding statins, bemoaning “the lack of a central institution where scientists who wish to question the actions or ethics of other scientists or scientific institutions can go.”

COPE suggested that I contact “the authors' institution so that they can review and consider what follow up may be appropriate.” Via transatlantic mail, I contacted LSTM board secretary R.E. Holland inquiring about a possible institutional investigation. Holland replied, also by letter: “Having reviewed the issue thoroughly, and having spoken to several experts in respect of resistance incident imaging, I have concluded there is no case for the authors to answer in respect of your complaint.”

Holland’s review assumed that Figure 2B was a screen shot. His letter said: “it is impossible to compare a snapshot of image data from one period to that of another and therefor there is no case to answer.” Holland’s answer was essentially the same as LSTM’s Director, Janet Hemingway. Holland did not explain the absence of yellow dots or the green dot in Sudan. He touted LSTM’s “rigorous research misconduct policy” which had been used in this case.

“[T]he paper conveys information that suffices for the message” 

Figure 2B and the statements by the authors and editors of the Lancet also passed muster with Lancet ombudsman, Malcolm Molyneux. However, Molyneux acknowledged the possibility that Figure 2B was not a screen capture. 

The word ‘reproduced,’ according to Molyneux, “can mean either of the possibilities - a screen-shot or a figure re-drawn from data.” He added: “I really do not think it matters.” In his view, if the authors changed the figure to suit their purposes—including adding dots—they were within their rights: “the paper conveys information that suffices for the message - removing or adding yellow dots or (a few) other dots would make no difference at all to that message.”

Molyneux's statement, “the paper conveys information that suffices for the message,” nicely captures the philosophy that is mis-informing too many papers in malaria.

Placing a green dot in Sudan or anywhere appears to be legitimate in the eyes of the Lancet ombudsman, as long as the number of dots added does not exceed "a few.” Leaving dots out is no infraction, according to Molyneux because “the legend to Fig 2 says ‘reproduced from...’ - it does not say ‘with no subtractions’.” 

He distinguished “falsification of data” from “simplification for purposes of clarity.” However, adding green dots that do not actually represent studies of insecticide resistance means those dots are fake, while changing the color of dots misrepresents the findings of actual studies. It is hard to see how that would not be falsification of data.

Continued Molyneux, “if the authors had been trying to manipulate the figure in order to make their case more compelling, we would expect them to err towards the red in the later time-period (2b)… In every single case you mention of a difference between the IRMapper and Fig 2b, the difference is from red to green, not the other way round.” 

Image manipulation requires no explanation. However, as I wrote to Molyneux, “the title of the paper is ‘Averting a malaria disaster.’ Unless the map shows that there is a disaster to avert then the title doesn't fit… A sea of red and yellow dots might lead readers to conclude that mosquitoes had already won.”

As it stands, figures “reproduced” in the Lancet may differ in unspecified, undisclosed ways from the source in order to convey the authors' message, which might differ from their scientific findings.

Tale of two resistances

If the malaria research community is downplaying insecticide resistance, it is exaggerating the threat of drug resistance in Southeast Asia spreading to Africa. Resistance to artemisinin is not spreading even in Southeast Asia and faces scientifically demonstrated obstacles to overtaking Africa. It’s not happening, but researchers are saying it is.

Arjen Dondorp heads malaria research at the Mahidol-Oxford Tropical Medicine Research Unit in Bangkok. He laid out an accurate chronology of the discovery of resistance to artemisinin-based malaria treatments. Resistance was first found in Western Cambodia, then at the Thai-Myanmar border, in Myanmar, Northern Cambodia, Northeastern Thailand, Eastern Cambodia, Southwestern Vietnam, and Southern Laos.

“Out of the ‘epicentre’ of Western Cambodia,” said Dondorp, “over time the resistant parasite has spread westward, northward, and eastward.” He concluded: “This is spread.”

However, Dondorp’s statement, if not simply false, is not scientifically supported. He described the spread of surveillance, a trick that could equally demonstrate that broken arms or bad breath are “spreading” in Southeast Asia just by conducting surveys in the same places and order he described for artemisinin resistance.

Genetic sequencing has, against expectation, found that nearly every artemisinin resistance hot spot emerged independently, not as a result of spread. Future research might change the current understanding of the epidemiology of artemisinin resistance, but the most comprehensive survey found only three instances of spread out of 112 samples from across the region.

More word play and dissembling are on view in a Lancet paper on resistance in Myanmar that included the word “spread” in its title but adduced little evidence and no claims for it. I wrote two of the authors, saying “the title of your paper, ‘Spread of artemisinin-resistant Plasmodium falciparum in Myanmar’ seems belied by the evidence actually in the paper (and other papers).” I asked them if they would correct “any misperceptions on my part,” but neither Mallika Imwong nor Charles Woodrow replied.

François Nosten, who runs a clinic in Mae Sot near the Thai-Myanmar border, also claims resistance is spreading. “Resistance to artemisinin,” according to Nosten, “has emerged in different places in SEA [Southeast Asia] but then it has spread.” I asked: “Can you describe unpublished data or point to papers where spread is documented?” Nosten, regarded by many as a public health hero, replied not with science but anecdote and sophistry: “We find that over 80% of our patients with malaria have parasites that are resistant to artemisinin. It did not emerge in each and every one independently, did it?” Nosten is correct it did not emerge in each patient independently but that is not at all the same as spread. To establish spread requires DNA sequencing from at least two places; Nosten claims spread based on a single cohort and no sequencing data.

A malaria press tour to Southeast Asia, funded by Malaria No More, featured journalist visits and interviews with Nosten and Dondorp. Stories in Slate and other outlets told readers artemisinin resistance was spreading and threatened a malaria apocalypse in Africa.

Distorted science is creating distorted journalism. An AFP story suggested that the reason artemisinin resistance hadn’t spread to Africa was that “international efforts to contain the spread of resistant parasites have been effective.” It is more the case that biologically it is difficult or impossible to install the multiple genetic changes required to create artemisinin resistance. However, the international containment efforts, by increasing drug pressure, might be forcing malaria down evolutionary pathways which could result in a more compact genetic form of resistance that could be more easily exported to Africa.

Meanwhile, the actual spread of insecticide resistance in Africa is ignored. Another journalist field trip funded by Malaria No More featured Tanzania as the destination. Insecticide resistance might have been among the briefing topics, but it did not appear once in an article for Vice written by one of the journalists on the trip.

No one has died from drug-resistant malaria. “As far as I know,” said Nosten, “there has been no confirmed fatal case.” Meanwhile, according to Nick Hamon’s sources, some part of 125,000 people (or more) have died from malaria because insecticide resistance has reduced the effectiveness of bed nets.

Debasing the currency: $4 trillion drawn on the account of science

Worth less than the paper it’s printed on (Source: Wikimedia)

In 2010, researchers concluded that malaria eradication was unlikely to break even and advised that “financial savings should not be a primary rationale for elimination.” But a few years later, an overlapping constellation of researchers discovered that eradication would quickly generate $4.1 trillion in economic benefits, in just 15 years.

The paper touting a $4.1 trillion windfall “is an advocacy document rather than an academic analysis,” according to Rima Shretta at the University of California, San Francisco (UCSF). Shretta is part of the UCSF group which led development of the 2010 Lancet series which found no cost savings from eradication. Shretta also served on the Action and Investment for Malaria task force that developed the advocacy document projecting $4.1 trillion in benefits.

To reconcile the academic analysis of the 2010 Lancet paper with the later discovery of trillions of dollars in benefits, Shretta seems to suggest scientists are free from the standards of science if they are engaged in advocacy. And functionally, it appears malaria advocacy has detached from science, although much of the advocacy comes from scientists.

The End: Malaria Goes Hollywood

Images, which can surreptitiously mislead, end up exposing a conscious mis-shaping of malaria research. Authors are making undisclosed and perhaps improper choices regarding the visual elements in their papers. However, within the papers, the same authors are free to make any number of choices that can decisively influence findings and there is little or no possibility of suggesting impropriety. The sources of data, how they are processed, type of model and parameters partly or entirely decide the research results. Authors can “mute” statements about the loss in bed net effectiveness caused by pyrethroid resistance. Editors can entitle a paper “spread of resistance” when there is none mentioned in the paper and the evidence contradicts the spread hypothesis. But when the philosophy of managing reader beliefs extends to choices about visual elements, the curtain is drawn aside and we see not a scientist but the Wizard of Oz.

A spot of bother in Maiduguri district, Nigeria (Source: Wikimedia)

Worldwide, in all but three of 155 countries, the trivalent oral polio vaccine has been replaced with bivalent oral vaccine. The bivalent formulation includes only attenuated versions of type 1 and 3 of poliovirus. The type 2 component has been dropped because, far more than the other types, it sometimes mutates back into virulent form. Also, type 2 polio was eradicated in 1999.

But just as the world moved to the bivalent vaccine, Nigeria reported finding a type 2 vaccine-derived virus in a sewage sample. Consequently, right on the heels of the vaccine switch, the type 2 vaccine is being immediately pressed back into service, although it will be used by itself, in monovalent form, according to the Global Polio Eradication Initiative.

Sequencing indicates the Nigerian virus has been circulating undetected since May of 2014. The sample comes from Maidaguri district, an area contested by government forces and Boko Horam, making vaccination problematic. 

Last September, WHO removed Nigeria from the list of polio-endemic countries. However, the CDC continued to advise that US travelers to Nigeria be immunized against polio.

Initially, the polio eradication project envisioned stamping out all type 2 vaccine-derived virus transmission before dropping the type 2 vaccine component. But plans to switch vaccines ultimately went ahead despite the likelihood of continued circulation of type 2 vaccine-derived virus somewhere in the world.

There are now multiple hotspots. Besides Nigeria, according to the CDC's Steve Wassilak, "We consider [the] Guinea and Myanmar outbreaks still active." In addition, Brazil reported what researchers described as a "highly evolved" type 2 vaccine-derived virus found in sea water off São Paulo. Found in January 2014, sequencing indicates the virus has been circulating undetected for eight years. Brazil has very high population immunity to polio, so this virus likely came from somewhere else, according to Wassilak. 

Eight years of undetected circulation suggests a perhaps large and as yet undiscovered surveillance gap somewhere in the world. Asked whether eight years set the record for undetected circulation, Wassilak answered: "Nigeria had documented circulation for 10 years." However, in Nigeria, there were multiple transmission chains, and it is not clear from Wassilak's answer if any one chain circulated eight years. The Brazilian isolate also had mutations at antigenic sites, suggesting possible evolution of resistance. However, researchers reported that type 2 antibodies still killed the virus.

The process of switching to the bivalent formulation also risks creating new type 2 vaccine derived virus. The switch was synchronized globally because if use of the trivalent vaccine continues anywhere, it might potentially infect children who have only been immunized with the bivalent vaccine. According to WHO:

"The primary risk associated with the cessation of use of type 2 OPV [oral polio vaccine] is the re-introduction of disease-causing type 2 poliovirus into a population with increasing susceptibility to type 2 poliovirus. The switch from tOPV to bOPV must therefore be globally synchronized to minimize the risk of new cVDPV type 2 emergence."

The precision of the large and un-rehearsable switch remains to be seen. Globally, susceptibility to type 2 vaccine derived virus is now rising given the switch to bivalent vaccine and the slow (and arguably belated) introduction of the injected vaccine, which includes all three virus types in a form in which mutation is not possible. Also, while the injected vaccine protects against paralysis caused by poliovirus, it does not prevent infection nor halt transmission. Polio circulated in Israel without causing any cases of paralysis because coverage with the injected vaccine was so high. Eventually, however, circulation might find someone missed by vaccination or with a compromised immune system, resulting in polio's hallmark acute flaccid paralysis.

The success in beating back wild poliovirus bodes well for the eradication effort to also smash outbreaks from vaccine-derived virus. But, out of the gate in the post-trivalent world, the race is already on. And, in Nigeria at least, type 2 vaccine-derived virus circulation has gone uninterrupted for a decade.

From Slate, we know time is running out to eliminate drug-resistant malaria. The Gates Foundation believes this too. But is the foundation’s logic irresistible or did Slate run an infomercial for the foundation funded by a $40,000 grant? The story (including a trip to Thailand) was paid for by Malaria No More which has received $20 million in Gates Foundation grants.

It’s not just Slate or only global health. Carefully restricted Gates Foundation grants to NPR, the PBS NewsHour, the Pulitzer Center on Crisis Reporting and other news organizations shape what gets covered, what doesn’t, when and how.

Trusted media organization receiving Gates Foundation grants are not following good journalistic practices. And like improper food labeling, undisclosed funding misleads news consumers about what they are actually getting.

"I do not see any conflict of interest." - Seth Berkley

Gavi's new board chair, Ngozi Okonjo-Iweala, simultaneously joined the sovereignty practice at Lazard, an investment bank which Bloomberg has described as "banker to the broke." A number of Gavi-supported countries are Lazard clients. Also, Gavi-eligible countries might consider retaining Lazard to enhance their prospects for Gavi funding. 

[See previous article, "Gavi Board Chair-elect Joins Lazard's Sovereignty Practice the Same Day."]

However, Gavi CEO, Seth Berkley, said: "I do not see any conflict of interest." Continued Berkeley: "Many of our Board members have other jobs and board positions and we are very careful to monitor any potential conflict of interest issues." As a sign of the legitimacy of the arrangement, Berkley said "the announcement of her work with Lazard and Gavi were coordinated by the communication team and announced the same day." Previously, Gavi spokesperson Rob Kelly denied such coordination of the announcement. Kelly also said Gavi had not facilitated Okonjo-Iweala's employment arrangement with Lazard.

As board chair of Gavi, Okonjo-Iweala will oversee and have signing authority on Gavi grants. Gavi recently raised $7.5 billion dollars to fund vaccine grants over the next four years. 

Although Gavi board members do indeed have other jobs, I asked Berkley;

Is it not the case that, as Gavi board chair, Ngozi Okonjo-Iweala will have a hand in distributing several billion dollars to finance ministries while, on  the other hand, as part of Lazard, Okonjo-Iweala will be receiving money from finance ministries?

Berkley did not reply.

The Gavi board, which elected Okonjo-Iweala unanimously, appears to have been unaware of her arrangement with Lazard. According to Berkley, "The Board delegated responsibility for due diligence to the Board-appointed Recruitment Committee." I asked Gavi board member Zulfiqar Buttha if the board knew of the Lazard affiliiation when the board elected Okonjo-Iweala. He wrote back: "No."

None of the Gavi board members I emailed responded regarding whether joint Gavi-Lazard positions represented a conflict of interest. I emailed:

1. Flavia Bustreo, WHO (Assistant Director-General)
2. Zulfiqar Buttha, Unaffilliated
3. Tim Evans, World Bank (Director)
4. Geeta Rao Gupta, Unicef (Deputy Executive Director)
5. Orin Levine, Gates Foundation (Director)
6. Katie Taylor, United States (Deputy , USAID)

Okonjo-Iweala's election appears to violate Gavi statutes and bylaws

In addition to conflict of interest problems, the election of Okonjo-Iweala appears to violate Gavi statutes and bylaws. Article 12 says: 

"Board members will select the Chair and a Vice Chair of the Board from among their own voting members..."

The Gavi bylaws appear to reinforce the statutes. Section 2.6 says:

"The Chair and Vice Chair will be selected according to Article 12 of the Statutes from among voting Board Members (not Alternate Board Members)."

Okonjo-Iweala was not a Gavi board member. Perhaps to circumvent Gavi statutes, Okonjo-Iweala was both named to the board and elected board chair at the September 2015 board meeting. By contrast, outgoing board chair, Dagfinn Høybråten, previously served on Gavi's board before becoming chair. 

I asked Seth Berkley if Okonjo-Iweala's election violated Gavi statutes. He did not reply.

Alleged disappearance of Nigerian oil money; Gavi investigation of Nigeria

Okonjo-Iweala is widely known as an anti-corruption crusader. However, the former finance minister of Nigeria has been caught up in allegations that Nigerian oil revenues were improperly diverted. Investigations continue. The newly-elected president of Nigeria, Muhammadu Buhari, recently said, "We have some documents where Nigerian crude oil was lifted illegally and the proceeds were put into some personal accounts instead of the federal government accounts." One estimate put the amount of fraud at $20 billion dollars. A former oil minister is being investigated and others might be named.

In theory, Gavi is still investigating misuse of Gavi funds distributed to Nigeria while Okonjo-Iweala was Finance Minister. A 2014 audit recommended that the Economic and Financial Crimes Commission carry out "a thorough and detailed investigation of the Gavi grants disbursed to Nigeria." In addition, the audit sought "a full-scale audit to cover both select, high-risk expenditures in prior years, and other expenditure from the period 2011-2013" not examined in the course of the 2014 audit.  

It is difficult to see how Gavi could pursue or cooperate fully with any investigation of wrongdoing that might involve its board chair. I emailed Simon Lamb, Gavi's Managing Director, Audit and Investigations, and asked if Gavi was following through on the 2014 audit recommendations. He did not reply.