Why COVID-19 Vaccine Efficacies Can’t Be Compared and Shopped

Why COVID-19 Vaccine Efficacies Can’t Be Compared and Shopped

With at least three different COVID-19 vaccines available in North America now, efficacy comparisons are becoming inevitable and could lead to vaccine shopping. This makes it important for vaccine messaging to be clear in highlighting the most important metrics within the vaccine data.

As the US Food and Drug Administration (FDA) handed out an emergency use authorization (EUA) to Janssen/Johnson & Johnson’s COVID-19 vaccine over the weekend — making it the third authorized COVID-19 vaccine in the US — many people became laser-focused on the 66 percent average efficacy of the single-dose shot. The knee-jerk reaction among some has been to deem the vaccine inferior to other authorized counterparts that have higher efficacies.

In particular, comparisons are being drawn between Johnson & Johnson’s offering and vaccines from Pfizer/BioNTech and Moderna, which both reported smashing efficacies of almost 95 percent when they were authorized in December of last year.

In order for a vaccine to be eligible for authorization or approval by the FDA, it needs to demonstrate an efficacy of at least 50 percent. Therefore, the Johnson & Johnson and the yet-to-be-US-authorized AstraZeneca/Oxford vaccines (the latter having an average efficacy of about 70 percent) meet the FDA’s vaccine efficacy threshold. To put this into perspective, the annual influenza vaccines are about 60 percent effective in their best years.

While the differences between the efficacies of the COVID-19 vaccines appear significant, there is more to it than meets the eye.

The 66 percent efficacy determined for the Johnson & Johnson vaccine is for the prevention of mild to moderate disease, an average derived from data from the company’s global Phase III trials. In the trials, the vaccine was shown to have an efficacy of 85 percent in preventing severe COVID-19 disease, an important metric in the efficacy parameter that makes it more comparable to the near 100 percent efficacy of Pfizer/BioNTech’s and Moderna’s shots in preventing severe disease.

However, overall, there are several reasons why vaccine efficacies cannot be directly compared, some of which include the design, criteria, endpoints, participant demographics and timing of the clinical trials that evaluated them.

And although things are improving with respect to vaccine hesitancy, there is fear that confidence in certain vaccines over others may lead to vaccine-specific hesitancy and vaccine shopping. Right now, 41 percent of adults in the US say they will get a COVID-19 vaccine as soon as it is made available to them for free, which is up from 34 percent in December, according to recent poll results from KFF.

Currently, in North America, there is little choice with respect to which vaccine an individual is offered, but that could potentially change as vaccine supplies from different makers increase in the coming months.

In light of this, vaccine makers and regulatory bodies will need to share vaccine data and science in a clear and rigorous manner in order to clearly communicate why there is no one best vaccine, and why each authorized vaccine is a superlative tool in battling COVID-19, particularly against its potential to be severe and deadly. This means bringing a focus to key metrics within the efficacy data of the vaccines.

Trial Timings and Variants

The Phase III clinical trials for the vaccines from Pfizer/BioNTech and Moderna collected much of their data during the summer last year when COVID-19 transmission had subsided a bit after the first wave. This compared to trials for the vaccines from Johnson & Johnson and AstraZeneca/Oxford, which went into late fall and the winter months (January/February of this year). Not only did this time period coincide with the second wave of the pandemic, during which record-high case numbers were being recorded in many places, but it also was in the background of emerging SARS-CoV-2 variants. Several variants of the virus emerged in late fall of last year, including the more transmissible B.1.1.7, and the particularly concerning B.1.351 variant which may be able to evade immune surveillance.

Of Johnson & Johnson’s nearly 44,000 trial participants in their global trials, 40 percent of them were in the US, another 40 percent were in Latin America and 15 percent were in South Africa — the latter being the site of the most rampant circulation of B.1.351 currently, and its place of origin.

Related: Novavax and Johnson & Johnson Seek Regulatory Approval for COVID-19 Vaccines After Posting Promising Efficacy Data

In the South African trial of Johnson & Johnson’s study, almost all COVID-19 cases (approximately 95 percent) were attributable to the B.1.351 variant by January. In addition to the N501Y mutation present in B.1.1.7, B.1.351 also has a concerning E484K mutation in the receptor binding domain of the spike protein of SARS-CoV-2, which is thought to enhance its binding to host cells. While in vitro lab studies have shown that antibodies generated from Moderna’s vaccine had a six-fold reduced neutralizing capacity against B.1.351, the action of the vaccine against the variant in humans is currently unknown and under investigation.

The presence of some of the concerning variants during trials for the Johnson & Johnson and AstraZeneca/Oxford vaccines is likely to have had an impact on efficacy results. Therefore, the period during which trials for the different vaccines were conducted has to be considered, as any given time period may be reflective of case cohorts attributable to a particular dominant variant(s) that may be less susceptible to the vaccines.

Nevertheless, it has been shown that most of the vaccines are efficacious against the new variants. A clearer picture of variant-specific efficacies will come as more data becomes available, including real-world data.

As proactive measures, many vaccine makers are in the process of developing variant boosters, particularly against B.1.351. They are also testing multiple boosters of the existing vaccines against variants. In addition, mixing and matching of different vaccines to help ramp up immunity against variants is yet another strategy that is being explored.

Criteria, Demographics and Endpoints

Classification criteria for disease severity was also subject to differences across trials for the different vaccines. COVID-19 is generally categorized as mild, moderate or severe. For example, as per FDA guidance, severe disease is classified as a positive PCR test for SARS-CoV-2 along with two symptoms from a list that includes high breathing rate, high heart rate and low oxygen saturation among others.

Moreover, the primary clinical endpoints in the vaccine trials have been occurrence of COVID-19 infection, with some trials having made distinctions between mild, moderate or severe disease while others did not. For example, while Johnson & Johnson’s co-primary endpoints were efficacy of a single dose of vaccine to prevent moderate to severe COVID-19 at two different time points (14 and 28 days post-vaccination), Pfizer/BioNTech’s primary endpoint was the incidence of COVID-19 without specification of disease severity.

A clinical endpoint is met when a statistically significant result can be derived between the study groups. Despite some trials not having taken disease severity into account in their endpoints, COVID-19 cases were still of course classified by severity. This is how efficacies of the vaccines against moderate versus severe disease, as well as hospitalizations and deaths, were determined, despite the latter not having been primary efficacy endpoints in any of the trials.

It may seem counterintuitive that if a vaccine can’t prevent mild or moderate illness very well, then how can it be effective against severe disease?

The immune system is complex and has different arms and tiers. The adaptive immune system has two main arms: humoral and cell-mediated. The humoral response consists of antibodies while the cell-mediated response, which is generally stronger and longer-lasting, involves the action of T cells and memory B cells.

As a first line of defense, it could be that mild infection is predominantly handled by antibodies and does not trigger recruitment of T cells and B cells. However, once an infection starts getting severe, reserves of those immune cells may be called up for battle. The sequence of events in immune responses is not always entirely clear or understood well. Responses may differ based on the nature of the pathogen, and the nature and level of immunity against it.

Additionally, trials for the different COVID-19 vaccines included different groups of participants, often with different co-morbidities and age demographics. As with most clinical trials, these trials were generally conducted among young, healthy individuals. And while Moderna and Pfizer/BioNTech did evaluate their vaccines in people over the age of 65, there has been limited data in this population in trials for AstraZeneca/Oxford’s vaccine. As such, the UK, and now Canada with its recent approval of the vaccine, are holding off on recommending the vaccine for those over 65.

Owing to some of these factors and differences between the trials for the different vaccines, direct comparisons between vaccine efficacies cannot be readily made.

Risk of Hospitalization and Death Priority Metrics

Undoubtedly the most important point that needs to be emphasized about vaccine efficacies and protection against COVID-19 is that all of the authorized vaccines have been shown to prevent deaths by 100 percent across all Phase III clinical trials for the different vaccines, as well as currently ongoing, real-world, Phase IV post-marketing surveillance trials. Hospitalizations are also reduced by nearly 100 percent.

This is huge because these are the two most important metrics that have been at the cornerstones of the pandemic — thwarting hospitalizations to not overwhelm healthcare systems, and curbing mortality. To achieve near perfect scores on these is nothing short of spectacular.

The fact that getting any of the three vaccines will prevent people from getting sick enough to necessitate hospitalization and potentially spiral towards death is a major win that needs to be at the center of vaccine conversations.

COVID-19 Vaccine Messaging

Officials from Johnson & Johnson, as well as other companies like Pfizer, as well as government officials, have been doing the rounds on media circuits to help explain the numbers and data in more detail.

“Our data actually includes these most challenging, pernicious, virulent strains and what we saw was an 85 percent effectiveness rate in the severe disease,” Gorsky told CNBC News on Monday. He explained that different trial timings, particularly before and after the emergence of new variants, and the global nature of Johnson & Johnson’s trials led to the different outcomes.

Anthony Fauci, NIAID chief and the White House’s chief medical adviser, delivered similar messages about reductions in hospitalizations and deaths in his appearances on Meet the Press and Face the Nation. He also said that he “would definitely take the Johnson & Johnson vaccine,” in a social media video released by The White House COVID-19 Response Team.

“One of the most important things is not overgeneralizing the message and saying ‘Oh, you should just get it whatever it is,” said Rob Jekielek, managing director at The Harris Poll. “Because there are people who are hesitant because they don’t think they have enough access to information or they don’t understand it. So it’s really important to be specific. … [M]ost important, across the board, is that the vaccines directly address severe COVID-19.”

Vaccine hesitancy can only be combatted by vaccine confidence, and that confidence will come from better communication of the data and its nuances. Converting hesitancy to acceptance will be a concerted effort that will need to involve engagement and communication between vaccine makers, healthcare practitioners, government officials and the public.

For now, the clearest message should be to get the vaccine that is made available to you, simply because they all almost completely cut your risk of being hospitalized due to COVID-19, and outright eliminate the risk of death. And these are absolutely major wins.