Misleading: The negative values observed months after the last vaccine dose don’t necessarily mean that the vaccines increased the risk of infection. Negative values can also result from very low vaccine effectiveness coupled with other factors such as the type of variants circulating and control measures.
RECLAMACION: A study by Kaiser Permanente “shows negative efficacy of the shots against all variants within 150 days. And this study shows the more you inject, the more you infect; specifically, over time, those with three doses fare worse than those with two.”
REVIEW
Data from clinical trials and safety surveillance indicate that COVID-19 vaccines approved in the U.S. are safe and effective at preventing severe illness and death. Nevertheless, false and misleading claims downplaying the vaccines’ effectiveness or greatly exaggerating their risks continue to proliferate online. One example of this is an article by Daniel Horowitz published in the Conservative Review on 4 October 2022, claiming that COVID-19 vaccines “are extremely unsafe and ineffective”.
This isn’t the first time Horowitz questioned the effectiveness of COVID-19 prevention and control measures, including COVID-19 vaccines. Health Feedback evaluated previous articles by Horowitz claiming that “masks don’t work” and that COVID-19 vaccines lead to new variants and increase the risk of pregnancy complications. Our reviews found these claims to be inaccurate and unsupported.
Specifically, the article in the Conservative Review cited a preprint—a manuscript that hasn’t undergone peer review yet—by U.S. healthcare company Kaiser Permanente at the request of Moderna evaluating the effectiveness of the Moderna COVID-19 vaccine[1]. Horowitz claimed that the results showed “negative efficacy of the shots against all variants within 150 days”, meaning that the vaccine increased rather than decreased the risk of infection over time.
The claim was picked up by other outlets, such as Children’s Health Defense, LifeSite News, and Natural News, which Media Bias Fact Check rates as biased and of low credibility for promoting unsupported claims, propaganda, and conspiracy theories. Other outlets, such as The Epoch Times and Zero Hedge, published articles that were more balanced in their reporting but nevertheless repeated the claim in their headlines.
As we will explain below, claims that the preprint found the Moderna COVID-19 vaccine to be ineffective misrepresent the authors’ conclusions and are misleading. In fact, the preprint shows that the vaccine remains highly effective against hospitalization, as stated in the Discussion:
“In conclusion, our data indicate that the 3-dose or 4-dose effectiveness of mRNA-1273 against infection with Omicron subvariants is moderate and short-lived, but protection against severe COVID-19 disease remains robust.” [emphasis added]
The Moderna COVID-19 vaccine remains effective against hospitalization, even though protection against infection wanes over time
The preprint by Tseng et al. evaluated the effectiveness of three and four doses of the Moderna COVID-19 vaccine against infection and hospitalization from January to June 2022. For that, the authors used a specific type of case-control study called test-negative design that has been routinely used for estimating the effectiveness of influenza vaccines[2].
This type of design differs from conventional case-control studies in the way that participants are selected. In case-control studies, the researchers first identify people who have the condition they want to study (cases) and then search for a group of individuals with similar characteristics who don’t have the condition (controls). In contrast, all the participants in negative-test designs are selected based on a predefined set of symptoms. The researchers subsequently separate them into cases and controls based on the result of a diagnostic laboratory test for the condition of interest.
In the case of the preprint, the participants had accessed the U.S. Kaiser Permanente Southern California healthcare system with symptoms compatible with COVID-19. Then, the researchers classified as cases those with a positive SARS-CoV-2 PCR test (92,427) and as controls those with a negative result (30,809). This design makes the case and control groups more comparable than in traditional case-control studies because both come from the same community and have similar healthcare-seeking behavior, which reduces the risk of bias[3].
The researchers also analyzed the virus sequences in each COVID-19 case to determine the subvariant responsible for the infection. They included subvariants that circulated during the study period, BA.1, BA.2, BA.2.12.1, BA.4, and BA.5.
Finally, the authors estimated vaccine effectiveness by comparing the proportion of vaccinated and unvaccinated people in the case and control groups. Then they analyzed the results by subvariant, vaccine dose, and time since vaccination
The authors found that three or four vaccine doses were 72.4 to 97.5% effective against hospitalization for all Omicron subvariants. In contrast, vaccine effectiveness against infection depended on the subvariant and waned rapidly. Three doses of the vaccine provided moderate to high protection for all subvariants during the first month (61 to 90.6%).
While vaccines remained 54.9% effective against infection with BA.1., five months after receiving the third dose, protection against newer variants waned rapidly (-16.4 to -26.8% after five months). A fourth dose was 64.3 to 75.7% effective against infection with all subvariants except BA.5. (30.8%), but the protection completely waned after three months (5 to 14%).
Therefore, articles claiming that the Moderna COVID-19 vaccine is ineffective based on these results are misleading. Such articles focused on vaccine effectiveness against infection and misrepresented the results of the preprint. At the same time, they ignored the data showing that the vaccine was effective against severe illness, which is one of the primary goals of vaccination.
Vaccine boosters decrease—not increase—the risk of infection, hospitalization, and death
The Conservative Review article claimed that the preprint showed a “negative efficacy of the shots against all variants within 150 days”. This means that the vaccine allegedly made people more prone to infection. In addition, the article claimed that each vaccine dose increased the risk.
Similar claims have circulated online since early 2022, citing different studies as supporting evidence. Health Feedback reviewed one of these claims in an earlier review and found it to be inaccurate.
Figures 2 and 3 of the preprint showed vaccine effectiveness estimates for three and four doses of the Moderna vaccine over time. Some of these values were indeed negative, particularly at five months after vaccination. However, the fact that some estimates appeared negative doesn’t necessarily mean that the vaccine increased the risk of infection. Indeed, the authors suggested that one explanation for the negative values was confounding factors, combined with very low vaccine effectiveness, which as we explained above, occurs with certain subvariants after some time following the last vaccination.
These confounders can affect calculations of vaccine effectiveness, making them appear negative when they are close to zero. Some of the possible confounders cited by the authors were changes in individual behavior, circulating variants, and control measures:
“For example, some negative VE estimates observed at >150 days after vaccination could be due to differential risk behaviors among vaccinated and unvaccinated individuals when protection from antibodies becomes minimal. Second, as predominant subvariants evolved during the study period, many other factors could also change over time, such as practice of non-pharmacologic interventions, availability of antiviral medications or monoclonal antibody treatments, preventive public health policy, and individual behaviors. These changes might impact the comparison of VE across subvariants.”
Current vaccine formulations remain protective against hospitalization. But given the rapid waning of protection against infection with newer variants, boosters can help reduce the risk. Hung Tseng, an epidemiologist and first author of the study, explained in an email to Lead Stories:
“Your risk of contracting COVID does not increase because you have received the vaccine. However, regular boosters are necessary to continue the protection against infection and hospitalization provided by COVID-19 vaccinations. Also, we may be seeing that people who have been vaccinated and boosted participating in more risky public health behaviors such as gathering in crowded indoor areas and putting themselves at risk of infection after the vaccine protection has waned.”
Indeed researchers have worked to develop Omicron-specific vaccine formulations to control the spread of the virus in the long term. In September 2022, the U.S. Food and Drug Administration authorized bivalent formulations of the Moderna and Pfizer-BioNTech mRNA COVID-19 vaccines as a booster dose. Bivalent formulations are vaccines with two components, one that protects against the original variant and another one specific for the Omicron subvariants BA.4 and BA.5. that are currently circulating in the U.S.
The U.S. Centers for Disease Control and Prevention recommend that people aged five and older receive these boosters if it has been at least two months since their last vaccine dose.
In support of its claim that COVID-19 vaccines increase the risk of infection, the Conservative Review article also cited a Twitter thread by the American Association of Physicians and Surgeons, an organization that has previously promoted misinformation about COVID-19 vaccines and treatments. The Twitter thread listed multiple studies that supposedly showed “a similar trend of negative efficacy resulting from the shots”. However, this interpretation is again misleading.
While the cited studies did show negative values at some points in time, all the studies attributed them to the effect of confounding factors similar to those explained in the preprint, including changes in risk behavior[4-8]. More importantly, these studies consistently showed high and durable vaccine effectiveness against severe illness, even though the protection against infection waned after a few months
Conclusion
Claims that the preprint by Tseng et al. is evidence that the Moderna COVID-19 vaccine increases the risk of infection are misleading and misrepresent the authors’ conclusions. Contrary to the claims, the preprint showed that the vaccine remained highly effective against hospitalization. While protection against infection waned over time, particularly with some Omicron subvariants, booster doses improved this protection. Variant-specific vaccine formulations may be necessary in the long term, but the scientific evidence so far indicates that COVID-19 vaccines continue to be effective against severe illness and death.
CORRECTION (20 December 2022):
This review was updated to correct an error in two values of vaccine effectiveness against infection with newer Omicron subvariants five months after receiving a third dose of the Moderna COVID-19 vaccine. The review originally stated that these values were 16.4 to 26.8% (positive values) when it should have said -16.4 to -26.8% (negative values). This correction doesn’t affect the verdict of the review.
UPDATE (26 October 2022):
This review’s Details section was updated to explain why the claim is misleading.
REFERENCES
- 1 – Tseng et al. (2022) Effectiveness of mRNA-1273 against infection and COVID-19 hospitalization with SARSCoV-2 Omicron subvariants: BA.1, BA.2, BA.2.12.1, BA.4, and BA.5. medRxiv. [Note: This is a preprint that has yet to be peer-reviewed at the time of this review’s publication.]
- 2 – De Serres et al. (2013) The test-negative design: validity, accuracy and precision of vaccine efficacy estimates compared to the gold standard of randomised placebo-controlled clinical trials. Eurosurveillance.
- 3 – Fukushima and Hirota (Vaccine) Basic principles of test-negative design in evaluating influenza vaccine effectiveness. Vaccine.
- 4 – Chemaitelly et al. (2022) Duration of mRNA vaccine protection against SARS-CoV-2 Omicron BA.1 and BA.2 subvariants in Qatar. Nature Communications.
- 5 – Rudan et al. (2022) BNT162b2 COVID-19 vaccination uptake, safety, effectiveness and waning in children and young people aged 12–17 years in Scotland. Lancet Regional Health – Europe.
- 6 – Lin et al. (2022) Effects of Vaccination and Previous Infection on Omicron Infections in Children. New England Journal of Medicine.
- 7 – Fleming-Dutra et al. (2022) Association of Prior BNT162b2 COVID-19 Vaccination With Symptomatic SARS-CoV-2 Infection in Children and Adolescents During Omicron Predominance. JAMA.
- 8 – Chiew et al. (2022) Effectiveness of primary series and booster vaccination against SARS-CoV-2 infection and hospitalisation among adolescents aged 12–17 years in Singapore: a national cohort study. Lancet Infectious Diseases.
