The Epidemiology of COVID-19 Vaccine-Induced Myocarditis.

Antibody response and safety of COVID-19 vaccine in peritoneal dialysis patients

COVID-19 vaccines in pregnancy.

Observations on improving COVID-19 vaccination responses in kidney transplant recipients: heterologous vaccination and immunosuppression modulation

Immunization Update 2021

COVID-19 Vaccination Is Safe and Effective in Patients With Inflammatory Bowel Disease: Analysis of a Large Multi-institutional Research Network in the United States

Safety of mRNA COVID-19 vaccines during pregnancy

Severe de novo liver injury after Moderna vaccination – not always autoimmune hepatitis

Coronavirus Disease 2019 and Vaccination of Children and Adolescents: Prospects and Challenges

De Novo and Relapsing Glomerular Diseases After COVID-19 Vaccination: What Do We Know So Far?

Should women undergoing in vitro fertilization treatment or who are in the first trimester of pregnancy be vaccinated immediately against COVID-19

Immunogenicity of BNT162b2 SARS-CoV-2 Vaccine in a Multicenter Cohort of Nursing Home Residents Receiving Maintenance Hemodialysis

Editorial: Coronavirus disease 2019 (COVID-19) - advances in epidemiology, diagnostics, treatments, host-directed therapies, pathogenesis, vaccines, and ongoing challenges.

Targeting the Gut Microbiota in Coronavirus Disease 2019: Hype or Hope?

By May 15, 2020, 65 days after the World Health Organization (WHO) declared the novel coronavirus disease 2019 (COVID-19) pandemic, 4.2 million individuals were confirmed as being infected with the causative agent, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), and 294,000 people had died from the disease1. These numbers are almost certainly an underestimation of the true morbidity and mortality of the disease to this point. From the beginning of this public health crisis, attention has focused on the development of vaccines against SARS-CoV-2. Many believe that immunization is the key advance in the war against COVID-19 and that control of SARS-CoV-2 allowing approximations of pre-pandemic social conditions will not be possible without a viable vaccine2. Until a viable vaccine is developed, extensive diagnostic testing, quarantining, and social distancing are the only control methods that we have3. This article was written for musculoskeletal physicians and scientists without immunology backgrounds. It seeks to provide a concise but comprehensive understanding of vaccine development with a special emphasis on efforts to establish immunization directed against SARS-CoV-2. This will help to inform readers of the challenges to be hurdled and allow them to track milestones reached as the global community labors toward an effective vaccine. In-depth reviews of complex immunology topics, viral epidemiology, the myriad historic aspects of vaccine development, and the economics of developing and utilizing different types of vaccines are beyond the scope of this work. Background SARS-CoV-2 To understand the process of vaccine development, one must start with a basic understanding of the pathogen against which it will be directed. SARS-CoV-2 is a beta coronavirus4. The virus has been found to be similar to another coronavirus whose usual hosts are bats. It is believed that wild animals being sold at a market in Wuhan, Hubei Province, the People's Republic of China, led to the transmission of SARS-CoV-2 to humans5. Members of the Coronaviridae family have a positive-sense, single-stranded RNA ([+]ssRNA) genome6. The (+)ssRNA can serve as messenger RNA (mRNA) in the host cell, allowing utilization of host ribosomal machinery to translate viral proteins. This coronavirus genome is highly conserved6. The first gene (open reading frame [ORF]1a and ORF1b) is involved in replication and transcription. Subsequent genes are related to structural proteins (Fig. 1). The spike protein (S) has an exposed location on the virion and is necessary for entry into the cell. Blocking of the spike protein epitopes that interact with angiotensin-converting enzyme 2 (ACE2), the host cellular receptor for SARS-CoV-2, should lead to neutralization (Fig. 2). ACE2 degrades angiotensin II (AngII), downregulating the renin-angiotensin-aldosterone system. ACE2 is distinct from the angiotensin type-I receptor, which leads to activating signal transduction by AngII7.Fig. 1: Illustration of the SARS-CoV-2 genome showing expressed structural component proteins and their location within the virion. The open reading frame (ORF) codes for nonstructural viral proteins participating in viral genome replication, transcription, and protein processing. The spike protein (S) binds to the ACE2 receptor, allowing virus entry into host cells. The envelope protein (E) has been found to bind to host gene regulatory proteins and putatively influences host cell gene expression. The membrane protein (M) seems to cooperate with S during binding and entry into the cell. The nucleocapsid protein (N) coils the viral mRNA inside the viral particle, organizing and protecting it.Fig. 2: SARS-CoV-2 virion entry into the host cell. The virus enters the host cell via the S protein interacting with ACE2. It is important to generate antibodies to the S protein rather than ACE2. This allows ACE2 to remain open to receive the host ligand and to avoid the possibility that the blocking antibody interferes with its normal physiologic function.COVID-19 is usually characterized by fever, cough, dyspnea, fatigue, and sore throat. Radiographic findings have demonstrated pneumonia with infiltrates on chest imaging8. Typically, radiographic changes appear on presentation. In Wuhan, 76% of diagnosed patients demonstrated changes on chest radiographs9. It is unknown whether radiographic changes arise prior to the onset of symptoms. Currently, there are no known curative treatments for COVID-19. Treatment is centered around supportive care while the afflicted individual clears the infection. In some cases, supplemental oxygen needs to be administered to help those with the associated pneumonia to maintain their oxygen saturation. Severe cases may benefit from antiviral medications, dexamethasone10, and ventilatory support with or without extracorporeal membrane oxygenation11,12. In March 2020, the WHO estimated the global mortality rate for those confirmed to have a SARS-CoV-2 infection to be 4.3%, although this varies by region13. Vaccines Vaccines meet the definition of a drug because they are substances other than food used in the prevention, diagnosis, alleviation, treatment, or cure of a disease14. They are administered to healthy individuals to prevent diseases caused by infectious agents to which they might be exposed in the future or, in some cases, to which they have been recently exposed. Immunization is the process of presenting antigens to a live host to induce an immune response. Vaccines can be utilized therapeutically and prophylactically. Most vaccines are developed as prophylaxis against infection. A primary goal of prophylaxis is to generate neutralizing antibodies that prevent viral entry into cells. However, in some cases, vaccines can be administered for short-term protection. This short-term protection is termed passive immunization. A smaller proportion of vaccines are developed for the therapeutic treatment of infections that have already become initiated. An example of this is the vaccine used after exposure to rabies15. Epidemiology of Infectious Disease and Vaccination During infectious disease outbreaks, vaccines have the ability to break chains of transmission, drastically slowing the propagation of infection and eventually eradicating the disease. The vast majority of individuals in modernized societies are immunized against a host of pathogens as children. This global public health effort has resulted in the worldwide elimination of smallpox16 and substantial progress toward the elimination of polio17. These accomplishments required concerted international efforts that were sustained over many decades. The epidemiologic dynamics of vaccination, especially in a population with a disease presence (either endemic or epidemic), can be described mathematically. The basic reproduction number (R0) is the intrinsic measure of transmissibility of a pathogen. It describes the mean number of secondary infections resulting from a single infection within a fully susceptible population (Fig. 3). For SARS-CoV-2, R0 has been estimated to be between 2 and 318.Fig. 3: Illustration of the basic reproduction number (R0) values for representative infectious diseases, demonstrating increasing infectivity with increasing values of R0. HIV = human immunodeficiency virus.Once substantial immunity, through either vaccination or natural exposure to the disease, starts to be established in the population, R0 becomes a less accurate measure of transmissibility, and a second number, the effective reproduction number (Re), becomes more representative of transmissibility. Re represents the actual number of transmissions per infection. For an explanation of how these numbers are calculated and their effect on viral epidemiology (Fig. 4), please see Appendix I.Fig. 4: The relationship between the basic reproduction number (R0) and the effective reproduction number (Re) of a pathogen. Re changes based on the percentage of the population that is immune, demonstrated by the different lines on the chart. For a disease with an R0 of 3.5, immunization of 70% of the population will be necessary to bring Re below 1.0. This can be compared with an immunization level of only 50% needed to bring Re below 1.0 for a disease with an R0 of 2.0. If Re, a measure of disease transmissibility, is <1, a disease outbreak will end without further measures being taken.Herd Immunity Herd immunity is a condition achieved in a population when the indirect protection of susceptible individuals against a specific pathogen is conferred by immune individuals. In effect, those with immunity within a population shield those without immunity from transmission, thereby limiting the spread of the disease19 (Fig. 5). A further explanation of how R0 and Re affect herd immunity (Figs. 6 and 7) is found in Appendix II.Fig. 5: Illustration demonstrating herd immunity. The spread of a hypothetical illness within a population is shown under 3 different conditions. The top row is a time early after the introduction of the pathogen within a population. The second row represents 1 replication cycle and the third row represents 2 replication cycles following the time represented in the top row. In the left column, the population is fully susceptible (no naturally acquired immunity or immunity through vaccination) to the pathogen. The middle column represents an intermediate state between full susceptibility and achievement of the herd immunity threshold (Pcrit). The column on the right represents how spread is hindered in a population that has already achieved Pcrit.Fig. 6: The relationship between the basic reproduction number of a virus, R0, and the proportion of the population that needs to be immunized to enter into a state in which herd immunity takes effect, Pcrit. R0 is unitless and Pcrit is measured as the percentage of the population. The slope of the curve is greatest between R0 values of 1 and 4, indicating that a large increase in the proportion of immune individuals within the population is needed to achieve Pcrit for an infectious disease with an R0 that is just slightly larger than another. The estimated R0 for SARS-CoV-2 is between 2 and 3, which is demarcated in the area between the dotted lines on the graph. This R0 range indicates that the percentage of the population that needs to become immune to reach Pcrit, either through infection or vaccination, is between 50% and 66.7%.Fig. 7: Vaccination affects the achievement of herd immunity. This graph shows vaccination affecting herd immunity to diphtheria. Diphtheria-tetanus-pertussis (DTP3) immunization coverage is expressed as a percentage of the population from 1982 to 2016. The red line on this graph represents the number of diphtheria cases reported worldwide by year. The blue line represents the immunization coverage as a percentage of the population by year. The dashed orange line represents the herd immunity threshold (Pcrit) for diphtheria assuming that it has an R0 of 6. The graph demonstrates how achieving immunization levels just slightly above Pcrit can drive the number of cases down to just a few thousand per year.SARS-CoV-2 Vaccines Under Development It is through recombinant nucleic acid technology that most efforts against COVID-19 are progressing. Theoretically, vaccines using recombinant nucleic acid technology can be developed and, therefore, deployed more quickly than those using more traditional means. However, a few are based on immunologic techniques used widely in the mid-twentieth century for the development of vaccines used in standard childhood immunization schedules. Table I summarizes the types of vaccines as well as their advantages and disadvantages. By June 2, 2020, >130 candidate vaccines targeting SARS-CoV-2 were under development worldwide according to the WHO20. Ten of these candidate vaccines have entered into human trials. Table II summarizes the SARS-CoV-2 vaccines currently in development. TABLE I - Advantages and Disadvantages of Different Vaccine Types* Type of Vaccine Example Vaccine(s) Advantages Disadvantages Potential Solutions to Development Problems Attenuated Sabin polio (host-range mutant) Smallpox (Jennerian method) Reliable Confer both humoral and cell-mediated immunity Limited need for booster doses Relatively inexpensive technology May yield less protective mutant viruses if passages are performed in non-human cell lines63 Can cause disease in immunocompromised individuals Enhanced storage and maintenance requirements (e.g., refrigeration, culture media) Inactivated Typhoid Salk polio Rabies Influenza Relatively fast Easy to scale up Relatively inexpensive development Alternate routes of administration (e.g., oral) Inactivation can sometimes damage key epitopes Can still theoretically cause disease (if not fully inactivated) Labor-intensive process to ensure no viable virion remains after inactivation Greater amounts of inoculum required to achieve immunity Immune response is less durable, requiring booster doses64 Titration of inactivation methods to prevent overtreatment, maintaining key epitopes Subunit Hepatitis B surface antigen Safe Easy to use Requires large amounts of isolated antigen Immunopotentiation2,65 Recombinant technology has made large-scale production possible Adjuvants can be added to vaccine preparations66 mRNA29 HIV† Zika† Influenza† Rabies† Can replicate using host machinery Highly potent Relatively fast Easy to scale up • Relative low cost of manufacturing Safe Confer both humoral and cell-mediated immunity dsRNA byproducts of production are PAMPs‡ and lead to immune recognition and degradation29 Inefficient delivery to target cells Instability of mRNA Modified nucleosides avoid host immune recognition Purification to eliminate dsRNA Multiple delivery methods devised to improve efficiency Inclusion of upstream and downstream untranslated sequences improves stability of mRNA DNA plasmid Prostate cancer† Melanoma† Can replicate using host machinery Highly potent Relatively fast Easy to scale up • Safe Confer both humoral and cell-mediated immunity Inefficient delivery to target cells Coupled treatments (e.g., electroporation) to improve DNA plasmid entry into human cells34 and enhance immune response32–33- Vector-based recombinant (e.g., adenovirus vector) Malaria† HIV† Ebola (in development) Confer both humoral and cell-mediated immunity Enter host cells via ubiquitously expressed cell surface receptors Can incorporate additional transgenes for biological adjuvants37,67 Highly potent Relatively fast Easy to scale up in low-resource settings68 Safe Easier, alternate routes of administration favoring IgA production (e.g., oral, intranasal)69,70 Vector immunity due to host recognition of adenovirus71 If replication-competent viral vectors are utilized, could cause adenovirus infection Administration via mucosa may require dose escalation Repeated immunizations can diminish neutralizing antibodies to adenovirus vectors, thereby decreasing vector immunity71 Utilize different adenovirus strains to evade vector immunity70,72,73 Utilize chimpanzee adenovirus vectors to evade vector immunity due to highly conserved genome74 Use of replication-competent viral vectors can boost immunogenicity *HIV = human immunodeficiency virus, dsRNA = double-stranded RNA, PAMP = pathogen-associated molecular pattern, and IgA = immunoglobulin A.†In trials.‡These are present and invariant in the pathogen but not in the potential host organism; PAMPs allow early recognition and immune activation by the host organism75. TABLE II - SARS-CoV-2 Candidate Vaccines Currently in Clinical Development* Vaccine Type Clinical Phase of Development Trial No. (Location) Chimpanzee adenovirus vector-based (nonreplicating) Phase 2b/3 2020-001228-32 (EU) mRNA (lipid nanoparticle encapsulated) Phase 2: scheduled to conclude September 2021 NCT04405076 (USA) Adenovirus-5 vector-based (nonreplicating) Phase 2: scheduled to conclude January 2021 ChiCTR2000031781 (China) Subunit (recombinant spike protein) Phase 1 and 2: scheduled to conclude July 2021 NCT04368988 (USA) mRNA (lipid nanoparticle encapsulated) Phase 1 and 2: scheduled to conclude June 2021 NCT04368728 (USA), 2020-001038-36 (EU) Inactivated with aluminum adjuvant Phase 1 and 2: scheduled to conclude July/August 2020 NCT04383574 (USA), NCT04352608 (USA) Inactivated Phase 1 and 2: scheduled to conclude November 2021 ChiCTR2000031809 (China) Inactivated Phase 1 and 2: scheduled to conclude November 2021 ChiCTR2000032459 (China) DNA plasmid (with electroporation) Phase 1: scheduled to conclude July 2021 NCT04336410 (USA) Inactivated Phase 1 China *As of June 2, 2020. Attenuated Vaccines Vaccines that use a weakened but viable form of the pathogen to establish an immune response are termed attenuated vaccines. The establishment of attenuated strains is an empiric rather than a designed process, the duration of which is influenced by the scale of the process and the time necessary to complete the passage of the virus in the cell culture. In rare cases, the attenuated variant can revert to a virulent form, resulting in the establishment of the disease in the immunized cohort. There are 4 different types of attenuated vaccines. At least 3 different groups are seeking to develop attenuated vaccines against COVID-1920. The most common form of an attenuated vaccine is the host range mutant. This type is usually achieved by the passage of the virus in the cell culture to generate nonvirulent mutants. Over multiple passages, the host range mutant acquires mutations, rendering it harmless while retaining the necessary antigens to generate immunity. The second type is naturally attenuated. This method requires empiric identification and recovery of variant or minimally variant strains during periods of outbreak21. The third type of attenuated vaccine is the temperature-sensitive mutant. This method isolates mutants that at of strains that can be at is the found at the respiratory but become nonvirulent at the or The type of attenuated termed is not being in the COVID-19 vaccine development Inactivated Vaccine vaccine requires large amounts of viral to be in and by either or (Fig. The of inactivation is to the virus nonvirulent without the epitopes to which an immune response is and can virus A needs to be achieved in the of vaccines. will lead to and of key to a neutralizing antibody response that is and protective with a live However, if complete inactivation not of the disease can from the vaccination through some vaccine types that are being developed against SARS-CoV-2. Inactivated viral are in and through or means. The of viral allows the of an immune response without of infection. Subunit viral proteins are and These proteins are into patients to be The proteins are by the host cells and the antigens are that an antibody response is mRNA mRNA the viral S protein is in the and is using technology as These mRNA are and are into viral proteins using host and the antigens are that an antibody response is DNA plasmid DNA for the viral S protein is into a DNA This plasmid enters host and machinery viral proteins. These viral proteins generate an antibody response. vector-based the gene for the viral S protein is into an adenovirus genome in These adenovirus vectors with entry into the host cell and the host machinery to viral proteins that generate an antibody Vaccines Subunit vaccines generate immunity by administration of a virus antigen (Fig. These antigens are by the immune to generate an antibody response. Typically, antigens are that will generate an immune response. For SARS-CoV-2, one of the candidate antigens for vaccine development is the spike A number of the of and are on developing spike protein mRNA Vaccines In this mRNA for the SARS-CoV-2 spike protein is into the host cells to it (Fig. proteins made by host cells will be as by the immune which will generate both an antibody and the response to mRNA vaccines have had that effective but have of these mRNA vaccine delivery and the of the and methods are standard at this 1 developing an mRNA vaccine against SARS-CoV-2 has that it could be as an measure to as early as DNA Vaccine the of the of SARS-CoV-2, the development of a DNA vaccine the (+)ssRNA genome of SARS-CoV-2, DNA sequences can be These sequences can be into or double-stranded DNA that for antigen DNA (Fig. can replicate using host is into host it can viral antigens that are as the host immune to generate an antibody and the response. This method of vaccine development advantages similar to those for mRNA vaccines nucleic are that the potential with protein which can recombinant multiple have demonstrated that can enter host cells and can in a immune This technology has been shown to be and in to there have been no DNA vaccines for use in Recombinant Vaccines Recombinant viral vectors have been as vaccine over the few for their ability to antigens and their immunogenicity (Fig. are DNA of recombinant adenovirus vector-based vaccines is that they can be made and highly effective through recombinant The of recombinant adenovirus vector-based vaccines is by rendering the virion replication This is achieved by the antigen in the of the adenovirus genome for viral replication, thereby its ability to most vaccines are administered either or adenovirus vector-based vaccines provide the possibility of This is a more administration than For these many different recombinant adenovirus vector-based vaccines targeting SARS-CoV-2 are currently under development. Immunization immunization may be achieved by administration of to a or recently infected protection against pathogens within the immunized The protective of the are eventually down or this the protective effect is because not or the immune to provide immunization. There are with and patients develop the at a of infection. Currently, there are a number of for the treatment of patients with Immunity to COVID-19 There and should a to develop a vaccine against SARS-CoV-2. health measures will need to the development, and administration of a vaccine against SARS-CoV-2 to a large proportion of the population. with these morbidity and mortality will The potential rate increase from to between and of the population while the the herd immunity threshold (Pcrit) may be to A number of potential immunity, immunity, and vaccine could the of a COVID-19 vaccine TABLE - and the of COVID-19 Vaccine Immunization on COVID-19 Vaccine Immunization immunity are of Immunity in individuals may require vaccination individuals may not be fully immune Vaccine is only effective May require vaccination or vaccination of more individuals than Pcrit Vaccine or use of vaccines to achieve Pcrit for development for vaccine development and is 4 the development can only if necessary the first Vaccine takes to develop similar efforts have of vaccine development, we still not have a vaccine Vaccine takes to develop molecular techniques more effective of recombinant Development Vaccines or manufacturing complete of vaccine development to from and Over worldwide on vaccine Development Vaccine to by the is especially A by the for that of not and were if they when a vaccine becomes A of found that were at and an additional were in a COVID-19 is an that could Pcrit or lead to the of COVID-19 The COVID-19 vaccine will not be the This was with polio and both of which through in the and types that were used when or were in the early The vaccines which groups to they are administered and toward and their delivery methods will with limiting vaccine is immunity. illness are of the This that SARS-CoV-2 may exposed individuals. may be due to either antibody and cell or The need for vaccination could the establishment of Pcrit. possibility is that only some individuals have full immunity. immunity is still it the numbers of infected individuals and those requiring care on the level of an care The of developed vaccines will most be by of blocking Vaccines that have entered into have already been for blocking antibodies in their The administration of different vaccines has been in achieving immunity. attenuated and polio vaccines administered have a for viruses with substantial as immunizations help to neutralizing antibody for conserved There is a complex of and on immune In antibody to vaccines are in while both humoral immune to novel that are related to multiple affect the time for immunity to a is 2 to 3 following the Problems that are of in developing a vaccine to RNA viruses are the of and is the of the viral disease rather than its This has led to and in immunized has been with a number of those against is the of infection. of infection when viruses are by cells via their or This has been shown to in human cells with It is not that vaccine administration will have a musculoskeletal However, the time to testing, and will have for the and of care It is standard methods or will be during the production and of viable as many of the currently being used have resulted in doses at that multiple vaccines and their will be to more and effective and administration to the Immunization with candidate vaccines should not lead to should be to allow the establishment of an immune response and of severe morbidity and mortality associated with The for the development of effective vaccines should be with the that the to achieve will be There will be many and we must to this The protective measures that we have social protective and the use of testing, will be a of COVID-19 has morbidity and Pcrit will be in to a of social Pcrit will be through immunization. an early and response toward developing a COVID-19 remain in Recombinant techniques may this A between the community and should the need for vaccine development with Appendix by the is with the of this article as a at

In late 2020, messenger RNA (mRNA) covid-19 vaccines gained emergency authorisation on the back of clinical trials reporting vaccine efficacy of around 95%,1, 2 kicking off mass vaccination campaigns around the world. Within 6 months, observational studies reporting vaccine effectiveness in the "real world" at above 90%, similar to trial results,3-6 became the trusted source of evidence upholding these campaigns. While the contemporary conversation about vaccine effectiveness has turned to waning protection, virus variants, and boosters, there has (with rare exception7) been surprisingly little discussion of the limitations of the methodologies of these early observational studies. The lack of critical discussion is notable, for even highly effective vaccinations could only partially explain the drop in rates of covid-19 cases, hospitalisations, and deaths by mid-2021. For example, by March 2021, cases in the UK and United States had dropped roughly fourfold from the January peak, when the "fully vaccinated" population only reached 20% and 5%, respectively. At the same time, in Israel, cases took longer to drop despite a substantially faster vaccine rollout (Figure 1). The vaccination campaigns in these countries can thus only be part of the story. We are aware of only one article that addresses methodological concerns in non-randomised studies of covid-19 vaccines.7 The author draws attention to potential biases and measurement issues, such as vaccination status misclassification, exposure differences, testing differences, attribution issues, and disease risk factor confounding. Many of these concerns are hard to confirm within specific studies due to data unavailability (e.g., testing differences) or cannot be fixed analytically (e.g., exposure and other unmeasured quantities). In this article, we focus on three major sources of bias for which there is sufficient data to verify their existence, and show how they could substantially affect vaccine effectiveness estimates using observational study designs—particularly retrospective studies of large population samples using administrative data wherein researchers link vaccinations and cases to demographics and medical history. Using the information on how cases were counted in observational studies, and published datasets on the dynamics and demographic breakdown of vaccine administration and background infections, we illustrate how three factors generate residual biases in observational studies large enough to render a hypothetical inefficacious vaccine (i.e., of 0% efficacy) as 50%–70% effective. To be clear, our findings should not be taken to imply that mRNA covid-19 vaccines have zero efficacy. Rather, we use the 0% case so as to avoid the need to make any arbitrary judgements of true vaccine efficacy across various levels of granularity (different subgroups, different time periods, etc.), which is unavoidable when analysing any non-zero level of efficacy. It is also important to note that under hypothetical conditions different from the actual events of early 2021, two of these sources of bias could bias results in the opposite direction, that is, underestimating actual vaccine effectiveness. Finally, to draw more precise conclusions about the impact of these biases on specific published studies, we urge that all code and data available to those studies be made public. In each of our three illustrations, we compare results based on observational study methods against randomised controlled trial (RCT) methods. For each comparison, one side represents a published study while the other is a counterfactual. In each case, we show how the gap between observational and RCT study results is due to a source of bias. The pivotal covid-19 vaccine trials used a primary endpoint of lab-confirmed, symptomatic covid-19.8-11 Not all covid cases, however, factored into the estimate of vaccine efficacy. Investigators did not begin counting cases until participants were at least 14 days (7 days for Pfizer) past completion of the dosing regimen, a timepoint public health officials subsequently termed "fully vaccinated."12 The rationale for excluding cases occurring before the start of this "case-counting window" was not provided in trial protocols–and legitimacy of excluding post-randomisation events has long been debated13—however, one Pfizer post-marketing document states that in the early period post-vaccination, "the vaccine has not had sufficient time to stimulate the immune system."14 In randomised trials, applying the "fully vaccinated" case counting window to both vaccine and placebo arms is easy. But in cohort studies, the case-counting window is only applied to the vaccinated group. Because unvaccinated people do not take placebo shots, counting 14 days after the second shot is simply inoperable. This asymmetry, in which the case-counting window nullifies cases in the vaccinated group but not in the unvaccinated group, biases estimates. As a result, a completely ineffective vaccine can appear substantially effective—48% effective in the example shown in Table 1. (The placebo data in Table 1 comes from the Pfizer Phase III randomised trial, and is the assumed case counts for the unvaccinated group in a counterfactual observational study occurring simultaneously; this setup illustrates the potential size of a case-counting window bias in a real-world setting as well as why this bias does not exist in a randomised trial.). We are aware of just one observational study3 that addressed case-counting window bias, by using matching and designating a pseudo-study enrolment date for the unvaccinated party in each matched pair of vaccinated and unvaccinated persons. While matching mitigates case-counting window bias, this method injects an artificial and severe age bias between unvaccinated and vaccinated groups: the matched subset underrepresented patients ≥ 70 years by 50% while over-representing patients ≤ 40 years by 50%. (This occurred because the propensity to receive the vaccine is highly influenced by age. Therefore, the number of one-to-one matched pairs of elderly patients is upper bounded by the number of unvaccinated elderly while the number of one-to-one matched pairs of younger patients is upper bounded by the number of vaccinated young.). In retrospective studies using large population samples, we propose a simple adjustment that can correct for case-counting window bias. The case rate from vaccination to the start of the case-counting window can be observed from the vaccinated group and applied to the unvaccinated group to estimate the number of cases to be excluded before computing the relative ratio of cases. This adjustment preserves the case-counting window, while assuming the vaccine is completely ineffective before its start. Because we use the 0% efficacy assumption, this simple adjustment returns the vaccine effectiveness estimate back to zero. A similar strategy has proved useful in influenza treatment analyses.16 Age is perhaps the most influential risk factor in medicine, affecting nearly every health outcome. Thus, great care must be taken in studies comparing vaccinated and unvaccinated to ensure that the groups are balanced by age. Failure to do so may lead to inaccurate estimates of vaccine effectiveness when the difference in outcomes can be explained, at least partially, by age bias. In trials, randomisation helps ensure statistically identical age distributions in vaccinated and unvaccinated groups, so that the average vaccine efficacy estimate is unbiased, even if vaccine efficacy and/or infection rates differ across age groups (see Figure 2A). However, unlike trials, in real life, vaccination status is not randomly assigned (see Figure 2B). While vaccination rates are high in many countries, the vaccinated remain, on average, older and less healthy than the unvaccinated because vaccines were prioritised for those older and at higher risk. Individuals also self-select for vaccination regardless of policy. Because covid-19 related risks (of infection, disease, and complications) also vary by age, this can confound the estimate of vaccine effectiveness. To illustrate this, consider the REACT-1 study.18 This study conducts PCR testing for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) on a random sample of England's population once a month. In June–July 2021 (the most recent data available), SARS-CoV-2 positivity rates varied considerably by age (from 1.7 to 15.6 positives per 1000 individuals), with higher rates among people under 25 years of age (see Figure 2C). REACT-1 also reports vaccination status. As seen in Figure 2B, almost half of the unvaccinated group is aged between 5 and 12, while the most common age group in the vaccinated was 45–54 years old. While details differ, age bias is present in all observational data sets. To understand the impact of age bias, consider a hypothetical vaccine with zero efficacy. The vaccinated and unvaccinated groups' case rates should be statistically identical if the vaccine were completely ineffective (Figure 2D). But age bias in observational data alters the age-weighted case rates in both the vaccinated and the unvaccinated groups, resulting in different infection rates by vaccination status. Since older people recorded lower infection rates, the age-weighted case rate of the (older) vaccinated group registered at 5.5 per 1000 while the corresponding value for the (younger) unvaccinated group was 11.2 per 1000 (Figure 2C). The resultant vaccine effectiveness, which is the relative ratio of these case rates, reflects the interaction between differential age distributions and the correlation of covid-19 incidence with age. The vaccine effectiveness appears as 51% even though the vaccine is completely ineffective by assumption. (Note that the direction of the age bias would reverse if older age groups had suffered higher case rates during the study period.). A viable adjustment method for this instance of Simpson's paradox19 induced by age bias should shift 51% back to zero. Simpson's paradox describes the condition in which aggregated and disaggregated analyses of the same data lead to contradictory findings, a common phenomenon in real-world data. Many observational studies incorporate an age term into regression models in an attempt to correct this age bias.4, 20, 21 But it has been discovered in a meta-analysis of influenza vaccine studies that standard regression adjustments insufficiently correct for the variety and magnitude of biases.22 From December 2020, the speedy dissemination of vaccines, particularly in wealthier nations (Figure 1), coincided with a period of plunging infection rates. However, accurately determining the contribution of vaccines to this decline is far from straightforward. Indeed, the considerable variation in case decline by country, such as the time lag observed in Israel—by far the quickest to reach 50% vaccinated relative to the UK and the United States—defies simple explanation (Figure 1, timepoint "B"). The sharp drop in infections complicates estimating vaccine effectiveness from observational data in a manner similar to age bias. The risk of virus exposure was considerably higher in January than in April. Thus exposure time was not balanced between unvaccinated and vaccinated individuals. Exposure time for the unvaccinated group was heavily weighted towards the early months of 2021 while the inverse pattern was observed in the vaccinated group. This imbalance is inescapable in the real world due to the timing of vaccination rollout. In addition, unlike trials, individuals in "real-world" studies do not stay in a single analysis subgroup throughout the study period: each person is unvaccinated on the first day of the study until the day of vaccination (or the end of the study should the person remain unvaccinated). Instead of crudely categorising individuals as either "vaccinated" or "unvaccinated," many observational studies split each person's exposure time into an unvaccinated period followed by a vaccinated period if the individual got vaccinated.4-6 This technique is essential in contexts where the vast majority of the population becomes vaccinated, to avoid losing a comparison population. However, this procedure injects a strong bias into the analysis subgroups because the unvaccinated exposure time is heavily skewed to the early period in a study while the exposure time for vaccinated people skews towards the end of the study period. For a hypothetical vaccine with zero efficacy, the case rates for vaccinated and unvaccinated should be equal during each week of the study period. Indeed in RCTs, changes in background infection rate do not bias estimates of vaccine efficacy because by design, vaccine and placebo arms follow a synchronised dosing schedule that ensures exposure (at-risk) time is balanced, even in the context of changing infection rates. But background infection rate bias can cause estimates of vaccine efficacy in "real world" studies to vary widely from 0%. For example, using infection rate data from an actual observational study of Danish nursing home residents,20 where infection rates rapidly declined simultaneous with vaccine rollout (from 12 per 1000 residents in December 2020, to almost 0 during the last 2 weeks of the study),20 vaccine effectiveness of a hypothetically ineffective vaccine appears as 67%, an illusion chiefly created because unvaccinated people were preferentially exposed to the earlier weeks of higher background infection rates (Figure 3). We note that the direction of this bias would reverse if the background infection rate were to have steadily risen during the study period (i.e., vaccinating into a wave rather than out of one). The Danish study was one of the first "real-world" studies to recognise this background infection rate bias. The researchers added a "calendar time" adjustment term to their Cox regression model to address this bias, which reduced their estimate of vaccine effectiveness from 96% to 64%.20 However, as with age bias, we believe that regression adjustment is unlikely to sufficiently cure this type of imbalance. Because the regression equation was not published, we could not make a more definitive assessment. A recent commentary discussed multiple factors that can bias estimates of covid-19 vaccine effectiveness, such as vaccination status misclassification, testing differences, and disease risk factor confounding.7 Our article complements these observations by providing examples based on actual data sets that quantify how case-counting window bias, age bias, and background infection rate bias can profoundly complicate the analysis of observational studies, shifting covid-19 vaccine effectiveness estimates by an absolute magnitude as high as 50% to 70%. Randomised trials aim to mitigate these biases by virtue of design features, such as randomisation, placebo controls, and blinding. But while randomised trials should offer far superior protection against these biases, premarketing trials left many important questions unstudied, such as the durability of protection, interaction with other countermeasures, and effectiveness in highest-risk and other important subpopulations. Pragmatic, placebo-controlled randomised trials might have addressed some of these limitations, but after manufacturers began unblinding their trials following the emergency use authorisation in December 2020, observational studies are all we have. Our analysis shows that real-world conditions such as non-randomised vaccination, crossovers, and trends in background infection rates introduce strong, complex biases into these observational datasets. Our contribution is to size up three important biases, the magnitude of which surprised us and may surprise you. We conclude that "real-world" studies using methodologies popular in early 2021 overstate vaccine effectiveness. Our finding highlights how difficult it is to conduct high-quality observational studies during a pandemic. While the current situation leaves much to be desired, several steps can be taken going forward to enhance the quality of observational studies. Greater awareness of these biases could promote more appropriate adjustments in future studies, including using quasi-experimental methods. In addition, journal editors could improve transparency and reproducibility of observational studies by requiring the disclosure of underlying data and code, as well as publishing modelling equations, tables of coefficients, and standard errors.23 Data availability severely restricted our choice of studies to examine, and also prevented us from analysing all three biases simultaneously, among the ones we selected. As shown in Table 2, we would have needed additional information, such as (a) cases from first dose by vaccination status; (b) age distribution by vaccination status; (c) case rates by vaccination status by age group; (d) match rates between vaccinated and unvaccinated groups on key matching variables; (e) background infection rate by week of study; and (f) case rate by week of study by vaccination status. In future work, we hope to analyse examples using hospitalisations or deaths as endpoints, which is possible only with broader data disclosure. The pandemic offers a magnificent opportunity to recalibrate our expectations about both observational and randomised studies. "Real world" studies today are still published as one-off, point-in-time analyses. But much more value would come from having results posted to a website with live updates, as epidemiological and vaccination data accrue. Continuous reporting would allow researchers to demonstrate that their analytical methods not only explain what happened during the study period but also generalise beyond it. Finally, randomised studies should not be considered irrelevant in the post-authorisation phase. An element of randomisation can be incorporated into real world vaccine distribution. Where populations are still largely unvaccinated and resources do not allow vaccinating everybody at once, designs such as the stepped-wedge cluster randomised rollout24, 25 should be given serious consideration for their ability to ethically derive important scientific information. Any tool that eliminates some amount of real-world bias would reduce the complexity of analysing observational data. Kaiser Fung and Peter Doshi came up with the idea for the paper, Kaiser Fung carried out the statistical analyses and wrote the first draft. All authors were involved in discussing the content, presentation, and editing the manuscript. We have the following interests to declare: Peter Doshi has received travel funds from the European Respiratory Society (2012) and Uppsala Monitoring Center (2018); grants from the FDA (through University of Maryland M-CERSI; 2020), Laura and John Arnold Foundation (2017-22), American Association of Colleges of Pharmacy (2015), Patient-Centered Outcomes Research Institute (2014-16), Cochrane Methods Innovations Fund (2016-18), and UK National Institute for Health Research (2011-14); was an unpaid IMEDS steering committee member at the Reagan-Udall Foundation for the FDA (2016-2020), and is an editor at The BMJ. KF, MJ: None. Data sharing is not applicable to this article as no new data were created or analysed in this study.