There has been a lot of discussion lately about safety and COVID-19. When should people feel safe to gather together, go to sporting events, movie theaters restaurants, etc? Also, when should our children be allowed to return to school. Education is a foundation in our society for several reasons. Two in particular is that education prepares children for adult hood, also it allows adult to work and be productive. Some “experts” and politicians have suggested that we will not be safe until there is an effective vaccine.
Vaccination is the most advanced technology humans have created to combat disease. However, its effectiveness and need are directly related to the virulence and infectiveness of the antigen. For example, polio was a highly infectious virus that spread easily. Roughly 90 percent of carriers were asymptomatic, so avoidance was difficult. The virus was fatal or severely disabling in 0.1 percent of those infected.1 Almost every child in the world would be exposed, and there was no treatment, so people lived in great fear. To crunch some numbers today there are roughly 117 million births annually across the globe. The polio vaccine prevents 117,000 deaths/disability per year. Measles is very similar.2 Prior to the measles vaccine there were roughly 550,000 measles related deaths per year. Now there are fewer than 100,000 cases annually. These are some of the best examples of the power of vaccination.
Let’s go through some of the COVID-19 statistics here in the U.S. before we address the issue of a vaccine and safety. As of May 1st, there are 1.24 million confirmed cases of COVID in the U.S. The death toll is 60,000. Testing has increased substantially over the past month and there are roughly 20,000 new cases detected each day.3 As of May 1st, 79 percent of the deaths have occurred in those over the age of 65. An estimated 92 percent of the deaths have occurred in those over the age of 55. There are roughly 330 million people living in the united states. Citizens under the age of 65 represent 270-280 million of the population. For simplicity purposes we will use 270 million.4 Serology samples collected in Los Angeles county and New York suggest there are far more infections than have been recording.5,6 Some estimates are as high as 20 percent. 7,8 We can assume there has been some sampling error and/or some level of false positives in these studies. For the calculations used today we can assume a 10 percent penetrance of the disease in the general population thus far.
I’ll start by discussing the theoretical design for a randomized controlled trail for a vaccine against COVID-19. If researchers want to run a trial for the effectiveness of a vaccine, they have to determine the number of patients required in each arm to achieve a statistical result that is unlikely to be due to random chance. This is known as statistical power. I won’t bore you with an official power analysis. We will use the following basic rule of thumb. To have any change of detecting a difference there should be at least 50 events in each arm of the trial. You would also need to determine an endpoint. For this theoretical design we will use fatality due to COVID-19. If it were a true trial, I would likely use fatality and severe lung infection (ARDS) due to COVID since these are the outcomes I would care most about. For, simplicity purposes I will only use fatality.
Let’s start by discussing the theoretical design for the entire population. So, there are 330 million people. We’ll assume there is 10 percent penetrance of the disease. As of May 1st, there had been 60,000 deaths.3 For an event rate we calculate (# of deaths/ those exposed = event rate x 100) equals 0.18 percent. To estimate the power, based on the rule above, required for the study you would divide 50 by the event rate (minimal # of events / event rate) (50/.0018) equals 27,777 patients required in each arm of the study. This is a reasonable trial size. However, if you have a theoretical effectiveness of 50 percent for your vaccine you would need to double this size to ensure you have at least 50 deaths in your experimental arm.
Next, we’ll approach a theoretical design for those 64 and younger. As of May 1st, there were roughly 15,000 deaths in this age group. They represent 270 million people. Again, we’ll assume a 10 percent penetrance of the disease. In this population the event rate is .055 percent. To estimate the power based on the rule above a researcher would need 90,909 patients in each arm of the study. If the theoretical effectiveness is again 50 percent a researcher would consider doubling the trial size. This trial is starting to get large, considering the potential adverse effects of a vaccine the law of diminishing returns may become an issue.
Next, we’ll approach a theoretical design for those 54 and under. As of May 1st, there were roughly 7000 deaths in this age group. They represent 250 million people. Again, we’ll assume a 10 percent penetrance of the disease. The event rate in this group is .028 percent. The estimated power based on the rule above would be 178,571 in each arm of the study. Law of exponentiation is taking effect.
Finally, we’ll design our study for those over 65 years of age. As of May 1st, there were roughly 30,000 deaths in this age group. They represent 50 million people. We’ll assume a 10 percent penetrance rate in this population. Our event rate in this group is 6 percent. The estimated power based on the rule above is 833 patients in each arm. Assuming a vaccine with a 50 percent effectiveness our trial size is 1666 patients in each arm to ensure the number of events needed. Yahtzee!
Now that we understand number needed for each trial, we can simulate what a vaccine trial might result. I’ll simulate the numbers in the groups at the extremes of the scenario above (less than 55, and over 65). First, those less than 55. For simplicity the study will have 178,571 in each arm. Let’s assume our vaccine worked better than expected and the trial ran to perfection. The vaccine showed a 75 percent relative risk reduction (sounds awesome). There were 12 deaths in the experimental arm and 50 deaths in the control arm. This results in an absolute risk reduction or ARR (control event rate – experimental event rate) of .021% (not so good). Next, we’ll calculate the number need to vaccinate. This number represents the number of people that need to be vaccinated to prevent one death. This is calculated by 1/ARR, which equals 4,762. For every 4,762 that are vaccinated we save one life from COVID-19 in this age group.
You get a lot more for your vaccine in those over 65. We’ll maintain the same assumptions above. We’ll hedge ourselves for this trial and our trial size will be 1,666 patients in each arm. There were 12 deaths in the experimental arm and 50 in the control arm for a relative risk reduction of 75%. The absolute risk reduction or ARR was 2.3%. The number needed to vaccinate is 43.5. Nice!
The original issue was that Americans will not be safe until there is a vaccine. With a number needed to vaccinate near 5,000, if every single person under the age of 55 were vaccinated it would theoretically prevent 50,000 deaths. That is assuming, at some time in the future, everyone is exposed to the virus. That may seem like a lot, but remember your base is 250 million people. Without the vaccine you could estimate total number of deaths in the age group at 70,000. There are two caveats to these numbers. Frist, they assume everyone will become infected with the virus. Second, it appears daunting if it occurred all at once. However, as immunity spreads, the virus will slow therefore death rates will slow. Even with a vaccine that is 75 percent effective you can expect 20,000 deaths. The lives of 250,000,000 people, put on hold, for this benefit, assuming everyone gets exposed and everyone agrees to be vaccinated.
Does having a vaccine make you feel safe? Knowing that you would not have full cooperation, how well do you expect the vaccine to perform? Where is the true value?
The event rate in those over 65 certainly justifies safety measures until a vaccine is available. For the rest of us, vaccination reduces individual risk very little. In the 1/1000s of a percentage point to be exact. However, it does benefit the whole of society in preventing spread of the virus through herd immunity. Anyone that knows me, knows I am a huge proponent of vaccination. Should we wait to open our schools, businesses, and events until a vaccine is available? That’s likely unnecessary. The risk of unintended consequences from 250 million people sheltering in place likely outweighs the risk to the individual from COVID.
Obviously, this article makes a lot of assumptions. The biggest assumption is those that have been exposed. The true number of those with prior infection or active infection is unknown. The assumptions were made based on small samples around the globe. Since the total exposure is unknown the true event rate is unknown. Because of this there could be substantial error in my logic. Also, I didn’t bother with a power calculation for simplicity purposes. The estimate figures used for the simulated trials could be inaccurate based on this. Finally, I gave an effectiveness of 75 percent to the theoretical vaccine. This would be on the favorable end of effectiveness compared to others. 9,10,11 Vaccine effectiveness can vary significantly from ineffective to near 100 percent effective.
Based on these assumptions this entire article could be rubbish. Only time will tell. If you see a vaccine trial with over 80,000 people in each arm, and less than 100 deaths in each arm, then you’ll know I was probably right. In that case we should’ve gotten back to our lives long before we did.
1. Mueller S, Wimmer E, Cello J. Poliovirus and poliomyelitis: a tale of guts, brains, and an accidental event. Virus Res. 2005;111(2):175-193. doi:10.1016/j.virusres.2005.04.008
2. Centers for Disease Control and Prevention (CDC). Global measles mortality, 2000-2008. MMWR Morb Mortal Wkly Rep. 2009;58(47):1321-1326.
3. CDC. Coronavirus Disease 2019 (COVID-19) in the U.S. Centers for Disease Control and Prevention. Published April 29, 2020. Accessed May 7, 2020.
https://www.cdc.gov/coronavirus/2019-ncov/cases-updates/cases-in-us.html
4. Population Clock. Accessed May 7, 2020. https://www.census.gov/popclock/
5. Sutton D, Fuchs K, D’Alton M, Goffman D. Universal Screening for SARS-CoV-2 in Women Admitted for Delivery. N Engl J Med. Published online April 13, 2020. doi:10.1056/NEJMc2009316
6. Los Angeles study suggests virus much more widespread. Modern Healthcare. Published April 20, 2020. Accessed May 7, 2020. https://www.modernhealthcare.com/safety-quality/los-angeles-study-suggests-virus-much-more-widespread
7. Mizumoto K, Kagaya K, Zarebski A, Chowell G. Estimating the asymptomatic proportion of coronavirus disease 2019 (COVID-19) cases on board the Diamond Princess cruise ship, Yokohama, Japan, 2020. Euro Surveill Bull Eur Sur Mal Transm Eur Commun Dis Bull. 2020;25(10). doi:10.2807/1560-7917.ES.2020.25.10.2000180
8. Gudbjartsson DF, Helgason A, Jonsson H, et al. Spread of SARS-CoV-2 in the Icelandic Population. N Engl J Med. Published online April 14, 2020. doi:10.1056/NEJMoa2006100
9. Dawood FS, Chung JR, Kim SS, et al. Interim Estimates of 2019-20 Seasonal Influenza Vaccine Effectiveness - United States, February 2020. MMWR Morb Mortal Wkly Rep. 2020;69(7):177-182. doi:10.15585/mmwr.mm6907a1
10. Watson JC, Pearson JA, Markowitz LE, et al. An evaluation of measles revaccination among school-entry-aged children. Pediatrics. 1996;97(5):613-618.
11. Bonten MJM, Huijts SM, Bolkenbaas M, et al. Polysaccharide conjugate vaccine against pneumococcal pneumonia in adults. N Engl J Med. 2015;372(12):1114-1125. doi:10.1056/NEJMoa1408544