J. Falatko
On November 18th, Pfizer released preliminary data for their COVID-19 vaccine BNT162b2. Earlier in the week they reported the vaccine was 95% effective during their pre-interim analysis with an excellent safety profile. The press release today was the first opportunity to look at some of the raw numbers from the trial.
The study had enrolled 43,611 participants. Of those 41,135 received both scheduled doses of the vaccine. I assume these individuals were divided equally between the vaccine and placebo. So, that gives roughly 21,000 participants in each arm. All of the participants completed at least 28 days of observation after the first dose of the vaccine. The primary endpoint was COVID infections. Adverse events were assessed as well.
COVID infection occurred in 170 participants. There were 8 infections in the vaccine group compared to 162 in the placebo group. Obviously, this was statistically significant. That is a big difference. This would be a 95% effectiveness rate, or to state a little differently a 95% risk reduction from the vaccine. They did not provide confidence intervals.
To break the numbers down a little further. The infection rate in the placebo group was 0.74%. The infection rate in the experimental group was 0.036%. This is equal to an absolute risk reduction of (-) 0.704%. With the negative signifying the risk was reduced. The absolute risk reduction can be used to calculate the number needed to vaccinate (NNV)
NNV = 1/ .00704 = 142
The NNV tells you how many individuals need to be vaccinated to prevent 1 COVID infection. So, for every 142 people that are vaccinated you would prevent one COVID infection.
They provided some data on severe COVID infections as well. There were 9 severe COVID infections in the placebo group compared to 1 in the vaccine group. They did not define severe COVID, but my guess is these participants at least required hospitalization. This gives a severe covid rate of 0.04% in the vaccine group and 0.005% in the control group. That’s a 90% relative risk reduction (effectiveness), but an absolute risk reduction of 0.035% for anyone keeping score.
NNV = 1/0.0035 = 2,857
Since they didn’t look at prevention of deaths due to COVID we would need to run a hypothetical scenario. You could estimate the death rate to be about 10% of the severe COVID rate. So, they would’ve needed to enroll about 10x the number of participants to get any analyzable mortality data. We can simply increase the NNV by a factor of 10 to get some idea of the number needed to prevent 1 death.
NNV = 1/0.00035 = 28,570.
Pfizer does not have any long-term safety data. Participants that received the vaccine reported fatigue (3.8%) and headache (2%) as the most common side effects.
When I look at this data and I think to myself, “What the heck? Are they living in the same world as the rest of us?” How did they only get 170 infections out of 42,000 participants? The prevalence of COVID in their trial was only 0.4%. Each participant was studied for at least 28 days and they only had 170 cases. Did the protocol include quarantining in your basement for those 28 days? It’s bad out there. Really bad. And it’s been bad for at least a month, according to everything I read. Daily case counts and deaths are higher than ever. When I watch the news, everything is Red. They don’t stop at just red. There are different shades of red, which I imagine is worse than just red.
If I were on the Pfizer board reviewing this study, I would be frustrated. It would not be ethical to purposefully expose participants to COVID. But maybe you could encourage them to mix it up a little bit. Go to your gym. Head to a restaurant. Catch a movie at your local movie theatre kind of mix it up. I want a combat ready vaccine. A Jon Snow at the “Battle of the Bastards” vaccine. A standing in a 10x10 room with no windows and no vents 5 people coughing, an opera singer, and a crying baby yelling at me simultaneously ready vaccine. I want to know how it performs when we’re mixing it up. So, I’m going to spend some time hypothesizing some real-time scenarios, and discussing relative and absolute risk.
This data is a good example of the difference between relative risk reduction and absolute risk reduction. The 95% effectiveness can be misleading if there are very few events. Relative risk is exactly that. Its relative between the two groups. Absolute risk reduction is the bang for your buck. It’s what you actually get based on the baseline risk. Based on this dataset the individual does not reduce their risk much. But, you have to ask yourself if the sample represents the place you live? The prevalence of a disease is so important in interpreting statistics from a research study. Prevalence is the proportion of individuals in a population living with the disease. If your relative risk reduction holds true, the absolute risk reduction changes greatly depending on the population you treat.
It is unlikely the prevalence is as low as 0.4% in the real world. If you keep the relative risk the same, and simply double the prevalence of the disease to 0.8%, your NNV to prevent one infection reduces to 68. If you triple the prevalence to 1.2%, your NNV reduces further to 45. At a 4% prevalence (Indiana), your NNV reduces to 13. So, every area is going to get different benefit from the vaccine based on the underlying prevalence of the disease.
California – prevalence 2.6% - NNV = 20
Illinois – prevalence 4.7% - NNV = 11.5
Texas – prevalence 3.6% - NNV = 15.2
Michigan – prevalence 3.0% - NNV = 18
New York – prevalence 2.5% - NNV = 21.4
South Dakota – prevalence 7.8% - NNV = 7.1
“That’s some applied epidemiology bro.”
Although I was disappointed to see the low case numbers in the released data, I don’t think they represent what’s happening in the world. South Dakota is a combat zone. If the reduction ratios of the vaccine hold up in a real-world setting, it will perform very impressively. It will benefit individuals and the population on a massive scale.
In my opinion there’s only two ways this pandemic ends; it either burns out (herd immunity), or it gets knocked out. I prefer the latter. Social distancing, staying home, wearing masks, that stuff is nice. But it’s hard to win when you’re only playing defense. Other countries may be playing better defense, but they are waiting for the same thing. I’m sure if COVID were some cinematic super villain looking at these numbers it would think, “Oh, that’s cute.” Kind of the same way we all looked at Arya Stark in the first episode of Game of Thrones, before she turned into the ultimate assassin.
The vaccine is the equivalent of an Atom Bomb. It's going to end the war and save countless lives. So, scrap this paltry dataset. Bring on the real world, and let’s end this thing.
Link: https://www.pfizer.com/news/press-release/press-release-detail/pfizer-and-biontech-conclude-phase-3-study-covid-19-vaccine
Comments