J. Falatko
Link to article: https://www.bmj.com/content/370/bmj.m3320
As more and more data for COVID becomes available, doctors are gaining more understanding of at-risk populations. This article was recently published by BMJ (british medical journal) looking into the risk of pregnant women with COVID-19. It was brought to my attention by several high-profile physicians that I follow on Twitter.
The study is a meta-analysis. For those of you whom may not know what this is, a meta-analysis is an amalgamation of all the available data surrounding a specific medical question. The studies are built through a rigorous process performed by the authors known as a systematic review. To be included, the studies need to meet criteria established by the authors and must be of moderate to high quality. Once the studies are selected the available data is combined and analyzed. The original data sets for the studies belong to the original authors. This is combining their results and re-running the analysis.
Meta-analyses are considered the highest level of evidence we have. However, there can be faults with them. If the underlying studies are of poor quality, you will get a poor result. This is commonly known as “garbage in, garbage out.” If the systematic review is not performed correctly, then there may have been some studies that were missed that could impact the end result. Also, how similar the results were from study to study is very important. In a good meta-analysis all of the studies are very similar (homogenous). If one study is significantly larger than the others, or the results vary from study to study, it will influence the final result.
Let’s dive into this study and see what they found.
Methods
When assessing the methods of a meta-analysis there are several questions that must be answered. First, is the question being asked sensible? Was the search for relevant studies exhaustive? Were the studies included free of bias, or of high quality? Finally, was the process for selecting studies reproducible?
The sensibility of the question for a meta-analysis boils down to the scope of the question. Very broad questions are difficult to answer with a meta-analysis or any research for that matter. An example of a broad question would be, “The effect of air quality on cancer incidence.” There are many different cancers. They have different risk factors, prognosis, treatments, etc. This question is too broad. A better question would be, “The effect of air quality on the development of lung cancer.” For narrow questions like " the effect of air quality and the development of chemotherapy resistant metastatic melanoma," meta-analyses do not make much sense, since data is sparse and often of poor quality.
In this study they were asking, “What is the effect of COVID-19 in pregnant women?” I would argue that this a decent question but it is fairly broad. We know that different populations of people are at different risks for complications from COVID based on their underlying health. A successful pregnancy is dependent upon the underlying health of the mother as well. A better question might be, “What is the effect of COVID in healthy pregnant women?” Compare them to health women without COVID. One could consider, “What is the effect of COVID-19 in pregnant women with underlying illnesses?" Then compare COVID outcomes to non-pregnant women with these illnesses.
Next, was the search for data exhaustive?
The authors performed an exhaustive search for data. They searched several international databases. They inquired about data from available registries. They contacted authors of the published studies to see if they had any unpublished data. They even searched blogs and preprint websites.
Were the studies included of high methodologic quality?
This was a meta-analysis of observational studies, which I despise. Observational studies are riddled with issues. Uneven groups, confounding, residual confounding, inadequate follow-up, are just some of the issues. A meta-analysis is a combination of studies. The issues in observational studies are compounded in a meta-analysis.
The authors used the Newcastle Ottawa Scale to determine validity of the underlying studies. This is the standard scale used for this purpose. The scale was completed by two independent reviewers. They found 67% of the included studies had a low risk of bias overall. Really, the other 33% shouldn't have made the cut. The most glaring issue was 84% of the studies had substantial risk of bias between the control and experimental groups. Goodness! This means that for most of the included studies the control group and the experimental group were different. Also, they found low risk of bias for adequate follow-up in only 22% of the studies. This suggests there was a lot of missing data.
For a meta-analysis, you want all of the data to be of high quality. If it isn’t, your final result is suspect. This is the trouble with grouping observational studies. If the underlying studies are poor, the result of the meta-analysis is poor.
Finally, was the process for selecting and assessing the studies reproducible. In their methods they state that two independent reviewers completed the search and assessment of the studies. If there were any disputes, they were resolved by a independent 3rd reviewer. This is standard for a meta-analysis.
Results
Overall 77 studies were included. Data was analyzed on 13,000 pregnant patients with COVID and 83,000 women without COVID. Pregnant women were less likely to have fever, myalgias, or shortness of breath. After sensitivity analysis this was mainly due to pregnant women being screened for COVID, not presenting to the hospital with symptoms. Pre-existing diabetes was more common in the pregnant women with COVID than non-pregnant women.
The number of deaths reported in pregnant women with confirmed COVID cases was 73. When compared to non-pregnant women of reproductive age ICU admission (CI 1.33-1.96)*, mechanical ventilation (CI 1.36-2.60), were higher in pregnant women. There was no difference in mortality.
In pregnant women with COVID compared to pregnant women without COVID, pre-term birth was more likely (CI 1.15-7.85). There was no increased risk in neonatal deaths.
* CI = confidence interval
Discussion
In this study the authors found that pregnant women with COVID were less likely to be symptomatic, but were more likely to require mechanical ventilation, and be admitted to the ICU from complications related to COVID. Something is fishy here. They were less likely to know they were sick and more likely to be severely ill at the same time. How can this be?
There were a couple of significant issues with the study. The first is the issue of a sensible question. Lately, my wife and I have been binge watching the show “Top Chef.” In this show Chef’s from around the country compete at different cooking challenges. The Chef that survives until the end without being dismissed by the judges for a bad dish is the winner. The Chef’s want to showcase their skills. They have an arsenal of cooking techniques. Sometimes a Chef will try to do too much with a dish and add too many ingredients. The flavors get muddled together and the dish falls apart.
The same thing happens when the question is too broad. The studies included were on the same topic, but they were addressing different questions. In this analysis, some of the studies were women screened for COVID when they came to the hospital for delivery. Other studies involved women going to the hospital because of COVID that happened to be pregnant. Some of the data comes from the hospitalized women in Wuhan province of China, while other data came from southern Connecticut, and the majority came from a CDC database. The studies varied in timing, location, type of test used, and reported outcomes. It’s a bit of a mess.
This was proven by the heterogeneity analysis. The studies in a meta-analysis should be homogenous (everything should look the same). You don’t want them to be different. The heterogeneity of a study is represented by the I2 test. The lower the better. A lower number indicates there was little difference between the results. In this case, most of the prevalence and symptom related data was > 80%, which indicates significant variability between the studies.
When comparing outcomes with non-pregnant women there were only a few statistically significant results with good I2. These were ICU admissions and mechanical ventilation. There was no difference in mortality. I’ve discussed ICU admission in previous posts. It’s an ok endpoint, but it can be subjective and vary considerably from one institution to another. Need for mechanical ventilation was significant but the overall event rate was low (0.5% vs. 0.3%). The I2 was 0, indicating consistency between the studies. However, only 4 studies recorded this outcome.
The other statistically significant result was in comparison to women without COVID and this was pre-term births. The I2 for this outcome was low at 0.9 and the result was statistically significant, but total events in the study were low at 7 vs. 18. Again, only 2 of the included studies recorded this result. Certainly, this trend would suggest COVID is associated with pre-term birth, but it is not substantial, and subject to provider bias. There was no difference in fetal distress or neonatal death.
Another issue is of the 83,000 women in the control group, 75,000 of them were from one study. The data for these 75,000 women were extracted from the CDC database. When the weight of one study is so much heavier than all the other studies, it doesn’t make much sense to combine the data. You’re better off reviewing the largest study.
Meta-analyses have become somewhat of a pandemic themselves lately. In the publication race they are low hanging fruit. They analyze big data, so you’re likely to get a significant result somewhere. Since they are touted as superior evidence to clinical trials, they almost always find their way into academic journals. Lumping poor evidence with a well conducted clinical trial, only dampens the signal from the best executed studies. They should answer a specific question, aggregate all the available data, and the underlying data should be of excellent quality. Its improbable that any meta-analysis investigating a question surrounding COVID meets these criteria. Most studies have been executed hastily with few patients and questionable methods. The studies included in this analysis are no different, so the results produced should be interpreted with caution.
As you can read, the statement that pregnant women are less likely to be symptomatic but more likely to have severe COVID is fundamentally not true. The data is more complicated than this.
Pregnancy complicates most diseases. If you have poor access to healthcare, and are unhealthy, your pregnancy is going to be tough. If you get COVID on top of that it’s probably going to get tougher. The same is true for non-pregnant women. A recent publication in BMJ showed a similar phenomenon with pre-term delivery and mortality in women. Even the data form this report shows that pre-existing disease significantly impacts the results.
The inverse is likely true for healthy individuals. Unfortunately, these studies are lumping everyone together, so our ability to assess risk is muddled. I’m sure pregnancy slightly increases the risk of complications from COVID. Similar to other viruses like, RSV, human metapneumovirus, and influenza that infect the respiratory tract. So, even though COVID is a threat, there are greater risks to your pregnancy. Do your best to protect yourself during your pregnancy. If you get sick, go see your doctor. Don't be afraid to go to the hospital, it is the safest place to deliver your baby, COVID or no COVID.