Introduction
This brief report is an assessment of the research methods and results from the randomized controlled trial investigating the effects of Remdesivir vs. placebo for the treatment of COVID-19. Remdesivir inhibits the activity of RNA polymerase, which is a protein that is utilized by RNA dependent viruses for replication. Since COVID-19 is an RNA virus, Remdesivir is a promising candidate.
Trial Design
I’ll start with the trial design. This was a randomized controlled trial, which means patients were divided into the experimental arm and control arm in a random order. Randomization is important because it equalizes the prognosis of each group. In order to further protect against systemic bias the randomization order was stratified based on disease severity. This means that patients with signs of severe disease were randomized in a separate order than others. Researchers do this to avoid the majority of severely ill patients ending up on one side by random chance. Randomization was concealed indicating that the order could not be predicted by investigators. The study was double blinded, which means the researchers and patients were unaware of the assigned group. They do not mention if adjudicators, statisticians, and data collectors were blinded. This makes manipulating the group after the trial starts difficult since the conductors of the study are unaware of assignment. Their randomization strategy and blinding were successful. They include a statement of no significant differences between the two groups. However, they do not provide the comparative analysis in the supplement material.
The study was stopped early for perceived benefit. This occurred at a prespecified interim analysis. The trial was stopped because anticipated enrollment had been achieved based on prespecified number of recoveries. (Early stoppage of a trial needs to be considered very carefully since it provides an opportunity for manipulation. I’ll explain later in this article.) The attrition (patients that left the study) is muddled. Less than half received the full 10 doses in the protocol, which is not good. Less than 10% withdrew from the trial in each arm. This is good. Roughly 20% were still in the protocol at the time of the analysis due to early stoppage. The totals are in the table below.
Enrolled
Received Treatment
Completed Protocol
Recovery unknown
Treatment
541
538
391
132
Control
522
521
340
169
Results
The results showed that a patient was likely to have fewer days of illness (11 vs. 15) if they were given Remdesivir. They used median days to recovery suggesting the data was not evenly distributed. This may be due to patients with mild to moderate disease moving down on the ordinal scale faster than expected. This is evident by the skew towards healthier patients since the confidence intervals for severely ill patients crossed 1 (no significance). The confidence interval represents a range in values in which the truth lies. If it crosses 1 there is no difference detected since the true value may rely on either side. You can only know the range of truth, not the actual truth. The 95% represents your degree of confidence. This is an important concept to understand.
It is quite possible that the study was underpowered to draw conclusions from the subgroups reported. The wide confidence intervals suggest this is the case. However, they performed an analysis correcting for underlying disease severity and obtained a similar result. This supports the validity of the “recovery rate ratio’s” reported.
Group
Confidence Interval
Entire study group (1.12-1.55)
Category 4 (mild) (0.94 – 2.03)
Category 6 (moderate) (0.79 – 1.81)
Mechanical Ventilation/ECMO (severe) (0.64 – 1.42)
There was no statistically significant difference in mortality. The 95% confidence interval was (0.47 - 1.04). The events were few. There were 32 and 54 deaths in the Remdesivir and placebo groups respectively. Differences occurred in severity groups 5 and 7. Exercising the lower bound principle in this case (truth is close to the lower edge of the interval) would suggest that Remdesivir may be quite effective at preventing death. However, due to low event rates there is a high risk that the result was due to random chance even though the confidence interval is getting close to 1.
There were no substantial differences in adverse events between the two groups.
Conclusion
Remdesivir does appear promising in the fight against COVID-19, but its use will be limited. First, the patients in this study were fairly ill. They were all hospitalized with most requiring supplemental oxygen or advanced care. The applicability of the results is limited due to the population studied. Second, Remdesivir is delivered intravenously limiting its application to a broader population.
All patient important outcomes were trending towards significance. Recovery occurred sooner, severity of illness trended towards significance, and mortality trended towards significance. I assume these would be broad categories patients care most about. Most people want to know if it will help them recover, keep them from worsening, and prevent them from dying. Viral shedding may have been a valuable surrogate for transmission prevention. Hard clinical outcomes outside of an ordinal scale like mechanical ventilation rates, end organ damage, development of ARDS are important endpoints to consider. Unfortunately, these endpoints were not evaluated.
In my opinion, the trial should have been allowed to run to completion. Although they had completed the expected enrollment, over 100 patients in each arm were still in the protocol. Less than half the patients received all 10 doses of Remdesivir. The purpose to stop the trial is understandable. The pressure on the investigators was likely immense and coming from multiple entities, but this weakened the amount and reliability of the data. There was likely an ethical debate as well. If it works, why continue to give placebo to those participating in the trial? They analyzed all of the enrolled patients even though a large number did not complete the protocol. To do this the researchers or adjudicators had to be unblinded to the assess an outcome prior to the end of the protocol, or else these patients would have been censored form the beginning creating a massive skew in the data. The other option would have been to not record outcomes for these 200 patients, which may have underpowered their study. I’m sure they ran the analysis without them and likely discovered this result. The primary endpoint would be difficult to manipulate. Subjectivity only comes into play in scale numbers 3 and 4.
Including those 200 patients in the analysis limits your mortality and “improvement in ordinal scale” data. Since they did not complete the 28-day protocol there were probably a significant number of those 200 patients in which their outcome had yet to be determined. They may have still been in the hospital, or on ventilators, or improved and then worsened. The authors did not present the 28 day mortality Kaplan meier survival curve, but did show a 14 day survival curve, which approached statistical significance. However, I would reiterate that the low number of events exposes this result to random chance despite the analysis. It is not difficult to imagine a scenario in which the next 2 or 3 deaths are in the Remdesivir group and all of the sudden your confidence interval widens further. This is the concept I am attempting to convey when I discuss low event numbers. It only takes a couple of events in one group to completely change the interpretation of the results.
The other issue with Remdesivir is that it is difficult to manufacture. The raw materials needed to make the drug come from a complicated supply chain and the production process is challenging and cannot be easily scaled. This will limit the access to large institutions around the world in major cities that see large numbers of patients. Available supplies will likely be purchased and stored by governments and large institutions in preparation for surges in cases limiting availability of the drug. It is unlikely that these results will be applicable to many people simply because they will not have access to the technology.
Remdesivir does appear to be effective at helping patients recover. Its ability to prevent death due to COVID is unknown. Its use will most likely be limited to hospitalized patients with moderate to severe disease at major referral centers due to limited access even though mild to moderate disease severity appear to benefit the most. It is unlikely to make a substantial impact in the fight against COVID due to its limited applicability, access, and administration issues. However, if I am sick in the hospital with COVID and on oxygen I hope I’m at a place that has some because I’ll be asking for it.