Making Sense of Non-Randomized Comparative Treatment Studies in Times of Covid-19: A Case Study of Tocilizumab

Author(s): Ruth Owen, Nawab Qizilbash, Sara Velázquez Díaz, José María Castellano Vázquez, Stuart Pocock

Background: Tocilizumab (TCZ) is an interleukin-6 inhibitor and the second established effective drug for the treatment of hospitalized patients with Covid-19. In this study, we sought to validate the recent positive findings from the randomized clinical trial RECOVERY and to evaluate the challenges in the analysis and interpretation of non-randomized comparative effectiveness studies in Covid-19.

Methods: We performed a retrospective cohort study using an openly available database of hospitalized Covid-19 patients in Spain. The primary outcome was all-cause in-hospital mortality at 28 days. We used multivariable Fine and Gray competing risk models which adjusted for both fixed and time-variant confounders to investigate the effect of TCZ on the primary outcome.

Results: We analyzed 2547 patients hospitalized with Covid-19 between 1st January and 28th June 2020. Patients in the TCZ group tended to have more severe Covid-19 at admission, as measured by biomarkers of disease severity including CRP, D-dimer and LDH. At 28 days, 91 out of 440 TCZ patients had died compared to 267 out of 2107 patients in the control group. In multivariable analysis, there was no evidence of an association between TCZ and the primary outcome (adjusted hazard ratio 1.20, 95% CI 0.86 to 1.64, P=0.26).

Conclusions: Our observational study failed to find a benefit of TCZ on all-cause in-hospital mortality in Covid-19 patients compared with randomized trials, highlighting the impact that unmeasured confounding and other sources of bias can have in a retrospective observational setting. For future observational studies, we recommend prospective data collection to ensure all variables have the necessary quality, completeness and timing for reliable treatment evaluation.