Limitations of models for guiding policy in the COVID-19 pandemic

“Abstract: At the outset of the COVID-19 epidemic in the UK, infectious disease modellers advised the government that unless a lockdown was imposed, most of the population would be infected within a few months and critical care capacity would be overwhelmed. This paper investigates the quantitative arguments underlying these predictions, and draws lessons for future policy…

Discussion: The analyses above show that of the four propositions on which the recommendation for lockdown was based, one – the assumption that 2% of those infected would require critical care – was unequivocally wrong, and another – that mitigation through focused protection would not be effective in limiting morbidity and mortality – was not seriously questioned. The other two propositions – that in an unmitigated epidemic 80% would be infected, and that only a lockdown could suppress the epidemic – were reliant on strong but unrealistic modelling assumptions and weak data…

The modelling reports at the outset of the epidemic in March 2020 failed to communicate that by relying on the unrealistic assumption of no unmeasured heterogeneity they were likely to overestimate the size of the epidemic, and that no reliable prediction of the effects of non-pharmaceutical interventions could be made without more information about the mode of transmission. This suggests that policy advice in future epidemics should rely less on models, with a greater priority given to the rapid establishment of high quality direct measurement studies.”