Forecasting for COVID-19 has failed

"COVID-19 is a major acute crisis with unpredictable consequences. Many scientists have struggled to make forecasts about its impact (Holmdahl & Buckee, 2020). However, despite involving many excellent modelers, best intentions, and highly sophisticated tools, forecasting efforts have largely failed...

Brilliant scientists expected 100,000,000 cases accruing within 4 weeks in the USA (Hains, 2020). Predictions for hospital and ICU bed requirements were also entirely misinforming. Public leaders trusted models (sometimes even black boxes without disclosed methodology) inferring massively overwhelmed health care capacity (Table 1) (IHME COVID-19 health service utilization forecasting team & Murray, 2020). However, very few hospitals were eventually stressed and only for a couple of weeks. Most hospitals maintained largely empty wards, expecting tsunamis that never came. The general population was locked and placed in horror-alert to save health systems from collapsing. Tragically, many health systems faced major adverse consequences, not by COVID-19 cases overload, but for very different reasons. Patients with heart attacks avoided hospitals for care (De Filippo, D’Ascenzo, Angelini, et al., 2020), important treatments (e.g. for cancer) were unjustifiably delayed (Sud et al., 2020) and mental health suffered (Moser, Glaus, Frangou, et al., 2020). With damaged operations, many hospitals started losing personnel, reducing their capacity to face future crises (e.g. a second wave)...

Table 3 lists some main reasons underlying this forecasting failure. Unsurprisingly, models failed when they used more speculation and theoretical assumptions and tried to predict long-term outcomes; for example,  using early SIR-based models to predict what would happen in the entire season. However, even forecasting built directly on data alone fared badly (Chin et al., 2020b, Marchant et al., 2020), failing not only in ICU bed predictions but also in next day death predictions when issues of long-term chaotic behavior do not come into play...

Failure in epidemic forecasting is an old problem. In fact, it is surprising that epidemic forecasting has retained much credibility among decision-makers, given its dubious track record. Modelling for swine flu predicted 3100–65,000 deaths in the UK... Eventually, 457 deaths occurred (UK government, 2009)...

Despite these obvious failures, epidemic forecasting continued to thrive, perhaps because vastly erroneous predictions typically lacked serious consequences… Upon acquiring solid evidence about the epidemiological features of new outbreaks, implausible, exaggerated forecasts... should be abandoned. Otherwise, they may cause more harm than the virus itself."