학술논문
Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United States
Document Type
article
Author
Cramer, Estee Y; Ray, Evan L; Lopez, Velma K; Bracher, Johannes; Brennen, Andrea; Rivadeneira, Alvaro J Castro; Gerding, Aaron; Gneiting, Tilmann; House, Katie H; Huang, Yuxin; Jayawardena, Dasuni; Kanji, Abdul H; Khandelwal, Ayush; Le, Khoa; Mühlemann, Anja; Niemi, Jarad; Shah, Apurv; Stark, Ariane; Wang, Yijin; Wattanachit, Nutcha; Zorn, Martha W; Gu, Youyang; Jain, Sansiddh; Bannur, Nayana; Deva, Ayush; Kulkarni, Mihir; Merugu, Srujana; Raval, Alpan; Shingi, Siddhant; Tiwari, Avtansh; White, Jerome; Abernethy, Neil F; Woody, Spencer; Dahan, Maytal; Fox, Spencer; Gaither, Kelly; Lachmann, Michael; Meyers, Lauren Ancel; Scott, James G; Tec, Mauricio; Srivastava, Ajitesh; George, Glover E; Cegan, Jeffrey C; Dettwiller, Ian D; England, William P; Farthing, Matthew W; Hunter, Robert H; Lafferty, Brandon; Linkov, Igor; Mayo, Michael L; Parno, Matthew D; Rowland, Michael A; Trump, Benjamin D; Zhang-James, Yanli; Chen, Samuel; Faraone, Stephen V; Hess, Jonathan; Morley, Christopher P; Salekin, Asif; Wang, Dongliang; Corsetti, Sabrina M; Baer, Thomas M; Eisenberg, Marisa C; Falb, Karl; Huang, Yitao; Martin, Emily T; McCauley, Ella; Myers, Robert L; Schwarz, Tom; Sheldon, Daniel; Gibson, Graham Casey; Yu, Rose; Gao, Liyao; Ma, Yian; Wu, Dongxia; Yan, Xifeng; Jin, Xiaoyong; Wang, Yu-Xiang; Chen, YangQuan; Guo, Lihong; Zhao, Yanting; Gu, Quanquan; Chen, Jinghui; Wang, Lingxiao; Xu, Pan; Zhang, Weitong; Zou, Difan; Biegel, Hannah; Lega, Joceline; McConnell, Steve; Nagraj, VP; Guertin, Stephanie L; Hulme-Lowe, Christopher; Turner, Stephen D; Shi, Yunfeng; Ban, Xuegang; Walraven, Robert; Hong, Qi-Jun; Kong, Stanley; van de Walle, Axel
Source
Proceedings of the National Academy of Sciences of the United States of America. 119(15)
Subject
Language
Abstract
Short-term probabilistic forecasts of the trajectory of the COVID-19 pandemic in the United States have served as a visible and important communication channel between the scientific modeling community and both the general public and decision-makers. Forecasting models provide specific, quantitative, and evaluable predictions that inform short-term decisions such as healthcare staffing needs, school closures, and allocation of medical supplies. Starting in April 2020, the US COVID-19 Forecast Hub (https://covid19forecasthub.org/) collected, disseminated, and synthesized tens of millions of specific predictions from more than 90 different academic, industry, and independent research groups. A multimodel ensemble forecast that combined predictions from dozens of groups every week provided the most consistently accurate probabilistic forecasts of incident deaths due to COVID-19 at the state and national level from April 2020 through October 2021. The performance of 27 individual models that submitted complete forecasts of COVID-19 deaths consistently throughout this year showed high variability in forecast skill across time, geospatial units, and forecast horizons. Two-thirds of the models evaluated showed better accuracy than a naïve baseline model. Forecast accuracy degraded as models made predictions further into the future, with probabilistic error at a 20-wk horizon three to five times larger than when predicting at a 1-wk horizon. This project underscores the role that collaboration and active coordination between governmental public-health agencies, academic modeling teams, and industry partners can play in developing modern modeling capabilities to support local, state, and federal response to outbreaks.