Journal Article > ResearchFull Text
PLoS Comput Biol. 23 May 2022; Volume 18 (Issue 5); e1008800.; DOI: 10.1371/journal.pcbi.1008800
Jarvis CI, Gimma A, Finger F, Morris TP, Thompson JA, et al.
PLoS Comput Biol. 23 May 2022; Volume 18 (Issue 5); e1008800.; DOI: 10.1371/journal.pcbi.1008800
The fraction of cases reported, known as 'reporting', is a key performance indicator in an outbreak response, and an essential factor to consider when modelling epidemics and assessing their impact on populations. Unfortunately, its estimation is inherently difficult, as it relates to the part of an epidemic which is, by definition, not observed. We introduce a simple statistical method for estimating reporting, initially developed for the response to Ebola in Eastern Democratic Republic of the Congo (DRC), 2018-2020. This approach uses transmission chain data typically gathered through case investigation and contact tracing, and uses the proportion of investigated cases with a known, reported infector as a proxy for reporting. Using simulated epidemics, we study how this method performs for different outbreak sizes and reporting levels. Results suggest that our method has low bias, reasonable precision, and despite sub-optimal coverage, usually provides estimates within close range (5-10%) of the true value. Being fast and simple, this method could be useful for estimating reporting in real-time in settings where person-to-person transmission is the main driver of the epidemic, and where case investigation is routinely performed as part of surveillance and contact tracing activities.
Journal Article > ResearchFull Text
PLoS Comput Biol. 11 February 2019; Volume 15 (Issue 2); e1006785.; DOI:10.1371/journal.pcbi.1006785
Funk S, Camacho A, Kucharski AJ, Lowe R, Eggo RM, et al.
PLoS Comput Biol. 11 February 2019; Volume 15 (Issue 2); e1006785.; DOI:10.1371/journal.pcbi.1006785
Real-time forecasts based on mathematical models can inform critical decision-making during infectious disease outbreaks. Yet, epidemic forecasts are rarely evaluated during or after the event, and there is little guidance on the best metrics for assessment. Here, we propose an evaluation approach that disentangles different components of forecasting ability using metrics that separately assess the calibration, sharpness and bias of forecasts. This makes it possible to assess not just how close a forecast was to reality but also how well uncertainty has been quantified. We used this approach to analyse the performance of weekly forecasts we generated in real time for Western Area, Sierra Leone, during the 2013-16 Ebola epidemic in West Africa. We investigated a range of forecast model variants based on the model fits generated at the time with a semi-mechanistic model, and found that good probabilistic calibration was achievable at short time horizons of one or two weeks ahead but model predictions were increasingly unreliable at longer forecasting horizons. This suggests that forecasts may have been of good enough quality to inform decision making based on predictions a few weeks ahead of time but not longer, reflecting the high level of uncertainty in the processes driving the trajectory of the epidemic. Comparing forecasts based on the semi-mechanistic model to simpler null models showed that the best semi-mechanistic model variant performed better than the null models with respect to probabilistic calibration, and that this would have been identified from the earliest stages of the outbreak. As forecasts become a routine part of the toolkit in public health, standards for evaluation of performance will be important for assessing quality and improving credibility of mathematical models, and for elucidating difficulties and trade-offs when aiming to make the most useful and reliable forecasts.
Journal Article > ResearchFull Text
PLoS Comput Biol. 16 May 2018; Volume 14 (Issue 5); DOI:10.1371/journal.pcbi.1006127
Pasetto D, Finger F, Camacho A, Grandesso F, Cohuet S, et al.
PLoS Comput Biol. 16 May 2018; Volume 14 (Issue 5); DOI:10.1371/journal.pcbi.1006127
Computational models of cholera transmission can provide objective insights into the course of an ongoing epidemic and aid decision making on allocation of health care resources. However, models are typically designed, calibrated and interpreted post-hoc. Here, we report the efforts of a team from academia, field research and humanitarian organizations to model in near real-time the Haitian cholera outbreak after Hurricane Matthew in October 2016, to assess risk and to quantitatively estimate the efficacy of a then ongoing vaccination campaign. A rainfall-driven, spatially-explicit meta-community model of cholera transmission was coupled to a data assimilation scheme for computing short-term projections of the epidemic in near real-time. The model was used to forecast cholera incidence for the months after the passage of the hurricane (October-December 2016) and to predict the impact of a planned oral cholera vaccination campaign. Our first projection, from October 29 to December 31, predicted the highest incidence in the departments of Grande Anse and Sud, accounting for about 45% of the total cases in Haiti. The projection included a second peak in cholera incidence in early December largely driven by heavy rainfall forecasts, confirming the urgency for rapid intervention. A second projection (from November 12 to December 31) used updated rainfall forecasts to estimate that 835 cases would be averted by vaccinations in Grande Anse (90% Prediction Interval [PI] 476-1284) and 995 in Sud (90% PI 508-2043). The experience gained by this modeling effort shows that state-of-the-art computational modeling and data-assimilation methods can produce informative near real-time projections of cholera incidence. Collaboration among modelers and field epidemiologists is indispensable to gain fast access to field data and to translate model results into operational recommendations for emergency management during an outbreak. Future efforts should thus draw together multi-disciplinary teams to ensure model outputs are appropriately based, interpreted and communicated.