Other > Pre-Print
bioRxiv. 2017 August 18; DOI:10.1101/177451
Funk S, Camacho A, Kucharski AJ, Lowe R, Eggo RM, et al.
bioRxiv. 2017 August 18; DOI:10.1101/177451
Real-time forecasts based on mathematical models can inform critical decision-making during infectious disease outbreaks. Yet, epidemic forecasts are rarely evaluated during or after the event, and there is little guidance on the best metrics for assessment. Here, we propose an evaluation approach that disentangles different components of forecasting ability using metrics that separately assess the calibration, sharpness and unbiasedness of forecasts. This makes it possible to assess not just how close a forecast was to reality but also how well uncertainty has been quantified. We used this approach to analyse the performance of weekly forecasts we generated in real time in Western Area, Sierra Leone, during the 2013–16 Ebola epidemic in West Africa. We investigated a range of forecast model variants based on the model fits generated at the time with a semi-mechanistic model, and found that good probabilistic calibration was achievable at short time horizons of one or two weeks ahead but models were increasingly inaccurate at longer forecasting horizons. This suggests that forecasts may have been of good enough quality to inform decision making requiring predictions a few weeks ahead of time but not longer, reflecting the high level of uncertainty in the processes driving the trajectory of the epidemic. Comparing forecasts based on the semi-mechanistic model to simpler null models showed that the best semi-mechanistic model variant performed better than the null models with respect to probabilistic calibration, and that this would have been identified from the earliest stages of the outbreak. As forecasts become a routine part of the toolkit in public health, standards for evaluation of performance will be important for assessing quality and improving credibility of mathematical models, and for elucidating difficulties and trade-offs when aiming to make the most useful and reliable forecasts.
Journal Article > Short ReportFull Text
BMC Research Notes. 2015 July 4; Volume 8 (Issue 292); DOI:10.1186/s13104-015-1244-3
Checchi F, Funk S, Chandramohan D, Haydon DT, Chappuis F
BMC Research Notes. 2015 July 4; Volume 8 (Issue 292); DOI:10.1186/s13104-015-1244-3
BACKGROUND
The duration of the stages of HAT is an important factor in epidemiological studies and intervention planning. Previously, we published estimates of the duration of the haemo-lymphatic stage 1 and meningo-encephalitic stage 2 of the gambiense form of human African trypanosomiasis (HAT), in the absence of treatment. Here we revise the estimate of stage 2 duration, computed based on data from Uganda and South Sudan, by adjusting observed infection prevalence for incomplete case detection coverage and diagnostic inaccuracy.
FINDINGS
The revised best estimate for the mean duration of stage 2 is 252 days (95% CI 171-399), about half of our initial best estimate, giving a total mean duration of untreated gambiense HAT infection of approximately 2 years and 2 months.
CONCLUSIONS
Our new estimate provides improved information on the transmission dynamics of this neglected tropical disease in Uganda and South Sudan. We stress that there remains considerable variability around the estimated mean values, and that one must be cautious in applying these results to other foci.
The duration of the stages of HAT is an important factor in epidemiological studies and intervention planning. Previously, we published estimates of the duration of the haemo-lymphatic stage 1 and meningo-encephalitic stage 2 of the gambiense form of human African trypanosomiasis (HAT), in the absence of treatment. Here we revise the estimate of stage 2 duration, computed based on data from Uganda and South Sudan, by adjusting observed infection prevalence for incomplete case detection coverage and diagnostic inaccuracy.
FINDINGS
The revised best estimate for the mean duration of stage 2 is 252 days (95% CI 171-399), about half of our initial best estimate, giving a total mean duration of untreated gambiense HAT infection of approximately 2 years and 2 months.
CONCLUSIONS
Our new estimate provides improved information on the transmission dynamics of this neglected tropical disease in Uganda and South Sudan. We stress that there remains considerable variability around the estimated mean values, and that one must be cautious in applying these results to other foci.
Journal Article > ResearchFull Text
medRxiv. 2019 August 17; DOI:10.1101/19003434
Funk S, Takahashi S, Hellewell J, Gadroen K, Carrion Martin AI, et al.
medRxiv. 2019 August 17; DOI:10.1101/19003434
The Katanga region in the Democratic Republic of Congo (DRC) has been struck by repeated epidemics of measles, with large outbreaks occurring in 2010–13 and 2015. In many of the affected health zones, reactive mass vaccination campaigns were conducted in response to the outbreaks. Here, we attempted to determine how effective the vaccination campaigns in 2015 were in curtailing the ongoing outbreak. We further sought to establish whether the risk of large measles outbreaks in different health zones could have been determined in advance to help prioritise areas for vaccination campaign and speed up the response. In doing so, we first attempted to identify factors that could have been used in 2015 to predict in which health zones the greatest outbreaks would occur. Administrative vaccination coverage was not a good predictor of the size of outbreaks in different health zones. Vaccination coverage derived from surveys, on the other hand, appeared to give more reliable estimates of health zones of low vaccination coverage and, consequently, large outbreaks. On a coarser geographical scale, the provinces most affected in 2015 could be predicted from the outbreak sizes in 2010–13. This, combined with the fact that the vast majority of reported cases were in under-5 year olds, would suggest that there are systematic issues of undervaccination. If this was to continue, outbreaks would be expected to continue to occur in the affected health zones at regular intervals, mostly concentrated in under-5 year olds. We further used a model of measles transmission to estimate the impact of the vaccination campaigns, by first fitting a model to the data including the campaigns and then re-running this without vaccination. We estimated the reactive campaigns to have reduced the size of the overall outbreak by approximately 21,000 (IQR: 16,000–27,000; 95% CI: 8300–38,000) cases. There was considerable heterogeneity in the impact of campaigns, with campaigns started earlier after the start of an outbreak being more impactful. Taken together, these findings suggest that while a strong routine vaccination regime remains the most effective means of measles control, it might be possible to improve the effectiveness of reactive campaigns by considering predictive factors to trigger a more targeted vaccination response.
Journal Article > ReviewFull Text
Lancet Infect Dis. 2021 March 1; Volume 21 (Issue 3); e37-e48.; DOI:10.1016/S1473-3099(20)30479-5
Ratnayake R, Finger F, Azman AS, Lantagne D, Funk S, et al.
Lancet Infect Dis. 2021 March 1; Volume 21 (Issue 3); e37-e48.; DOI:10.1016/S1473-3099(20)30479-5
Globally, cholera epidemics continue to challenge disease control. Although mass campaigns covering large populations are commonly used to control cholera, spatial targeting of case households and their radius is emerging as a potentially efficient strategy. We did a Scoping Review to investigate the effectiveness of interventions delivered through case-area targeted intervention, its optimal spatiotemporal scale, and its effectiveness in reducing transmission. 53 articles were retrieved. We found that antibiotic chemoprophylaxis, point-of-use water treatment, and hygiene promotion can rapidly reduce household transmission, and single-dose vaccination can extend the duration of protection within the radius of households. Evidence supports a high-risk spatiotemporal zone of 100 m around case households, for 7 days. Two evaluations separately showed reductions in household transmission when targeting case households, and in size and duration of case clusters when targeting radii. Although case-area targeted intervention shows promise for outbreak control, it is critically dependent on early detection capacity and requires prospective evaluation of intervention packages.
Journal Article > ResearchFull Text
PLoS Comput Biol. 2022 May 23; Volume 18 (Issue 5); e1008800.; DOI: 10.1371/journal.pcbi.1008800
Jarvis CI, Gimma A, Finger F, Morris TP, Thompson JA, et al.
PLoS Comput Biol. 2022 May 23; Volume 18 (Issue 5); e1008800.; DOI: 10.1371/journal.pcbi.1008800
The fraction of cases reported, known as 'reporting', is a key performance indicator in an outbreak response, and an essential factor to consider when modelling epidemics and assessing their impact on populations. Unfortunately, its estimation is inherently difficult, as it relates to the part of an epidemic which is, by definition, not observed. We introduce a simple statistical method for estimating reporting, initially developed for the response to Ebola in Eastern Democratic Republic of the Congo (DRC), 2018-2020. This approach uses transmission chain data typically gathered through case investigation and contact tracing, and uses the proportion of investigated cases with a known, reported infector as a proxy for reporting. Using simulated epidemics, we study how this method performs for different outbreak sizes and reporting levels. Results suggest that our method has low bias, reasonable precision, and despite sub-optimal coverage, usually provides estimates within close range (5-10%) of the true value. Being fast and simple, this method could be useful for estimating reporting in real-time in settings where person-to-person transmission is the main driver of the epidemic, and where case investigation is routinely performed as part of surveillance and contact tracing activities.
Journal Article > ResearchFull Text
Emerg Infect Dis. 2015 March 1; Volume 21 (Issue 3); 393-399.; DOI:10.3201/eid2103.141892
Kucharski AJ, Camacho A, Checchi F, Waldman RJ, Grais RF, et al.
Emerg Infect Dis. 2015 March 1; Volume 21 (Issue 3); 393-399.; DOI:10.3201/eid2103.141892
In some parts of western Africa, Ebola treatment centers (ETCs) have reached capacity. Unless capacity is rapidly scaled up, the chance to avoid a generalized Ebola epidemic will soon diminish. The World Health Organization and partners are considering additional Ebola patient care options, including community care centers (CCCs), small, lightly staffed units that could be used to isolate patients outside the home and get them into care sooner than otherwise possible. Using a transmission model, we evaluated the benefits and risks of introducing CCCs into Sierra Leone's Western Area, where most ETCs are at capacity. We found that use of CCCs could lead to a decline in cases, even if virus transmission occurs between CCC patients and the community. However, to prevent CCC amplification of the epidemic, the risk of Ebola virus-negative persons being exposed to virus within CCCs would have to be offset by a reduction in community transmission resulting from CCC use.
Journal Article > ResearchFull Text
PLoS Comput Biol. 2019 February 11; Volume 15 (Issue 2); e1006785.; DOI:10.1371/journal.pcbi.1006785
Funk S, Camacho A, Kucharski AJ, Lowe R, Eggo RM, et al.
PLoS Comput Biol. 2019 February 11; Volume 15 (Issue 2); e1006785.; DOI:10.1371/journal.pcbi.1006785
Real-time forecasts based on mathematical models can inform critical decision-making during infectious disease outbreaks. Yet, epidemic forecasts are rarely evaluated during or after the event, and there is little guidance on the best metrics for assessment. Here, we propose an evaluation approach that disentangles different components of forecasting ability using metrics that separately assess the calibration, sharpness and bias of forecasts. This makes it possible to assess not just how close a forecast was to reality but also how well uncertainty has been quantified. We used this approach to analyse the performance of weekly forecasts we generated in real time for Western Area, Sierra Leone, during the 2013-16 Ebola epidemic in West Africa. We investigated a range of forecast model variants based on the model fits generated at the time with a semi-mechanistic model, and found that good probabilistic calibration was achievable at short time horizons of one or two weeks ahead but model predictions were increasingly unreliable at longer forecasting horizons. This suggests that forecasts may have been of good enough quality to inform decision making based on predictions a few weeks ahead of time but not longer, reflecting the high level of uncertainty in the processes driving the trajectory of the epidemic. Comparing forecasts based on the semi-mechanistic model to simpler null models showed that the best semi-mechanistic model variant performed better than the null models with respect to probabilistic calibration, and that this would have been identified from the earliest stages of the outbreak. As forecasts become a routine part of the toolkit in public health, standards for evaluation of performance will be important for assessing quality and improving credibility of mathematical models, and for elucidating difficulties and trade-offs when aiming to make the most useful and reliable forecasts.
Journal Article > ResearchFull Text
Proc Natl Acad Sci U S A. 2017 February 13 (Issue 9)
Lau MSY, Dalziel BD, Funk S, McClelland A, Tiffany A, et al.
Proc Natl Acad Sci U S A. 2017 February 13 (Issue 9)
The unprecedented scale of the Ebola outbreak in Western Africa (2014-2015) has prompted an explosion of efforts to understand the transmission dynamics of the virus and to analyze the performance of possible containment strategies. Models have focused primarily on the reproductive numbers of the disease that represent the average number of secondary infections produced by a random infectious individual. However, these population-level estimates may conflate important systematic variation in the number of cases generated by infected individuals, particularly found in spatially localized transmission and superspreading events. Although superspreading features prominently in first-hand narratives of Ebola transmission, its dynamics have not been systematically characterized, hindering refinements of future epidemic predictions and explorations of targeted interventions. We used Bayesian model inference to integrate individual-level spatial information with other epidemiological data of community-based (undetected within clinical-care systems) cases and to explicitly infer distribution of the cases generated by each infected individual. Our results show that superspreaders play a key role in sustaining onward transmission of the epidemic, and they are responsible for a significant proportion ([Formula: see text]61%) of the infections. Our results also suggest age as a key demographic predictor for superspreading. We also show that community-based cases may have progressed more rapidly than those notified within clinical-care systems, and most transmission events occurred in a relatively short distance (with median value of 2.51 km). Our results stress the importance of characterizing superspreading of Ebola, enhance our current understanding of its spatiotemporal dynamics, and highlight the potential importance of targeted control measures.
Journal Article > ResearchFull Text
PLoS Curr. 2015 February 10; DOI:10.1371/currents.outbreaks
Camacho A, Kucharski AJ, Aki-Sawyerr Y, White M, Flasche S, et al.
PLoS Curr. 2015 February 10; DOI:10.1371/currents.outbreaks
Between August and November 2014, the incidence of Ebola virus disease (EVD) rose dramatically in several districts of Sierra Leone. As a result, the number of cases exceeded the capacity of Ebola holding and treatment centres. During December, additional beds were introduced, and incidence declined in many areas. We aimed to measure patterns of transmission in different regions, and evaluate whether bed capacity is now sufficient to meet future demand.
Journal Article > ResearchFull Text
BMC Med. 2019 March 12; Volume 17; DOI:10.1186/s12916-019-1288-7
Finger F, Funk S, White K, Siddqui MR, Edmunds KL, et al.
BMC Med. 2019 March 12; Volume 17; DOI:10.1186/s12916-019-1288-7
Between August and December 2017, more than 625,000 Rohingya from Myanmar fled into Bangladesh, settling in informal makeshift camps in Cox’s Bazar district and joining 212,000 Rohingya already present. In early November, a diphtheria outbreak hit the camps, with 440 reported cases during the first month. A rise in cases during early December led to a collaboration between teams from Médecins sans Frontières—who were running a provisional diphtheria treatment centre—and the London School of Hygiene and Tropical Medicine with the goal to use transmission dynamic models to forecast the potential scale of the outbreak and the resulting resource needs.
Methods
We first adjusted for delays between symptom onset and case presentation using the observed distribution of reporting delays from previously reported cases. We then fit a compartmental transmission model to the adjusted incidence stratified by age group and location. Model forecasts with a lead time of 2 weeks were issued on 12, 20, 26 and 30 December and communicated to decision-makers.
Results
The first forecast estimated that the outbreak would peak on 19 December in Balukhali camp with 303 (95% posterior predictive interval 122–599) cases and would continue to grow in Kutupalong camp, requiring a bed capacity of 316 (95% posterior predictive interval (PPI) 197–499). On 19 December, a total of 54 cases were reported, lower than forecasted. Subsequent forecasts were more accurate: on 20 December, we predicted a total of 912 cases (95% PPI 367–2183) and 136 (95% PPI 55–327) hospitalizations until the end of the year, with 616 cases actually reported during this period.
Conclusions
Real-time modelling enabled feedback of key information about the potential scale of the epidemic, resource needs and mechanisms of transmission to decision-makers at a time when this information was largely unknown. By 20 December, the model generated reliable forecasts and helped support decision-making on operational aspects of the outbreak response, such as hospital bed and staff needs, and with advocacy for control measures. Although modelling is only one component of the evidence base for decision-making in outbreak situations, suitable analysis and forecasting techniques can be used to gain insights into an ongoing outbreak.
Methods
We first adjusted for delays between symptom onset and case presentation using the observed distribution of reporting delays from previously reported cases. We then fit a compartmental transmission model to the adjusted incidence stratified by age group and location. Model forecasts with a lead time of 2 weeks were issued on 12, 20, 26 and 30 December and communicated to decision-makers.
Results
The first forecast estimated that the outbreak would peak on 19 December in Balukhali camp with 303 (95% posterior predictive interval 122–599) cases and would continue to grow in Kutupalong camp, requiring a bed capacity of 316 (95% posterior predictive interval (PPI) 197–499). On 19 December, a total of 54 cases were reported, lower than forecasted. Subsequent forecasts were more accurate: on 20 December, we predicted a total of 912 cases (95% PPI 367–2183) and 136 (95% PPI 55–327) hospitalizations until the end of the year, with 616 cases actually reported during this period.
Conclusions
Real-time modelling enabled feedback of key information about the potential scale of the epidemic, resource needs and mechanisms of transmission to decision-makers at a time when this information was largely unknown. By 20 December, the model generated reliable forecasts and helped support decision-making on operational aspects of the outbreak response, such as hospital bed and staff needs, and with advocacy for control measures. Although modelling is only one component of the evidence base for decision-making in outbreak situations, suitable analysis and forecasting techniques can be used to gain insights into an ongoing outbreak.