Other > Pre-Print
bioRxiv. 2017 August 18; DOI:10.1101/177451
Funk S, Camacho A, Kucharski AJ, Lowe R, Eggo RM, et al.
bioRxiv. 2017 August 18; DOI:10.1101/177451
Real-time forecasts based on mathematical models can inform critical decision-making during infectious disease outbreaks. Yet, epidemic forecasts are rarely evaluated during or after the event, and there is little guidance on the best metrics for assessment. Here, we propose an evaluation approach that disentangles different components of forecasting ability using metrics that separately assess the calibration, sharpness and unbiasedness of forecasts. This makes it possible to assess not just how close a forecast was to reality but also how well uncertainty has been quantified. We used this approach to analyse the performance of weekly forecasts we generated in real time in Western Area, Sierra Leone, during the 2013–16 Ebola epidemic in West Africa. We investigated a range of forecast model variants based on the model fits generated at the time with a semi-mechanistic model, and found that good probabilistic calibration was achievable at short time horizons of one or two weeks ahead but models were increasingly inaccurate at longer forecasting horizons. This suggests that forecasts may have been of good enough quality to inform decision making requiring predictions a few weeks ahead of time but not longer, reflecting the high level of uncertainty in the processes driving the trajectory of the epidemic. Comparing forecasts based on the semi-mechanistic model to simpler null models showed that the best semi-mechanistic model variant performed better than the null models with respect to probabilistic calibration, and that this would have been identified from the earliest stages of the outbreak. As forecasts become a routine part of the toolkit in public health, standards for evaluation of performance will be important for assessing quality and improving credibility of mathematical models, and for elucidating difficulties and trade-offs when aiming to make the most useful and reliable forecasts.
Journal Article > Meta-AnalysisFull Text
Epidemics. 2019 March 2; Volume 27; DOI:10.1016/j.epidem.2019.03.001
Robert A, Camacho A, Edmunds WJ, Baguelin M, Muyembe JJT, et al.
Epidemics. 2019 March 2; Volume 27; DOI:10.1016/j.epidem.2019.03.001
Health care workers (HCW) are at risk of infection during Ebola virus disease outbreaks and therefore may be targeted for vaccination before or during outbreaks. The effect of these strategies depends on the role of HCW in transmission which is understudied. To evaluate the effect of HCW-targeted or community vaccination strategies, we used a transmission model to explore the relative contribution of HCW and the community to transmission. We calibrated the model to data from multiple Ebola outbreaks. We quantified the impact of ahead-of-time HCW-targeted strategies, and reactive HCW and community vaccination. We found that for some outbreaks (we call "type 1″) HCW amplified transmission both to other HCW and the community, and in these outbreaks prophylactic vaccination of HCW decreased outbreak size. Reactive vaccination strategies had little effect because type 1 outbreaks ended quickly. However, in outbreaks with longer time courses ("type 2 outbreaks"), reactive community vaccination decreased the number of cases, with or without prophylactic HCW-targeted vaccination. For both outbreak types, we found that ahead-of-time HCW-targeted strategies had an impact at coverage of 30%. The vaccine strategies tested had a different impact depending on the transmission dynamics and previous control measures. Although we will not know the characteristics of a new outbreak, ahead-of-time HCW-targeted vaccination can decrease the total outbreak size, even at low vaccine coverage.
Journal Article > ResearchFull Text
Lancet Global Health. 2018 May 3; Volume 6 (Issue 6); DOI:10.1016/S2214-109X(18)30230-4
Camacho A, Bouhenia M, Alyusfi R, Alkohlani A, Naji MAM, et al.
Lancet Global Health. 2018 May 3; Volume 6 (Issue 6); DOI:10.1016/S2214-109X(18)30230-4
In war-torn Yemen, reports of confirmed cholera started in late September, 2016. The disease continues to plague Yemen today in what has become the largest documented cholera epidemic of modern times. We aimed to describe the key epidemiological features of this epidemic, including the drivers of cholera transmission during the outbreak.
Journal Article > ResearchFull Text
PLoS Comput Biol. 2019 February 11; Volume 15 (Issue 2); e1006785.; DOI:10.1371/journal.pcbi.1006785
Funk S, Camacho A, Kucharski AJ, Lowe R, Eggo RM, et al.
PLoS Comput Biol. 2019 February 11; Volume 15 (Issue 2); e1006785.; DOI:10.1371/journal.pcbi.1006785
Real-time forecasts based on mathematical models can inform critical decision-making during infectious disease outbreaks. Yet, epidemic forecasts are rarely evaluated during or after the event, and there is little guidance on the best metrics for assessment. Here, we propose an evaluation approach that disentangles different components of forecasting ability using metrics that separately assess the calibration, sharpness and bias of forecasts. This makes it possible to assess not just how close a forecast was to reality but also how well uncertainty has been quantified. We used this approach to analyse the performance of weekly forecasts we generated in real time for Western Area, Sierra Leone, during the 2013-16 Ebola epidemic in West Africa. We investigated a range of forecast model variants based on the model fits generated at the time with a semi-mechanistic model, and found that good probabilistic calibration was achievable at short time horizons of one or two weeks ahead but model predictions were increasingly unreliable at longer forecasting horizons. This suggests that forecasts may have been of good enough quality to inform decision making based on predictions a few weeks ahead of time but not longer, reflecting the high level of uncertainty in the processes driving the trajectory of the epidemic. Comparing forecasts based on the semi-mechanistic model to simpler null models showed that the best semi-mechanistic model variant performed better than the null models with respect to probabilistic calibration, and that this would have been identified from the earliest stages of the outbreak. As forecasts become a routine part of the toolkit in public health, standards for evaluation of performance will be important for assessing quality and improving credibility of mathematical models, and for elucidating difficulties and trade-offs when aiming to make the most useful and reliable forecasts.
Journal Article > CommentaryFull Text
Vaccine. 2019 October 23; Volume 37; DOI:10.1016/j.vaccine.2019.09.038
van Zandvoort K, Checchi F, Diggle E, Eggo RM, Gadroen K, et al.
Vaccine. 2019 October 23; Volume 37; DOI:10.1016/j.vaccine.2019.09.038
Streptococcus pneumoniae is a common human commensal that causes a sizeable part of the overall childhood mortality in low income settings. Populations affected by humanitarian crises are at especially high risk, because a multitude of risk factors that are enhanced during crises increase pneumococcal transmission and disease severity. Pneumococcal conjugate vaccines (PCVs) provide effective protection and have been introduced into the majority of routine childhood immunisation programmes globally, though several barriers have hitherto limited their uptake during humanitarian crises. When PCV coverage cannot be sustained during crises or when PCV has not been part of routine programmes, mass vaccination campaigns offer a quick acting and programmatically feasible bridging solution until services can be restored. However, we currently face a paucity of evidence on which to base the structure of such campaigns. We believe that, now that PCV can be procured at a substantially reduced price through the Humanitarian Mechanism, this lack of information is a remaining hurdle to PCV use in humanitarian crises. Considering the difficulties in conducting research in crises, we propose an evidence generation pathway consisting of primary data collection in combination with mathematical modelling followed by quasi-experimental evaluation of a PCV intervention, which can inform on optimal vaccination strategies that consider age targeting, dosing regimens and impact duration.
Conference Material > Slide Presentation
van Zandvoort K, Bobe M, Hassan AM, Ismail M, Saed M, et al.
MSF Scientific Day International 2023. 2023 June 7; DOI:10.57740/t9ve-m563
Conference Material > Abstract
van Zandvoort K, Bobe M, Buqul A, Ismail M, Saed M, et al.
MSF Scientific Day International 2023. 2023 June 7; DOI:10.57740/cxek-yg24
INTRODUCTION
Despite a likely high burden of disease caused by Streptococcus pneumoniae in humanitarian crises, pneumococcal conjugate vaccines (PCV’s) are rarely used in such settings. Routine immunisation is rarely feasible in crises, and there is little evidence on alternative delivery strategies for PCV. We used modelling to evaluate the effects of different vaccination strategies within humanitarian crisis settings, aiming to identify those which could quickly reduce and sustain low transmission of vaccine serotypes.
METHODS
We conducted a nested carriage and contact survey in a camp for internally displaced people (IDP) in Somaliland to parameterise a transmission model and used it to assess the potential impact and optimal age targeting of PCV campaigns. We extrapolated this model to other representative humanitarian crisis settings: an acute-phase IDP camp, a protracted crisis in a rural setting, and an urban setting with mixed IDP and host communities. For each we explored the impact and efficiency of campaigns with different target age groups and dosing strategies.
ETHICS
This study was approved by the Ethics Review Boards of the London School of Hygiene and Tropical Medicine and the Republic of Somaliland Ministry of Health Development.
RESULTS
We found high prevalence of nasopharyngeal carriage of Streptococcus pneumoniae; 37% (95% confidence interval (CI), 32-42) in all ages, and 76% (95% CI, 70-82) in children <5 years in the Somaliland IDP camp. 53% (95% CI, 45-61) of serotypes are included in the PCV13 vaccine. People had, on average, 9 (9-10) contacts per day, with high mixing rates between children and intergenerational contacts in older age groups. Our model projects that, for the Somaliland IDP camp, a single PCV campaign including children <5 years can temporarily establish substantial herd protection, averting 37% (95% credible interval (CrI) 24-48) of invasive pneumococcal disease cases in the 2 years following the campaign. Extending age eligibility to children up to 10 or 15 years old could further increase this impact by 49% (95% CrI, 39-50) and 53% (95% CrI, 40-64) respectively. Increased migration rates and close contact with unvaccinated host populations reduces the impact. These factors might require wider age targeting and more frequent repeat campaigns until routine services could be re-established.
CONCLUSION
We show that PCV campaigns could be an effective option to reduce the burden of pneumococcal disease in humanitarian crises until routine immunisation can be implemented. Our results are based on modelled estimates, intervention studies are needed to evaluate their feasibility and effectiveness in real settings.
CONFLICTS OF INTEREST
None declared
Despite a likely high burden of disease caused by Streptococcus pneumoniae in humanitarian crises, pneumococcal conjugate vaccines (PCV’s) are rarely used in such settings. Routine immunisation is rarely feasible in crises, and there is little evidence on alternative delivery strategies for PCV. We used modelling to evaluate the effects of different vaccination strategies within humanitarian crisis settings, aiming to identify those which could quickly reduce and sustain low transmission of vaccine serotypes.
METHODS
We conducted a nested carriage and contact survey in a camp for internally displaced people (IDP) in Somaliland to parameterise a transmission model and used it to assess the potential impact and optimal age targeting of PCV campaigns. We extrapolated this model to other representative humanitarian crisis settings: an acute-phase IDP camp, a protracted crisis in a rural setting, and an urban setting with mixed IDP and host communities. For each we explored the impact and efficiency of campaigns with different target age groups and dosing strategies.
ETHICS
This study was approved by the Ethics Review Boards of the London School of Hygiene and Tropical Medicine and the Republic of Somaliland Ministry of Health Development.
RESULTS
We found high prevalence of nasopharyngeal carriage of Streptococcus pneumoniae; 37% (95% confidence interval (CI), 32-42) in all ages, and 76% (95% CI, 70-82) in children <5 years in the Somaliland IDP camp. 53% (95% CI, 45-61) of serotypes are included in the PCV13 vaccine. People had, on average, 9 (9-10) contacts per day, with high mixing rates between children and intergenerational contacts in older age groups. Our model projects that, for the Somaliland IDP camp, a single PCV campaign including children <5 years can temporarily establish substantial herd protection, averting 37% (95% credible interval (CrI) 24-48) of invasive pneumococcal disease cases in the 2 years following the campaign. Extending age eligibility to children up to 10 or 15 years old could further increase this impact by 49% (95% CrI, 39-50) and 53% (95% CrI, 40-64) respectively. Increased migration rates and close contact with unvaccinated host populations reduces the impact. These factors might require wider age targeting and more frequent repeat campaigns until routine services could be re-established.
CONCLUSION
We show that PCV campaigns could be an effective option to reduce the burden of pneumococcal disease in humanitarian crises until routine immunisation can be implemented. Our results are based on modelled estimates, intervention studies are needed to evaluate their feasibility and effectiveness in real settings.
CONFLICTS OF INTEREST
None declared
Journal Article > ResearchFull Text
Philos Trans R Soc Lond B Biol Sci. 2017 April 10; Volume 372 (Issue 1721); 20160302.; DOI:10.1098/rstb.2016.0302
Funk S, Ciglenecki I, Tiffany A, Gignoux EM, Camacho A, et al.
Philos Trans R Soc Lond B Biol Sci. 2017 April 10; Volume 372 (Issue 1721); 20160302.; DOI:10.1098/rstb.2016.0302
The Ebola epidemic in West Africa was stopped by an enormous concerted effort of local communities and national and international organizations. It is not clear, however, how much the public health response and behavioural changes in affected communities, respectively, contributed to ending the outbreak. Here, we analyse the epidemic in Lofa County, Liberia, lasting from March to November 2014, by reporting a comprehensive time line of events and estimating the time-varying transmission intensity using a mathematical model of Ebola transmission. Model fits to the epidemic show an alternation of peaks and troughs in transmission, consistent with highly heterogeneous spread. This is combined with an overall decline in the reproduction number of Ebola transmission from early August, coinciding with an expansion of the local Ebola treatment centre. We estimate that healthcare seeking approximately doubled over the course of the outbreak, and that isolation of those seeking healthcare reduced their reproduction number by 62% (mean estimate, 95% credible interval (CI) 59-66). Both expansion of bed availability and improved healthcare seeking contributed to ending the epidemic, highlighting the importance of community engagement alongside clinical intervention.This article is part of the themed issue 'The 2013-2016 West African Ebola epidemic: data, decision-making and disease control'.