Other > Pre-Print
bioRxiv. 2017 August 18; DOI:10.1101/177451
Funk S, Camacho A, Kucharski AJ, Lowe R, Eggo RM, et al.
bioRxiv. 2017 August 18; DOI:10.1101/177451
Real-time forecasts based on mathematical models can inform critical decision-making during infectious disease outbreaks. Yet, epidemic forecasts are rarely evaluated during or after the event, and there is little guidance on the best metrics for assessment. Here, we propose an evaluation approach that disentangles different components of forecasting ability using metrics that separately assess the calibration, sharpness and unbiasedness of forecasts. This makes it possible to assess not just how close a forecast was to reality but also how well uncertainty has been quantified. We used this approach to analyse the performance of weekly forecasts we generated in real time in Western Area, Sierra Leone, during the 2013–16 Ebola epidemic in West Africa. We investigated a range of forecast model variants based on the model fits generated at the time with a semi-mechanistic model, and found that good probabilistic calibration was achievable at short time horizons of one or two weeks ahead but models were increasingly inaccurate at longer forecasting horizons. This suggests that forecasts may have been of good enough quality to inform decision making requiring predictions a few weeks ahead of time but not longer, reflecting the high level of uncertainty in the processes driving the trajectory of the epidemic. Comparing forecasts based on the semi-mechanistic model to simpler null models showed that the best semi-mechanistic model variant performed better than the null models with respect to probabilistic calibration, and that this would have been identified from the earliest stages of the outbreak. As forecasts become a routine part of the toolkit in public health, standards for evaluation of performance will be important for assessing quality and improving credibility of mathematical models, and for elucidating difficulties and trade-offs when aiming to make the most useful and reliable forecasts.
Journal Article > ResearchFull Text
BMC Infect Dis. 2018 April 11; Volume 18 (Issue 1); 172.; DOI:10.1186/s12879-018-3073-1
le Polain de Waroux O, Cohuet S, Ndazima D, Kucharski AJ, Juan-Giner A, et al.
BMC Infect Dis. 2018 April 11; Volume 18 (Issue 1); 172.; DOI:10.1186/s12879-018-3073-1
BACKGROUND
Quantification of human interactions relevant to infectious disease transmission through social contact is central to predict disease dynamics, yet data from low-resource settings remain scarce.
METHODS
We undertook a social contact survey in rural Uganda, whereby participants were asked to recall details about the frequency, type, and socio-demographic characteristics of any conversational encounter that lasted for ≥5 min (henceforth defined as 'contacts') during the previous day. An estimate of the number of 'casual contacts' (i.e. < 5 min) was also obtained.
RESULTS
In total, 566 individuals were included in the study. On average participants reported having routine contact with 7.2 individuals (range 1-25). Children aged 5-14 years had the highest frequency of contacts and the elderly (≥65 years) the fewest (P < 0.001). A strong age-assortative pattern was seen, particularly outside the household and increasingly so for contacts occurring further away from home. Adults aged 25-64 years tended to travel more often and further than others, and males travelled more frequently than females.
CONCLUSION
Our study provides detailed information on contact patterns and their spatial characteristics in an African setting. It therefore fills an important knowledge gap that will help more accurately predict transmission dynamics and the impact of control strategies in such areas.
Quantification of human interactions relevant to infectious disease transmission through social contact is central to predict disease dynamics, yet data from low-resource settings remain scarce.
METHODS
We undertook a social contact survey in rural Uganda, whereby participants were asked to recall details about the frequency, type, and socio-demographic characteristics of any conversational encounter that lasted for ≥5 min (henceforth defined as 'contacts') during the previous day. An estimate of the number of 'casual contacts' (i.e. < 5 min) was also obtained.
RESULTS
In total, 566 individuals were included in the study. On average participants reported having routine contact with 7.2 individuals (range 1-25). Children aged 5-14 years had the highest frequency of contacts and the elderly (≥65 years) the fewest (P < 0.001). A strong age-assortative pattern was seen, particularly outside the household and increasingly so for contacts occurring further away from home. Adults aged 25-64 years tended to travel more often and further than others, and males travelled more frequently than females.
CONCLUSION
Our study provides detailed information on contact patterns and their spatial characteristics in an African setting. It therefore fills an important knowledge gap that will help more accurately predict transmission dynamics and the impact of control strategies in such areas.
Journal Article > ResearchFull Text
Epidemics. 2018 December 1; Volume 25; 72-79.; DOI:10.1016/j.epidem.2018.05.008
le Polain de Waroux O, Flasche S, Kucharski AJ, Langendorf C, Ndazima D, et al.
Epidemics. 2018 December 1; Volume 25; 72-79.; DOI:10.1016/j.epidem.2018.05.008
Although patterns of social contacts are believed to be an important determinant of infectious disease transmission, it remains unclear how the frequency and nature of human interactions shape an individual's risk of infection. We analysed data on daily social encounters individually matched to data on S. pneumoniae carriage and acute respiratory symptoms (ARS), from 566 individuals who took part in a survey in South-West Uganda. We found that the frequency of physical (i.e. skin-to-skin), long (≥1 h) and household contacts - which capture some measure of close (i.e. relatively intimate) contact - was higher among pneumococcal carriers than non-carriers, and among people with ARS compared to those without, irrespective of their age. With each additional physical encounter the age-adjusted risk of carriage and ARS increased by 6% (95%CI 2-9%) and 7% (2-13%) respectively. In contrast, the number of casual contacts (<5 min long) was not associated with either pneumococcal carriage or ARS. A detailed analysis by age of contacts showed that the number of close contacts with young children (<5 years) was particularly higher among older children and adult carriers than non-carriers, while the higher number of contacts among people suffering from ARS was more homogeneous across contacts of all ages. Our findings provide key evidence that the frequency of close interpersonal contact is important for transmission of respiratory infections, but not that of casual contacts. Those results are essential for both improving disease prevention and control efforts as well as informing research on infectious disease dynamics and transmission models, and more studies should be undertaken to further validate our results.
Journal Article > ResearchFull Text
Emerg Infect Dis. 2015 March 1; Volume 21 (Issue 3); 393-399.; DOI:10.3201/eid2103.141892
Kucharski AJ, Camacho A, Checchi F, Waldman RJ, Grais RF, et al.
Emerg Infect Dis. 2015 March 1; Volume 21 (Issue 3); 393-399.; DOI:10.3201/eid2103.141892
In some parts of western Africa, Ebola treatment centers (ETCs) have reached capacity. Unless capacity is rapidly scaled up, the chance to avoid a generalized Ebola epidemic will soon diminish. The World Health Organization and partners are considering additional Ebola patient care options, including community care centers (CCCs), small, lightly staffed units that could be used to isolate patients outside the home and get them into care sooner than otherwise possible. Using a transmission model, we evaluated the benefits and risks of introducing CCCs into Sierra Leone's Western Area, where most ETCs are at capacity. We found that use of CCCs could lead to a decline in cases, even if virus transmission occurs between CCC patients and the community. However, to prevent CCC amplification of the epidemic, the risk of Ebola virus-negative persons being exposed to virus within CCCs would have to be offset by a reduction in community transmission resulting from CCC use.
Journal Article > ResearchFull Text
PLoS Comput Biol. 2019 February 11; Volume 15 (Issue 2); e1006785.; DOI:10.1371/journal.pcbi.1006785
Funk S, Camacho A, Kucharski AJ, Lowe R, Eggo RM, et al.
PLoS Comput Biol. 2019 February 11; Volume 15 (Issue 2); e1006785.; DOI:10.1371/journal.pcbi.1006785
Real-time forecasts based on mathematical models can inform critical decision-making during infectious disease outbreaks. Yet, epidemic forecasts are rarely evaluated during or after the event, and there is little guidance on the best metrics for assessment. Here, we propose an evaluation approach that disentangles different components of forecasting ability using metrics that separately assess the calibration, sharpness and bias of forecasts. This makes it possible to assess not just how close a forecast was to reality but also how well uncertainty has been quantified. We used this approach to analyse the performance of weekly forecasts we generated in real time for Western Area, Sierra Leone, during the 2013-16 Ebola epidemic in West Africa. We investigated a range of forecast model variants based on the model fits generated at the time with a semi-mechanistic model, and found that good probabilistic calibration was achievable at short time horizons of one or two weeks ahead but model predictions were increasingly unreliable at longer forecasting horizons. This suggests that forecasts may have been of good enough quality to inform decision making based on predictions a few weeks ahead of time but not longer, reflecting the high level of uncertainty in the processes driving the trajectory of the epidemic. Comparing forecasts based on the semi-mechanistic model to simpler null models showed that the best semi-mechanistic model variant performed better than the null models with respect to probabilistic calibration, and that this would have been identified from the earliest stages of the outbreak. As forecasts become a routine part of the toolkit in public health, standards for evaluation of performance will be important for assessing quality and improving credibility of mathematical models, and for elucidating difficulties and trade-offs when aiming to make the most useful and reliable forecasts.
Journal Article > ResearchFull Text
PLoS Curr. 2015 February 10; DOI:10.1371/currents.outbreaks
Camacho A, Kucharski AJ, Aki-Sawyerr Y, White M, Flasche S, et al.
PLoS Curr. 2015 February 10; DOI:10.1371/currents.outbreaks
Between August and November 2014, the incidence of Ebola virus disease (EVD) rose dramatically in several districts of Sierra Leone. As a result, the number of cases exceeded the capacity of Ebola holding and treatment centres. During December, additional beds were introduced, and incidence declined in many areas. We aimed to measure patterns of transmission in different regions, and evaluate whether bed capacity is now sufficient to meet future demand.
Journal Article > ResearchFull Text
BMC Med. 2019 March 12; Volume 17; DOI:10.1186/s12916-019-1288-7
Finger F, Funk S, White K, Siddqui MR, Edmunds KL, et al.
BMC Med. 2019 March 12; Volume 17; DOI:10.1186/s12916-019-1288-7
Between August and December 2017, more than 625,000 Rohingya from Myanmar fled into Bangladesh, settling in informal makeshift camps in Cox’s Bazar district and joining 212,000 Rohingya already present. In early November, a diphtheria outbreak hit the camps, with 440 reported cases during the first month. A rise in cases during early December led to a collaboration between teams from Médecins sans Frontières—who were running a provisional diphtheria treatment centre—and the London School of Hygiene and Tropical Medicine with the goal to use transmission dynamic models to forecast the potential scale of the outbreak and the resulting resource needs.
Methods
We first adjusted for delays between symptom onset and case presentation using the observed distribution of reporting delays from previously reported cases. We then fit a compartmental transmission model to the adjusted incidence stratified by age group and location. Model forecasts with a lead time of 2 weeks were issued on 12, 20, 26 and 30 December and communicated to decision-makers.
Results
The first forecast estimated that the outbreak would peak on 19 December in Balukhali camp with 303 (95% posterior predictive interval 122–599) cases and would continue to grow in Kutupalong camp, requiring a bed capacity of 316 (95% posterior predictive interval (PPI) 197–499). On 19 December, a total of 54 cases were reported, lower than forecasted. Subsequent forecasts were more accurate: on 20 December, we predicted a total of 912 cases (95% PPI 367–2183) and 136 (95% PPI 55–327) hospitalizations until the end of the year, with 616 cases actually reported during this period.
Conclusions
Real-time modelling enabled feedback of key information about the potential scale of the epidemic, resource needs and mechanisms of transmission to decision-makers at a time when this information was largely unknown. By 20 December, the model generated reliable forecasts and helped support decision-making on operational aspects of the outbreak response, such as hospital bed and staff needs, and with advocacy for control measures. Although modelling is only one component of the evidence base for decision-making in outbreak situations, suitable analysis and forecasting techniques can be used to gain insights into an ongoing outbreak.
Methods
We first adjusted for delays between symptom onset and case presentation using the observed distribution of reporting delays from previously reported cases. We then fit a compartmental transmission model to the adjusted incidence stratified by age group and location. Model forecasts with a lead time of 2 weeks were issued on 12, 20, 26 and 30 December and communicated to decision-makers.
Results
The first forecast estimated that the outbreak would peak on 19 December in Balukhali camp with 303 (95% posterior predictive interval 122–599) cases and would continue to grow in Kutupalong camp, requiring a bed capacity of 316 (95% posterior predictive interval (PPI) 197–499). On 19 December, a total of 54 cases were reported, lower than forecasted. Subsequent forecasts were more accurate: on 20 December, we predicted a total of 912 cases (95% PPI 367–2183) and 136 (95% PPI 55–327) hospitalizations until the end of the year, with 616 cases actually reported during this period.
Conclusions
Real-time modelling enabled feedback of key information about the potential scale of the epidemic, resource needs and mechanisms of transmission to decision-makers at a time when this information was largely unknown. By 20 December, the model generated reliable forecasts and helped support decision-making on operational aspects of the outbreak response, such as hospital bed and staff needs, and with advocacy for control measures. Although modelling is only one component of the evidence base for decision-making in outbreak situations, suitable analysis and forecasting techniques can be used to gain insights into an ongoing outbreak.
Journal Article > ResearchFull Text
Philos Trans R Soc Lond B Biol Sci. 2017 April 10; Volume 372 (Issue 1721); 20160302.; DOI:10.1098/rstb.2016.0302
Funk S, Ciglenecki I, Tiffany A, Gignoux EM, Camacho A, et al.
Philos Trans R Soc Lond B Biol Sci. 2017 April 10; Volume 372 (Issue 1721); 20160302.; DOI:10.1098/rstb.2016.0302
The Ebola epidemic in West Africa was stopped by an enormous concerted effort of local communities and national and international organizations. It is not clear, however, how much the public health response and behavioural changes in affected communities, respectively, contributed to ending the outbreak. Here, we analyse the epidemic in Lofa County, Liberia, lasting from March to November 2014, by reporting a comprehensive time line of events and estimating the time-varying transmission intensity using a mathematical model of Ebola transmission. Model fits to the epidemic show an alternation of peaks and troughs in transmission, consistent with highly heterogeneous spread. This is combined with an overall decline in the reproduction number of Ebola transmission from early August, coinciding with an expansion of the local Ebola treatment centre. We estimate that healthcare seeking approximately doubled over the course of the outbreak, and that isolation of those seeking healthcare reduced their reproduction number by 62% (mean estimate, 95% credible interval (CI) 59-66). Both expansion of bed availability and improved healthcare seeking contributed to ending the epidemic, highlighting the importance of community engagement alongside clinical intervention.This article is part of the themed issue 'The 2013-2016 West African Ebola epidemic: data, decision-making and disease control'.
Journal Article > CommentaryFull Text
Lancet Microbe. 2024 August 1; Volume 5 (Issue 8); 100881.; DOI:10.1016/S2666-5247(24)00104-6
van Hoek AJ, Funk S, Flasche S, Quilty BJ, van Kleef E, et al.
Lancet Microbe. 2024 August 1; Volume 5 (Issue 8); 100881.; DOI:10.1016/S2666-5247(24)00104-6