Conference Material > Slide Presentation
Guglielmetti L, Khan U, Velasquez GE, Gouillou M, Lachenal N, et al.
MSF Scientific Day International 2024. 2024 May 16; DOI:10.57740/HWpBuX
Conference Material > Abstract
Ahortor E, Mahazu S, Manful T, Erber A, Ablordey A
MSF Scientific Day International 2024. 2024 May 16; DOI:10.57740/MAt4h7
INTRODUCTION
Buruli ulcer caused by Mycobacterium ulcerans is a devastating necrotic skin disease. PCR, recommended for confirmation of Buruli ulcer by WHO, requires an adequately equipped laboratory, often delaying diagnosis and treatment of patients in remote or humanitarian settings. We aimed to assess loop- mediated isothermal amplification (LAMP), which is a molecular assay for isothermal amplification of DNA suggested for timely diagnosis of Buruli ulcer in low-resource settings.
METHODS
This study combines quantitative and qualitative methods. First, we evaluated a simple rapid syringe DNA extraction method (SM) in comparison with a conventional extraction method (CM), followed by a LAMP assay targeting IS2404 for the detection of M ulcerans, either using a pocket warmer (pw) or a heat block (hb) for incubation of the reaction. 83 clinical specimens (swabs and fine-needle aspirates from different centres in Ghana) were tested. We assessed sensitivity, specificity, and positive and negative predictive value (PPV and NPV). Second, we explored the diagnostic workflow for Buruli ulcer at a community-based health centre in rural Ghana, a potential target setting. We used observations and interviews with researchers and healthcare workers (HCWs) and community-based surveillance volunteers. We discuss evaluation results in relation to the target setting and requirements of a target product profile for Buruli ulcer diagnosis.
RESULTS
DNA extraction using SM followed by IS2404 PCR (IS2404 PCRSM) identified M ulcerans DNA in 73 of 83 clinical specimens. The sensitivity, specificity, PPV, and NPV of IS2404 PCRSM were 90.12%, 100%, 100%, and 65.21%, respectively, compared
with the reference standard IS2404 PCR with the CM protocol. Evaluation of the LAMP assay on 64 SM DNA extracts showed a sensitivity, specificity, PPV, and NPV of 83.6%, 100%, 100%, and 50%, respectively, using either pw (pwLAMPSM) or hb (hbLAMPSM) for incubation, compared with the same reference standard. The limit of detection of both pwLAMPSM and hbLAMPSM was 30 target copies. Interviews confirmed that, despite great engagement from HCWs and volunteers, patients met challenges regarding transport and costs for initial diagnosis and follow- up and often sought alternative treatments first. Diagnostic confirmation via PCR in a reference laboratory led to a delay in the initiation of treatment. A diagnosis at the point of care, following clinical screening, was considered advantageous to prevent delays and loss to follow-up, therefore ensuring timely patient treatment.
CONCLUSION
Our findings support the potential use of pwLAMP for rapid diagnosis of Buruli Ulcer in patients with a suspected infection at the community or primary health-care level, with limited equipment and without reliable electricity supply such as found in humanitarian settings.
Buruli ulcer caused by Mycobacterium ulcerans is a devastating necrotic skin disease. PCR, recommended for confirmation of Buruli ulcer by WHO, requires an adequately equipped laboratory, often delaying diagnosis and treatment of patients in remote or humanitarian settings. We aimed to assess loop- mediated isothermal amplification (LAMP), which is a molecular assay for isothermal amplification of DNA suggested for timely diagnosis of Buruli ulcer in low-resource settings.
METHODS
This study combines quantitative and qualitative methods. First, we evaluated a simple rapid syringe DNA extraction method (SM) in comparison with a conventional extraction method (CM), followed by a LAMP assay targeting IS2404 for the detection of M ulcerans, either using a pocket warmer (pw) or a heat block (hb) for incubation of the reaction. 83 clinical specimens (swabs and fine-needle aspirates from different centres in Ghana) were tested. We assessed sensitivity, specificity, and positive and negative predictive value (PPV and NPV). Second, we explored the diagnostic workflow for Buruli ulcer at a community-based health centre in rural Ghana, a potential target setting. We used observations and interviews with researchers and healthcare workers (HCWs) and community-based surveillance volunteers. We discuss evaluation results in relation to the target setting and requirements of a target product profile for Buruli ulcer diagnosis.
RESULTS
DNA extraction using SM followed by IS2404 PCR (IS2404 PCRSM) identified M ulcerans DNA in 73 of 83 clinical specimens. The sensitivity, specificity, PPV, and NPV of IS2404 PCRSM were 90.12%, 100%, 100%, and 65.21%, respectively, compared
with the reference standard IS2404 PCR with the CM protocol. Evaluation of the LAMP assay on 64 SM DNA extracts showed a sensitivity, specificity, PPV, and NPV of 83.6%, 100%, 100%, and 50%, respectively, using either pw (pwLAMPSM) or hb (hbLAMPSM) for incubation, compared with the same reference standard. The limit of detection of both pwLAMPSM and hbLAMPSM was 30 target copies. Interviews confirmed that, despite great engagement from HCWs and volunteers, patients met challenges regarding transport and costs for initial diagnosis and follow- up and often sought alternative treatments first. Diagnostic confirmation via PCR in a reference laboratory led to a delay in the initiation of treatment. A diagnosis at the point of care, following clinical screening, was considered advantageous to prevent delays and loss to follow-up, therefore ensuring timely patient treatment.
CONCLUSION
Our findings support the potential use of pwLAMP for rapid diagnosis of Buruli Ulcer in patients with a suspected infection at the community or primary health-care level, with limited equipment and without reliable electricity supply such as found in humanitarian settings.
Conference Material > Slide Presentation
Wardley T, West KP, Tesfay B, Robinson N, Parry L, et al.
MSF Scientific Day International 2024. 2024 May 16; DOI:10.57740/EQ5OG2MuMi
Conference Material > Slide Presentation
Rapoud D, Cramer E, Al Asmar M, Sagara F, Ndiaye B, et al.
MSF Scientific Day International 2024. 2024 May 16; DOI:10.57740/2acXDPpuix
Conference Material > Abstract
Sterk E, Schramm B, Riccio E, Gabut M, Fontana L, et al.
MSF Scientific Day International 2024. 2024 May 16; DOI:10.57740/Dz2BnS7
INTRODUCTION
The 2014 West Africa Ebola outbreak underlined inadequacies of current personal protective equipment (PPE), such as being uncomfortable and hot, causing excessive sweating and rapid exhaustion, and limiting interactions between health workers and patients. The smartPPE development project responded to the urgent need for a more comfortable, simpler, and sustainable PPE solution for filovirus-outbreak front-line workers. A one- piece, reusable smartPPE with ventilation system was developed to address these challenges. We assessed ease-of-use, comfort, functionality, and perceived doffing-safety of the smartPPE prototype compared with currently used PPE (current-PPE) under simulated field conditions.
METHODS
In June 2023, we conducted a mixed-methods crossover usability study in a controlled high-heat/high-humidity indoor site in Brindisi, Italy. Ten test users (three female, seven with filovirus-front-line experience) assessed smartPPE and current- PPE in four guided sessions covering donning, (emergency) doffing, clinical tasks, and heavy physical WATSAN activities. User feedback was collected through structured questionnaires. Temperature, humidity, session duration, and vital signs were measured, and perceived exertion was assessed using Borg- scores (scale 6–20).
RESULTS
Median temperature and humidity were higher inside current- PPE than inside smartPPE (difference: 2.3°C [IQR 1.8–3.0] and 12.6 percentage points [8.8–19.6], respectively). Users endured heavy work sessions for significantly longer in smartPPE than in current-PPE (80.0 min [IQR 75–84] vs 49.5 min [45–56]). Median increases in body temperature (1.1°C [IQR 0.7–1.6] vs 0.7°C [0.3–0.9]; p<0.001) and respiratory rate (3.5 rpm [1–5] vs 1.5 rpm [0–3]; p=0.034), and reductions in O2 saturation (–2% [–5 to –1] vs –1.5% [–3 to 0]; p=0.027) were higher with current-PPE than with smartPPE. Peripheral vision was similarly rated, but hearing was compromised with smartPPE at ≥5 m. Median exertion- scores were lower for smartPPE (clinical tasks 8.5 [IQR 7–11] vs 15.5 [14–16] p<0.01; heavy physical activities 14 [13–17] vs 18 [17–20] p=0.035). All users preferred smartPPE for overall and thermal comfort, breathing, and doffing-safety; nine (90%) favoured it for non-verbal communication, eight (80%) for vision or longer-interval heavy WATSAN activities, six (60%) for longer- interval patient care, six (60%) for short-term clinical activities, and six (60%) for emergency doffing. Reported concerns were airflow obstruction while bending, hearing difficulties attributed to ventilation noise, and adjustments for headgear, ventilation, and suit fitting.
CONCLUSION
Test users confirmed the usability of smartPPE and favoured it, especially for doffing-safety, longer-interval clinical or physical work, and improved non-verbal interactions, whereas hearing was challenged by the ventilation. Adjustments are currently underway before design freeze. Stakeholder commitment will be crucial to ensure production at scale.
The 2014 West Africa Ebola outbreak underlined inadequacies of current personal protective equipment (PPE), such as being uncomfortable and hot, causing excessive sweating and rapid exhaustion, and limiting interactions between health workers and patients. The smartPPE development project responded to the urgent need for a more comfortable, simpler, and sustainable PPE solution for filovirus-outbreak front-line workers. A one- piece, reusable smartPPE with ventilation system was developed to address these challenges. We assessed ease-of-use, comfort, functionality, and perceived doffing-safety of the smartPPE prototype compared with currently used PPE (current-PPE) under simulated field conditions.
METHODS
In June 2023, we conducted a mixed-methods crossover usability study in a controlled high-heat/high-humidity indoor site in Brindisi, Italy. Ten test users (three female, seven with filovirus-front-line experience) assessed smartPPE and current- PPE in four guided sessions covering donning, (emergency) doffing, clinical tasks, and heavy physical WATSAN activities. User feedback was collected through structured questionnaires. Temperature, humidity, session duration, and vital signs were measured, and perceived exertion was assessed using Borg- scores (scale 6–20).
RESULTS
Median temperature and humidity were higher inside current- PPE than inside smartPPE (difference: 2.3°C [IQR 1.8–3.0] and 12.6 percentage points [8.8–19.6], respectively). Users endured heavy work sessions for significantly longer in smartPPE than in current-PPE (80.0 min [IQR 75–84] vs 49.5 min [45–56]). Median increases in body temperature (1.1°C [IQR 0.7–1.6] vs 0.7°C [0.3–0.9]; p<0.001) and respiratory rate (3.5 rpm [1–5] vs 1.5 rpm [0–3]; p=0.034), and reductions in O2 saturation (–2% [–5 to –1] vs –1.5% [–3 to 0]; p=0.027) were higher with current-PPE than with smartPPE. Peripheral vision was similarly rated, but hearing was compromised with smartPPE at ≥5 m. Median exertion- scores were lower for smartPPE (clinical tasks 8.5 [IQR 7–11] vs 15.5 [14–16] p<0.01; heavy physical activities 14 [13–17] vs 18 [17–20] p=0.035). All users preferred smartPPE for overall and thermal comfort, breathing, and doffing-safety; nine (90%) favoured it for non-verbal communication, eight (80%) for vision or longer-interval heavy WATSAN activities, six (60%) for longer- interval patient care, six (60%) for short-term clinical activities, and six (60%) for emergency doffing. Reported concerns were airflow obstruction while bending, hearing difficulties attributed to ventilation noise, and adjustments for headgear, ventilation, and suit fitting.
CONCLUSION
Test users confirmed the usability of smartPPE and favoured it, especially for doffing-safety, longer-interval clinical or physical work, and improved non-verbal interactions, whereas hearing was challenged by the ventilation. Adjustments are currently underway before design freeze. Stakeholder commitment will be crucial to ensure production at scale.
Conference Material > Slide Presentation
Finger F, Mimbu N, Ratnayake R, Meakin S, Bahati JB, et al.
MSF Scientific Day International 2024. 2024 May 16; DOI:10.57740/tC1av3293
Conference Material > Abstract
Gotham D, Martin M, Barber M, Kazounis E, Batts C, et al.
MSF Scientific Day International 2024. 2024 May 16; DOI:10.57740/aMphKRQ
INTRODUCTION
Clinical trials are a cornerstone of medical innovation. Nonetheless, little information on the cost of conducting clinical trials is available, especially for clinical trials in the global south. This lack of data and transparency hinders the creation of reliable cost estimates and adequate funding of clinical trials in resource- limited settings. Following the recent adoption of the Médecins Sans Frontières (MSF) Clinical Trial Transparency Policy, we present a detailed cost report for TB-PRACTECAL.
METHODS
TB-PRACTECAL was an open-label, phase 2–3, multicentre randomised trial of all-oral regimens for the treatment of drug- resistant tuberculosis. Trial planning began in 2013 and work on publications continued into 2023. The trial took place in six sites across Belarus, South Africa, and Uzbekistan, and enrolled 552 patients. We analysed accounting data for the TB-PRACTECAL project, comprehensively including different costs, presented into 27 categories, by site, and by year, and at the per-patient level.
RESULTS
Total costs for TB-PRACTECAL were €33.9 million, of which 26% were at central level (costs incurred by the UK clinical trial team including trial planning, management, quality assurance, and analysis of results), while 72% were at the trial site level (across all six sites) and 2% were uncategorisable. At trial sites, the largest cost category was staff (43%), followed by external diagnostic services (11%), medicines (9%), other medical consumables (7%), external non-medical services (6%), and transport and travel (6%). Among medicines, the costliest were bedaquiline (46% of medicine costs), linezolid (16%), imipenem/ cilastatin (10%), and delamanid (9%). The mean cost per patient enrolled was €61,460 across the whole trial (including trial management overhead). When only site-level costs were considered, per-patient costs ranged between €19,998 and €45,942 across the six sites.
CONCLUSION
The costs of TB-PRACTECAL were similar to previously reported estimates for comparable clinical trials. However, TB- PRACTECAL included additional costs that would not typically be incurred in a commercial trial, such as investments in clinical research infrastructure and purchase of investigative medical products. To our knowledge, this is the first time MSF, or any other entity, published and analysed the disaggregated costs of a specific clinical trial. These data could help generate reliable predictions for future clinical trials and support planning and involvement, particularly in low-resource settings. Additionally, this study highlights the role of clinical trial cost disclosure in supporting both practical and policy discussions around the development of a more equitable system of biomedical R&D and fairer medicine pricing. Additionally, we developed a financial reporting template to facilitate future reporting of clinical trial cost by MSF and other entities investing in research.
Clinical trials are a cornerstone of medical innovation. Nonetheless, little information on the cost of conducting clinical trials is available, especially for clinical trials in the global south. This lack of data and transparency hinders the creation of reliable cost estimates and adequate funding of clinical trials in resource- limited settings. Following the recent adoption of the Médecins Sans Frontières (MSF) Clinical Trial Transparency Policy, we present a detailed cost report for TB-PRACTECAL.
METHODS
TB-PRACTECAL was an open-label, phase 2–3, multicentre randomised trial of all-oral regimens for the treatment of drug- resistant tuberculosis. Trial planning began in 2013 and work on publications continued into 2023. The trial took place in six sites across Belarus, South Africa, and Uzbekistan, and enrolled 552 patients. We analysed accounting data for the TB-PRACTECAL project, comprehensively including different costs, presented into 27 categories, by site, and by year, and at the per-patient level.
RESULTS
Total costs for TB-PRACTECAL were €33.9 million, of which 26% were at central level (costs incurred by the UK clinical trial team including trial planning, management, quality assurance, and analysis of results), while 72% were at the trial site level (across all six sites) and 2% were uncategorisable. At trial sites, the largest cost category was staff (43%), followed by external diagnostic services (11%), medicines (9%), other medical consumables (7%), external non-medical services (6%), and transport and travel (6%). Among medicines, the costliest were bedaquiline (46% of medicine costs), linezolid (16%), imipenem/ cilastatin (10%), and delamanid (9%). The mean cost per patient enrolled was €61,460 across the whole trial (including trial management overhead). When only site-level costs were considered, per-patient costs ranged between €19,998 and €45,942 across the six sites.
CONCLUSION
The costs of TB-PRACTECAL were similar to previously reported estimates for comparable clinical trials. However, TB- PRACTECAL included additional costs that would not typically be incurred in a commercial trial, such as investments in clinical research infrastructure and purchase of investigative medical products. To our knowledge, this is the first time MSF, or any other entity, published and analysed the disaggregated costs of a specific clinical trial. These data could help generate reliable predictions for future clinical trials and support planning and involvement, particularly in low-resource settings. Additionally, this study highlights the role of clinical trial cost disclosure in supporting both practical and policy discussions around the development of a more equitable system of biomedical R&D and fairer medicine pricing. Additionally, we developed a financial reporting template to facilitate future reporting of clinical trial cost by MSF and other entities investing in research.
Conference Material > Abstract
Rapoud D, Cramer E, Al Asmar M, Sagara F, Ndiaye B, et al.
MSF Scientific Day International 2024. 2024 May 16; DOI:10.57740/rxwuURR8
INTRODUCTION
Antimicrobial resistance (AMR) is a major threat to public health and could cause 10 million deaths per year by 2050. Access to high-quality diagnostic tests is a key intervention to tackle AMR, leading to better patient care, provision of data for global surveillance, and more rational use of antibiotics. Despite technological advances, antimicrobial susceptibility testing (AST) interpretation is complex and requires expert clinical microbiologists, which are lacking in low- and middle-income countries (LMIC). To fill the gap, The Médecins Sans Frontières (MSF) Foundation developed Antibiogo, a smartphone-based application to support laboratory technicians with AST interpretation. We aimed to assess the clinical performance of Antibiogo in intended use settings as per European regulations for in-vitro diagnostic medical devices.
METHODS
Antibiogo combines image processing, machine learning, and expert system technologies for the provision of final results (S/I/R: Susceptible, Intermediate, or Resistant). In 2022, we assessed the clinical performance of Antibiogo according to European regulations in three microbiology laboratories in Jordan (MSF Reconstructive Surgery Hospital, Amman), Mali (MSF Paediatric Hospital, Koutiala), and Senegal (Pasteur Institute, Dakar). In each site, clinical AST performed for routine purposes was processed in parallel with Antibiogo. AST pictures and inhibition zone diameter values measured with Antibiogo were interpreted by an expert microbiologist who was masked to Antibiogo interpretation. We calculated S/I/R category agreement between the microbiologist and Antibiogo, as well as minor (mD), major (MD) and very major discrepancies (VMD).
RESULTS
We included 378 fresh isolates in the study, representing 11 different pathogens. The overall category agreement was 88.8% (95% CI 87.9–89.7), ranging per pathogen from 67.1% (63.2–70.8) (for Pseudomonas aeruginosa) to 98.1% (94.4–99.6) (for Haemophilus influenzae), with 10.2% (9.4–11.1) mD, 1.6% MD (1.2–2.3), and 0.25% VMD (0.08–0.59). From these results, Antibiogo was validated for 11 WHO priority pathogens. From an operational need identified, to proof of concept and evaluation, it became the first MSF CE-marked in-vitro diagnostic (IVD) test in May 2022. As of January 2024, it has been implemented in five MSF laboratories (in Central African Republic, Democratic Republic of the Congo, Jordan, Mali, and Yemen), and in public laboratories in Mali upon request from the Ministry of Health.
CONCLUSION
It will take 400 years to address the shortfall of microbiologists in LMIC at the present rate of training. In the meantime, technology can help fill the gap. In parallel to deployment of Antibiogo in additional countries and regions, developments are ongoing, and an improved version of the app will be released in 2024.
Antimicrobial resistance (AMR) is a major threat to public health and could cause 10 million deaths per year by 2050. Access to high-quality diagnostic tests is a key intervention to tackle AMR, leading to better patient care, provision of data for global surveillance, and more rational use of antibiotics. Despite technological advances, antimicrobial susceptibility testing (AST) interpretation is complex and requires expert clinical microbiologists, which are lacking in low- and middle-income countries (LMIC). To fill the gap, The Médecins Sans Frontières (MSF) Foundation developed Antibiogo, a smartphone-based application to support laboratory technicians with AST interpretation. We aimed to assess the clinical performance of Antibiogo in intended use settings as per European regulations for in-vitro diagnostic medical devices.
METHODS
Antibiogo combines image processing, machine learning, and expert system technologies for the provision of final results (S/I/R: Susceptible, Intermediate, or Resistant). In 2022, we assessed the clinical performance of Antibiogo according to European regulations in three microbiology laboratories in Jordan (MSF Reconstructive Surgery Hospital, Amman), Mali (MSF Paediatric Hospital, Koutiala), and Senegal (Pasteur Institute, Dakar). In each site, clinical AST performed for routine purposes was processed in parallel with Antibiogo. AST pictures and inhibition zone diameter values measured with Antibiogo were interpreted by an expert microbiologist who was masked to Antibiogo interpretation. We calculated S/I/R category agreement between the microbiologist and Antibiogo, as well as minor (mD), major (MD) and very major discrepancies (VMD).
RESULTS
We included 378 fresh isolates in the study, representing 11 different pathogens. The overall category agreement was 88.8% (95% CI 87.9–89.7), ranging per pathogen from 67.1% (63.2–70.8) (for Pseudomonas aeruginosa) to 98.1% (94.4–99.6) (for Haemophilus influenzae), with 10.2% (9.4–11.1) mD, 1.6% MD (1.2–2.3), and 0.25% VMD (0.08–0.59). From these results, Antibiogo was validated for 11 WHO priority pathogens. From an operational need identified, to proof of concept and evaluation, it became the first MSF CE-marked in-vitro diagnostic (IVD) test in May 2022. As of January 2024, it has been implemented in five MSF laboratories (in Central African Republic, Democratic Republic of the Congo, Jordan, Mali, and Yemen), and in public laboratories in Mali upon request from the Ministry of Health.
CONCLUSION
It will take 400 years to address the shortfall of microbiologists in LMIC at the present rate of training. In the meantime, technology can help fill the gap. In parallel to deployment of Antibiogo in additional countries and regions, developments are ongoing, and an improved version of the app will be released in 2024.
Conference Material > Slide Presentation
Wilson JM, Chowdhury F, Hassan S, Harriss E, Alves F, et al.
MSF Scientific Day International 2024. 2024 May 16; DOI:10.57740/R7W2C8dil
Conference Material > Abstract
Wardley T, West KP, Tesfay B, Robinson N, Parry L, et al.
MSF Scientific Day International 2024. 2024 May 16; DOI:10.57740/a4TlzIISm
INTRODUCTION
Climate and environmental conditions are critical factors in malaria transmission. Médecins Sans Frontières (MSF) teams in South Sudan have seen changes in the timing and intensity of malaria seasonal peaks over the past decade. The Malaria Anticipation Project (MAP) aims to develop predictive early warning systems to better predict and act upon any expected rise in malaria cases, through routine surveillance.
METHODS
Predictive models were developed using environmental data collected from climate and space agencies and weekly outpatient department (OPD) malaria case count in Lankien hospital (Jonglei State, South Sudan) as the epidemiological input and output. An ensemble modelling approach was developed using linear regression and extreme gradient boosting (XGBoost) models in a recursive modelling framework. The models were developed using data from 2012–2020, verified with data from 2020–2022, and then monitored in real time in the 2022/23 season. To assess model performance, observed OPD malaria cases were compared with the monthly average cases and classified into categories to assess how often the model prediction was in the same category as the observed number of cases. We
also conducted a qualitative survey to explore community understanding of malaria and its relationship to climate.
RESULTS
During model development, the predictive performance was very high at 2 weeks’ lead time (75% classification accuracy). Model performance remained satisfactory at up to 8 weeks’ lead time (70% classification accuracy), while beyond this, it became increasingly susceptible to large prediction errors. In the 2020/21 and 2021/22 malaria seasons, the predictive performance at 2 weeks’ lead time was good, but it overpredicted for both seasons at 4–8 weeks. The 2022–23 season saw the lowest number of malaria cases of any year in the data used to train the model. The models predicted that the number of cases would be below the long-term average for Lankien hospital, but overpredicted the burden. Across all models, the shorter the lead time of the models, the greater their predictive performance.
CONCLUSION
This modelling approach has the potential to inform anticipatory action within an operationally useful timeframe. Given the models are trained on historical data and cannot include all factors affecting malaria transmission, if relationships between malaria and other conditions change over time, this will impact model performance, demonstrating the limits of forecasting approaches. The next stage of the MAP project will focus on replicability in other settings and pilot implementation to understand operational feasibility and improve performance.
Climate and environmental conditions are critical factors in malaria transmission. Médecins Sans Frontières (MSF) teams in South Sudan have seen changes in the timing and intensity of malaria seasonal peaks over the past decade. The Malaria Anticipation Project (MAP) aims to develop predictive early warning systems to better predict and act upon any expected rise in malaria cases, through routine surveillance.
METHODS
Predictive models were developed using environmental data collected from climate and space agencies and weekly outpatient department (OPD) malaria case count in Lankien hospital (Jonglei State, South Sudan) as the epidemiological input and output. An ensemble modelling approach was developed using linear regression and extreme gradient boosting (XGBoost) models in a recursive modelling framework. The models were developed using data from 2012–2020, verified with data from 2020–2022, and then monitored in real time in the 2022/23 season. To assess model performance, observed OPD malaria cases were compared with the monthly average cases and classified into categories to assess how often the model prediction was in the same category as the observed number of cases. We
also conducted a qualitative survey to explore community understanding of malaria and its relationship to climate.
RESULTS
During model development, the predictive performance was very high at 2 weeks’ lead time (75% classification accuracy). Model performance remained satisfactory at up to 8 weeks’ lead time (70% classification accuracy), while beyond this, it became increasingly susceptible to large prediction errors. In the 2020/21 and 2021/22 malaria seasons, the predictive performance at 2 weeks’ lead time was good, but it overpredicted for both seasons at 4–8 weeks. The 2022–23 season saw the lowest number of malaria cases of any year in the data used to train the model. The models predicted that the number of cases would be below the long-term average for Lankien hospital, but overpredicted the burden. Across all models, the shorter the lead time of the models, the greater their predictive performance.
CONCLUSION
This modelling approach has the potential to inform anticipatory action within an operationally useful timeframe. Given the models are trained on historical data and cannot include all factors affecting malaria transmission, if relationships between malaria and other conditions change over time, this will impact model performance, demonstrating the limits of forecasting approaches. The next stage of the MAP project will focus on replicability in other settings and pilot implementation to understand operational feasibility and improve performance.