The eLMIS & EIR project

Principal investigator: Aleksandra Torbica (CeRGAS) 

Team UB/Cergas: Viviana Mangiaterra, Carlo Federici, Claudio Jommi, Marianna Cavazza, Maria Verykiou, Flaminia Sabbatucci, Erin Ferenchick 

Partners: MMGH Consulting GmbH (MMGH), Africa Health Consulting (Guinea), senior consultants Edith Rodriguez and Luis Castillo (Honduras), the  Centre for Impact Innovation and Capacity Building for Health Information and Nutrition (CIIC-HIN) (Rwanda) and the  National Institute for Medical Research – Mbeya Medical Research Center (NIMR-MMRC) (Tanzania) 

Sponsor: Bill and Melinda Gates Foundation, in collaboration with WHO and Gavi 

Duration: July 2020 - July 2023 

Abstract: 

With the increasing digitalization of health systems in low and middle-income countries (LMICs), there is growing interest from governments, donors and implementing partners to introduce and scale-up digital systems to support more effective and efficient immunization service delivery. Upon the request by the Bill and Melinda Gates Foundation (BMGF), the World Health Organization (WHO) and Gavi, the Vaccine Alliance, a multi-country evaluation was performed with the overall aim of generating robust actionable evidence to enable future decisions on the introduction and scale-up of electronic immunization  registries (eIR) and electronic logistics management information systems (eLMIS) in low- and middle-income countries  (LMICs). The following definitions of eIR and eLMIS were used in the evaluation. 

An eIR is a computerized, confidential population-based system capturing individual-level information on vaccine doses administered. It collects and consolidates vaccination data from vaccination providers for better immunization strategies. Its goal is to facilitate immunization coverage estimation, allow for individualized follow-up on defaulters (persons who are overdue for a vaccine dose), generate information for provider assessment and feedback, and ultimately provide vaccination clinical decision support. An eLMIS is a computerized system used to capture vaccine consumption (both actual and forecasted), stock and ordering data from all levels of the immunization logistics system. Its goal is to inform logistics decisions and manage the supply chain. 

While current evidence suggests that these electronic tools can contribute to improved data quality and use, and ultimately positively impact the functioning of immunization programs, limited evidence exists on their real impact when implemented in a specific setting. In addition, while existing evaluations to date have explored critical factors influencing the success of their implementation, they have offered  little insights into their costing and affordability. Using both quantitative and qualitative data collected from health centres at different administrative levels, as well as secondary available data, in this project, we assessed the programmatic and economic impact of implementing eIR and/or eLMIS systems in four LMICs, Guinea, Honduras, Rwanda and Tanzania 

Main results:

Despite notable differences between an eIR and eLMIS in terms of how they are used and how their impact can be evaluated,  this evaluation suggests that greater use of these electronic tools for decision support through an integrated, cross-programmatic approach could assist in better managing immunization programs, as well as health systems more broadly.

With the elimination of paper processes, use of electronic tools could generate significant savings and improve the use of resources, increasing the likelihood for these interventions to be cost-effective. Investments in promoting the uptake and   use of the electronic systems could be good value for money and may be offset by the savings resulting from such   enhancement. 

However, the inability to meet a set of criteria for success, as well as a lack of clarity on specific goals for the introduction of   these electronic tools, should be seen as a warning that investments might be at risk of not delivering the expected results. This evaluation has highlighted several limitations of the current use of the tools across each country. Unless these shortcomings can be addressed, funding through national or international immunization budgets would not be recommended and other interventions with more direct impact on immunization outcomes should be prioritized.

More research is needed to understand whether removing barriers of implementation and promoting use of these systems to inform decision-making would enable immunization programs and health systems more broadly to achieve improved immunization outcomes.

Given the challenges of evaluating electronic tools, in those countries planning to implement or scale-up such systems, monitoring and evaluation activities should be carefully planned at the onset of implementation, as well as continued once the systems are fully scaled up with all functionalities. This type of adaptative learning will allow for course correction to ensure that the tools are used for maximum impact.  

Dissemination: 

Presentations - Results presented in several webinars and conferences including a Webinar with participants from country representatives and senior officials from various international organizations such as US CDC, UNICEF and PAHO. 

Results were also presented at the biannual conference of the International Health Economics Association (IHEA) in Cape Town, South Africa; the Annual African Vaccinology Course / Vaccines for Africa Initiative; the German Health Economics Association workgroup in Frankfurt, Germany and the 17th TechNet Conference in Panama City, Panama