Assessment of routine childhood immunization data quality, Bono Region, Ghana 2023

Emmanuel George Bachan¹, Jane Addae-Kyeremeh¹, Georgia Ghartey^2,&, Eric Kofi Nyarko¹, Prince Quarshie¹, Kofi Amo-Kodieh¹, Franklin Asiedu-Bekoe³

 

¹Bono Regional Health Directorate, Ghana Health Service, Sunyani, Ghana, ²Ghana Field Epidemiological and Laboratory Training Program, School of Public Health, University of Ghana, Legon, Ghana, ³Public Health Division, Ghana Health Service, Accra, Ghana

 

 

^&Corresponding author
Georgia Ghartey, Ghana Field Epidemiological and Laboratory Training Program, School of Public Health, University of Ghana, Legon, Ghana

 

 

 

 

Data quality is the degree to which a dataset satisfies the user's needs [1]. Achieving high data quality requires constant assessment and reassessment in an iterative fashion while correcting anomalies or inaccuracies, cross-checking information from numerous sources, and performing frequent audits.[2] The quality of data collected from routine public health services, such as childhood immunization programs, is essential for assessing the effectiveness of services, making well-informed decisions, tracking program performance and identifying problem areas [3].

 

The Immunization Agenda 2030 highlights data as one of its four core principles and entreats the alignment of activities of all partners within and outside of the immunization community to ensure fit-for-purpose data for decision-making at all levels [4].

 

Poor immunization data quality may lead to misrepresentation of vaccination coverage, misinformed policy decisions, unobserved outbreaks, disproportionate allocation of resources, and increased financial loss to the state. [5]. To mitigate this, Ghana continues to invest in training health workers, supportive supervision, provision of data collection tools, monthly data validation and quarterly data quality audits. Over the years, community health workers, field technicians, midwives, disease control officers, medical records officers and health information officers have been involved in regular data entry training, periodic data validation, and review meetings aimed at minimizing reporting errors.

 

Global immunization rates decreased from 83.0% in 2020 to 81.0% in 2021, the lowest in the decade. In Africa, immunization coverage for many vaccine-preventable diseases fell below the continent´s 90‒95% target for eradication of these diseases. In 2022, Ghana recorded a 9% increase in the number of districts achieving more than 90% coverage (64%) for Penta3 from the previous proportion of 55% recorded by the country in 2020. The Bono region recorded a 12.5% increase in Penta3 coverage between 2020 and 2022 [6].

 

Although immunization data for the region is available and seems to provide a picture of the situation in the region, due to poor data capture and reporting practices, inadequate skilled personnel and inadequate reporting logistics, administrative data is rather prone to manipulation or falsification and tends to be of poorer quality, as compared to survey data. There has been no recent assessment of the quality of the immunization data to deduce the reliability and usability of the data for taking targeted actions in the region. Additionally, although the District Health Information Management System (DHIMS) has Standard Operating Procedures (SOPs) for assessing data quality, the system focuses on collecting monthly data for completeness and timeliness, with little to no data quality assessments conducted on the data´s accuracy, consistency and factors contributing to it. The Ghana Health Service (GHS) Health Information Management System´s SOPs recommend that data quality verification (including accuracy and consistency) be conducted at least every quarter to aid in public health decision-making and vaccination program implementation [7]. Failure to do this could primarily have an impact on program planning, disproportionate financial and human resource allocation, misdirection of intervention efforts, inefficient use of resources and indirectly complicate efforts to protect vulnerable populations from vaccine-preventable diseases. Therefore, this study assessed the accuracy and consistency of routine childhood immunization data collected in the Bono region from January 2023 to March 2023 (ie. quarter one of 2023).

 

 

Evaluation design

 

We conducted a descriptive cross-sectional study to evaluate the quality of routine childhood immunization data collected from January 2023 to March 2023 in 36 health facilities in the Bono region.

 

Evaluation setting

 

The Bono region is one of the sixteen administrative regions of Ghana, serving as a business and economic centre for neighbouring regions including Ahafo and Bono East, resulting in high immigration rates. The region shares boundaries with Savannah, Bono East, Ahafo, Western North locally and La Cote d'Ivoire internationally and consists of 12 districts. The population of the region is 1,259,945 people and 4% (233,089/1,259,945) of this population are children under 5 years and at risk of childhood vaccine-preventable diseases [8]. The region has one regional hospital, 19 hospitals, 47 clinics, 62 health centres, 21 maternity homes, 309 community-based health planning services centres and approximately 20 alternative health service centres. As of June 2023, 382 health facilities, including CHPS zones, health centres and hospitals, were providing immunization services in the region with a few private and alternative healthcare facilities not providing immunization services.

 

Data source

 

Data for the study were abstracted from DHIMS2, hardcopy vaccination reports and tally sheets in public health facilities.

 

Variables

 

The number of children vaccinated against BCG, IPV, Penta 3, Measles-Rubella 2, and RTS´S 3 was extracted from DHIMS2, the hardcopy monthly vaccination report and the EPI Tally sheet.

 

Sampling

 

We categorized health facilities as high, moderate and low based on their vaccination schedule workload. A high vaccination schedule workload health facility included facilities with more than one outreach per day. A moderate-load health facility was a health centre or CHPS zone with one outreach in a day, and a low-load health facility included a CHPS zone with one outreach in a week. Based on the resources available, 3 health facilities were randomly selected from each of the 12 districts of the region. These 3 facilities included one high vaccination schedule, one moderate-load health facility and one low-load health facility. A total of 36 public health facilities (3 from each of the 12 districts) in the region providing vaccination services from January 2023 to March 2023 were included in the study. One community health officer was randomly selected and interviewed from each of the selected facilities.

 

Operational definition

 

In this evaluation, we defined data accuracy as the measured comparison between EPI tally booklets and abstracted DHIMS 2 data as adapted from the WHO data quality review guidelines [9]. To obtain this, the EPI tally sheet figure was divided by the DHIMS 2 figure and a result of 1 indicated accurate data, <1 indicated underreported data and >1 was overreported. For consistency, the hardcopy monthly vaccination report figure was compared with the abstracted DHIMS 2 figure to confirm whether the data matched. Where it matched, it was termed consistent and inconsistent when it did not match.

 

Data collection

 

We adapted the WHO data review tool to design a data collection tool to compare primary and secondary source immunization data from health facilities from January 2023 to March 2023 to evaluate data accuracy and consistency [2]. We collected monthly immunization data on BCG, IPV, Penta 3, Measles-Rubella 2, and RTS´S 3 from the selected health facilities. Health staff providing immunization services were interviewed as key informants to elicit reasons for data quality issues identified in health facilities following the CDC's Updated Guidelines for Evaluating Public Health Surveillance Systems [9].

 

Data analysis

 

Quantitative data were entered into Microsoft Excel 365 (version 2306) and cleaned. Data analysis was performed using the guidelines from the adapted WHO data quality review protocol. Descriptive data analysis was performed on retrieved data in terms of accurate reporting (accurate, underreporting, overreporting), consistency (consistent and inconsistent reporting) by place (district and health facility), by vaccine and time (January 2023 to March 2023). It was summarized into frequencies and proportions and presented in charts and maps. Reasons for poor data quality were analyzed thematically.

 

Health facility performance in terms of data accuracy and consistency was compared to a target of 100% which is the requirement per the SOP.

 

Ethical considerations

 

The study was carried out as part of a routine administrative process by the Bono Regional Health Directorate in fulfilment of Ghana Health Service´s mandatory requirements and Standard Operating Procedures for improving data quality hence ethical review or approval was not deemed necessary.

 

Administrative approval to review records and interview health staff involved in providing immunization services was obtained from the Bono Regional Health Directorate, districts, subdistricts and health facilities. Reviewed data from health facilities were only used for the evaluation of routine childhood immunization data quality and not for any other purpose. The data obtained were stored on a password-protected computer to prevent unauthorized access. Confidentiality and anonymity for all health staff interviewed were ensured, and their consent was sought.

 

 

We reviewed 108 monthly vaccination reports for 6 selected vaccines from 36 public health facilities for the period of January 2023 to March 2023, and 36 community health officers from these facilities were also interviewed.

 

The study revealed that 30.6% (11/36) of the assessed health facilities reported accurate data and 61.1% (22/36) reported consistent data (Figure 1). For all 6 vaccines assessed in the 36 health facilities of the region from January 2023 to March 2023, we observed a 57.0% mean immunization data accuracy (ranging from 41.0% to 69.0%) and 81.5% mean immunization data consistency (ranging from 72.0% to 97.0%). Mean underreporting and overreporting stood at 24.0% and 19.0% respectively, while mean data inconsistency was 18.5%.

 

Up to 36.1% (13/36) of health facilities assessed reported less than 50% data accuracy and 33.3% (12/36) were within 50-99% data accuracy. More than half of health facilities, 61.1% (22/36), reported 100% consistent data, 11.1% (4/36) of health facilities assessed reported less than 50% data consistency and 27.8% (10/36) were within 50-99% data consistency (Figure 1).

 

BCG was the most accurate (69.0%) and consistent (97.0%) vaccine reported among the six vaccines assessed. IPV and MR2 had the least accurate and least consistent vaccination data respectively (Figure 2).

 

More than half of the districts 58.3% (7/12) had health facilities reporting 50-99% data accuracy, and 41.7% of districts (5/12) had less than 50% data accuracy reporting (Figure 3A). Additionally, 75% (9/12) had health facilities reporting 50-99% and 25% (3/12) of districts reported 100% data consistency (Figure 3B).

 

Immunization data were most accurate (82.3%) and consistent (85.2%) in February 2023 and least consistent (74.9%) in March 2023 (Figure 4).

 

Immunization data quality issues reported by Community Health Officers

 

Thirty-six community health officers interviewed reported on data quality issues in the region as mentioned in the following themes below.

 

Availability and use of standardized data collection tools

 

The main concern reported concerning data inaccuracy (underreporting and overreporting) was inadequate prints of the standard EPI tally booklets for collecting data, which led to the use of improvised tally sheets that did not completely capture all indicators when reporting. This was further compounded by a lack of weekly and daily totals in the EPI Tally Booklet and not using the standard EPI tally booklet in the health facilities.

 

Adherence to good documentation practices

 

It was also found that health staff in some instances were not adding their improvised book tallied figures used at static to outreach level tallied figures, hence the underreporting and overreporting.

 

Inconsistent data were reported in some health facilities due to a lack of post-data entry crosschecking to ensure that hardcopy immunization report figures were entered correctly into DHIMS and vice versa. Other reasons included changes made at the district level during data validation and entry in consultation with the health facility during data validation, but the copy of the health facility report remains unchanged (ie. dose not indicated the validated figures).

 

Workload

 

Some rural health facilities had inadequate health staff compared to urban health facilities, which led to a high workload and reduced motivation to double-check for consistency and accuracy of data from static outreach sessions.

 

 

Our study revealed that the average immunization data accuracy for all six vaccines assessed in the 36 public health facilities of the region assessed was only 57%. Inaccurate childhood immunization data due to underreporting and overreporting stood at more than 10%. This suggests poor data accuracy as this is far from the expected 100% data accuracy per the WHO data quality review guidelines[2] and hence the need for improvement of data accuracy in the region. These rates are similar to those reported in studies conducted in Ethiopia in 2021 and Ghana in 2020, but are lower than those reported in a study conducted in China [10, 11]

 

On the other hand, the study reported more than half of health facilities, had 100% consistent data for all 6 vaccines assessed. This could be due to the high level of uniformity and similarity in the DHIMS2 database figures and hardcopy monthly vaccination report figures across the health facilities. This result is the opposite in a study conducted n Kano state of Nigeria, where 18.5% of health facilities had good data consistency in the number of doses given to children for all three of the selected vaccines assessed [12].

 

Our evaluation found that data reported on the BCG vaccine were the most accurate compared to other vaccines. This could be due to the administration strategy used by the region where all newborns are booked and given the vaccine on particular days to avoid wastage of the vaccine. This was contrary to a study conducted in Enugu Nigeria in 2019, where BCG was said to be underreported in all 103 health facilities studied for all months of 2020 due to termination of bookings in some parts of Nigeria, inconsistent record-keeping, resource constraints, and inadequate training [13].

 

Overreporting and underreporting of immunization data were found in the above similar studies. In all these studies, poor immunization documentation practices and data transcription from the source document to reporting forms and entry into the database were the major factors contributing to the inaccuracy of immunization data.

 

These findings hold significant implications for the region's immunization data quality efforts toward improving immunization data accuracy in healthcare facilities. Given the findings discussed, it is crucial that policy interventions target the transition of the use of paper data collection to electronic data collection tools, to reduce errors in paper data entry and strengthen data validation and feedback mechanisms, which is critical for effective health planning and immunization coverage monitoring. There is also the need for policy enhancements that target the mandatory regularization of data quality audits in the running of data management systems. Again, there is a need for policies that promote human resource capacity building through enhanced and regular staff training on immunization data management.

 

Study limitations

 

The study's reliance on retrospective data might introduce measurement bias, as changes in reporting practices over time could impact the accuracy and consistency of measurements. To minimize this, we compared data from different sources, which helped to identify any inconsistencies and inaccuracies in reporting practices and assess their potential impact on the study's conclusions. Further research covering a longer period would be valuable in exploring accuracy and consistency across different periods.

 

 

The accuracy of routine immunization data in the Bono region for 2023 quarter one (January 2023 to March 2023) was low but moderately consistent. This was facilitated by inadequately trained health workers, unsustainable data validation practices and inadequate follow-up on the implementation of monthly data validation recommendations. Health facility staff involved in immunization were supported on the job by providing corrective actions on immunization data quality issues identified and feedback provided to district health teams to follow up and ensure sustainable data quality practice.

 

We recommend that the National EPI Unit should supply adequate revised standard EPI tally booklets with totals rows included to health facilities; regional and health district teams should conduct training on proper data collection techniques, intensifying supervision, preventing underreporting and overreporting, implementing data validation processes, exploring digital tools for data entry, conducting regular data quality audits, and addressing staffing issues in rural facilities, all aimed at enhancing the effectiveness of immunization programs and informed decision-making for public health.

 

 

The authors declare no competing interest.

 

 

Emmanuel George Bachan, Jane Addae-Kyeremeh, Eric Kofi Nyarko, Prince Quarshie and Kofi Amo-Kodieh conceived, designed and coordinated the data collection of the study. Emmanuel George Bachan, Eric Kofi Nyarko, Jane Addae-Kyeremeh, and Georgia Ghartey, analyzed the data; Emmanuel George Bachan, Georgia Ghartey, Eric Kofi Nyarko, Jane Addae-Kyeremeh, Prince Quarshie and Kofi Amo-Kodieh reviewed the manuscript critically for important intellectual content. All the authors have read and approved the final manuscript.

 

 

We are grateful to Dr Franklin Asiedu-Bekoe-Director Public Health, Professor Ernest Kenu-School of Public Health, University of Ghana, Paul Bediako, Enock Dabie, and Mark Adjei Acheampong of Bono regional health directorate, all Bono regional district health directorates and community health officers interviewed, for their contributions in helping us carry out the study successfully.

 

 

Figure 1: Accuracy and consistency of routine childhood immunization data reported by health facilities- Bono region, January 2023 to March 2023

Figure 2: Accuracy and consistency of routine childhood immunization data by type of vaccine Bono region, January 2023 to March 2023

Figure 3: Childhood immunization data accuracy and consistency by districts, Bono region, January 2023 to March 2023.

Figure 4: Accuracy and consistency of routine childhood immunization data reported in health facilities by month, Bono region, January 2023 to March 2023

 

 

1. Ehsani-Moghaddam B, Martin K, Queenan JA. Data quality in healthcare: A report of practical experience with the Canadian Primary Care Sentinel Surveillance Network data. HIM J [Internet]. 2019 Dec 5 [cited 2025 Feb 4];50(1-2):88-92. https://doi.org/10.1177/1833358319887743 Subscription or purchase required to view full text. PubMed | Google Scholar


2. World Health Organization.Data quality review: a toolkit for facility data quality assessment: Module 1: Framework and metrics [Internet]. Geneva (Switzerland): World Health Organization; 2017 [cited 2025 Feb 4]. 30 p. Download FINAL_Module 1_Framework and Metrics_17129.pdf. Google Scholar


3. Bloland P, MacNeil A.Defining & assessing the quality, usability, and utilization of immunization data . BMC Public Health [Internet]. 2019 Apr 4 [cited 2025 Feb 20];19(1):380. https://doi.org/10.1186/s12889-019-6709-1 . PubMed | Google Scholar


4. World Health Organization. Immunization agenda 2030: A global strategy to leave no one behind [Internet]. Geneva (Switzerland): World Health Organization; 2020 Apr 1 [cited 2025 Feb 4]. 58 p. Download ia2030-document-en.pdf. Google Scholar


5. Scobie HM, Edelstein M, Nicol E, Morice A, Rahimi N, MacDonald NE, Danovaro-Holliday CM, Jawad J, SAGE Working Group on Immunization and Surveillance Data Quality and Use.Improving the quality and use of immunization and surveillance data: Summary report of the Working Group of the Strategic Advisory Group of Experts on Immunization . Vaccine [Internet]. 2020 Sep 17[Version of Record 2020 Oct 13; cited 2025 Feb 4];38(46):7183-97. https://doi.org/10.1016/j.vaccine.2020.09.017 . PubMed | Google Scholar


6. DHIMS 2 [Internet]. Accra (Ghana): Republic of Ghana, Ministry of Health. [date unknown]- [cited 2025 Feb 4]. Credentials required to access database.


7. Ghana Health Service. Health Information Management System Standard Operating Procedures. 4th ed. Accra (Ghana): Ghana Health Service; c2020. 417 p. Google Scholar


8. Ghana Health Service. Profile- Bono Region [Internet]. Accra (Ghana): Ghana Health Service; c2024 [cited 2025 Feb 4].


9. German RR, Lee LM, Horan JM, Milstein RL, Pertowski CA, Waller MN; Guidelines Working Group Centers for Disease Control and Prevention (CDC). Updated guidelines for evaluating public health surveillance systems: recommendations from the Guidelines Working Group. MMWR Recomm Rep [Internet]. 2001 Jul 27 [cited 2025 Feb 4];50(RR-13):1-35; quiz CE1-7. PubMed | Google Scholar


10. Hu Y, Zhang X, Li Q, Chen Y.Auditing the Immunization Data Quality from Routine Reports in Shangyu District, East China. IJERPH [Internet]. 2016 Nov 18 [cited 2025 Feb 4] ;13(11):1158. https://doi.org/10.3390/ijerph13111158 . PubMed | Google Scholar


11. Ziema SA, Asem L. Assessment of immunization data quality of routine reports in Ho municipality of Volta region, Ghana. BMC Health Serv Res [Internet]. 2020 Nov 4 [cited 2025 Feb 4];20(1):1013. https://doi.org/10.1186/s12913-020-05865-4 . PubMed | Google Scholar


12. Akerele A, Obansa R, Adegoke OJ, Haladu S, Adeoye OB, Usifoh N, Etamesor S, Uba B, Waziri NE. Determinants of improved data consistency across routine immunization data tools for health facilities in Kano State, Nigeria. PAMJ [Internet]. 2020 Jul 2 [cited 2025 Feb 4]; 36:148. https://doi.org/10.11604/pamj.2020.36.148.20498 . PubMed | Google Scholar


13. Ugwu GO, Bisi-Onyemaechi AI, Uche EG, Odii A, Nympha EO, Joy UI, Charity O, Chinyere OC, Uzochukwu B, Obinna EO. Childhood routine immunization data in Enugu: findings from a quality assessment survey. Nigerian Journal of Clinical Practice [Internet]. 2023 Jun 26 [cited 2025 Feb 4];26(Suppl 1):S12-8. https://doi.org/10.4103/njcp.njcp_546_22 . PubMed | Google Scholar