Evaluation of influenza-like illness sentinel surveillance system, Sunyani Municipality, Ghana

Abdul Gafaru Mohammed^1,2, Christopher Tamal¹, Magdalene Akos Odikro^1,2,&, Delia Bandoh¹, Donne Ameme¹, Ernest Kenu^1,2

 

¹Ghana Field Epidemiology and Laboratory Training Program, School of Public Health, University of Ghana, Accra, Ghana, ²School of Public Health, University of Ghana, Legon, Accra, Ghana

 

 

^&Corresponding author
Magdalene Akos Odikro, School of Public Health, University of Ghana, Legon, Accra, Ghana.

 

 

 

 

Influenza-like illnesses (ILI) are viral respiratory tract diseases often characterized by fever, headache, myalgia, coryza, cough, and sore throat in humans. Persons infected may become infectious before, during or after the onset of clinical manifestations [1]. The causative pathogen for ILI can be transmitted both directly and indirectly through contact with contaminated fomites [2]. According to the World Health Organization (WHO), most influenzas in global circulation are of zoonotic origin. Influenza-like illness morbidity and mortality are highest among adults older than 65 years, children younger than two years, pregnant women, and persons with certain medical conditions such as asthma, kidney disease, heart disease, liver disease, and diabetes [3].

 

Globally, WHO estimated that seasonal flu causes severe disease in about 3-5 million people and may result in 290,000-650,000 deaths annually [4]. The effect of seasonal influenza epidemics on developing countries is not fully understood. Still, research shows that about 99% of deaths in children under five years of age with influenza-related lower respiratory infections occur in developing countries [4]. Despite the progress made in describing the epidemiology and burden of ILI in Sub-Saharan Africa, most countries in the region still lack longitudinal national surveillance data to inform preventive and control strategies.

 

In 2007, Ghana initiated influenza surveillance at sentinel sites for routine monitoring of acute respiratory illness to obtain data on circulating strains [5]. In collaboration with the Ministry of Defense and Ghana Health Service, the National Influenza Centre operates sentinel surveillance in 27 sites across the country [6]. Two of these sites are located in the Sunyani municipality. The ILI surveillance system was established with the objectives; to establish and monitor baseline rates of severe respiratory disease, including tracking the severity and impact of influenza. To describe and monitor vulnerable groups at the highest risk of severe illness and facilitate the detection of antigenic or genetic changes in circulating viruses or the appearance of antiviral resistance. It is recommended that from no more than two years after implementation, influenza surveillance systems should be periodically evaluated to identify steps needed to fill gaps in system performance [7]. We assessed the ILI sentinel surveillance system to determine if it was meeting the objectives for which it was set and to evaluate its usefulness and attributes.

 

 

Evaluation Site

 

The Sunyani municipality is one of Ghana´s 260 Metropolitan, Municipal and District Assemblies (MMDAs). It forms part of the 12 municipalities and districts in the Bono Region with the capital as Sunyani. The projected 2019 population of the municipality was 151,378, with about 18.5% of the total population being children <5 years old [8]. The municipality has six sub-districts; Sunyani Central, Abesim, New Dormaa, AntwiKrom, Penkwase, and New Town Baakoniaba. The municipality falls within the wet Semi-Equatorial Climatic Zone of Ghana. The monthly temperatures vary between 23°C and 33°C with the lowest around August and the highest being observed around March and April. The climatic condition of the Sunyani Municipality has been reported to be a significant contributing parameter that triggers the prevalence of diseases such as malaria, and respiratory tract infections, among others [9]. The municipality has 37 health facilities with two facilities designated as sentinel sites for ILI surveillance; the 3 Medical Reception Station (MRS) and the Sunyani Municipal Hospital.

 

Evaluation Design

 

We conducted a descriptive evaluation of the ILI surveillance system spanning 2015-2019 using a tool based on the Center for Disease Control and Prevention (CDC) updated guidelines for evaluating public health surveillance Systems (2001). Morbidity and mortality data on ILI was extracted from the District Health Information Management System (DHIMS 2) and weekly IDSR reporting forms. The indicators in the updated CDC technical guidelines were used to assess the performance of the system in terms of its attributes, objectives, and usefulness

 

Data collection

 

We interviewed key surveillance officers operating the system at the two sentinel sites using a semi-structured questionnaire. We collected and reviewed data from the sentinel registers, case investigation forms, and the weekly Integrated Disease Surveillance and Response (IDSR). We also extracted ILI morbidity and mortality data for 2015-2019 from the District Health Information Management System 2 (DHIMS 2).

 

Data Analysis

 

The usefulness of the ILI surveillance system was determined by assessing the interventions informed by the system. The system was considered useful if it had informed any intervention within the Sunyani Municipality in the period evaluated.

 

The Moving Epidemic Method (MEM) in determining thresholds for ILI and SARI surveillance systems was adopted in assessing if any outbreak of ILI occurred between 2015 and 2019. MEM is a standardized method of reporting influenza activity adopted by the European Centre for Disease Prevention and Control that allows intra- and intercountry comparisons using historical data. It defines the baseline influenza activity and establishes an epidemic threshold above which the activity parameter is considered to be in an epidemic period. Pre-alert threshold: upper limit of the 40% confidence interval (0.525 SD above the mean). Alert threshold: upper limit of the 90% confidence interval (1.645 SD above the mean) and Epidemic threshold: upper limit of the 97.5% confidence interval (2.2412 SD above the mean).

 

Indicators of the nine attributes based on the CDC´s updated guidelines for evaluating surveillance systems were assigned scores. Each attribute was assessed separately. The indicators as specified for each attribute in the CDC updated guidelines were used in assessing the good or poor performance of the system in terms of that attribute.

 

For each of the attributes, we scored each indicator 1 or 0 points. An indicator is scored 1 if the key finding aligns with the CDC updated guidelines. An indicator is also scored 0 if the key finding is otherwise. Overall, we scored the attribute good performance if the sum of all indicator scores was more than or equal to two-thirds of the total for the attribute. We also scored the attribute poor performance if the sum of all indicator scores was less than or equal to one-third of the total score for the attribute. We gave attributes an overall score of either 2 (poor performance) or 3 (good performance). A score of 3 represents two-thirds or more, and those with one-third or less were represented by 2 [2] (Table 1).

 

Positive predictive value (PPV) was calculated by dividing the number of true positives (TP) by the number of people who tested positive. The PPV ranges from 0% to 100% showing the proportion of cases tested positive which are actually true positives.

 

In assessing the performance of the system in terms of meeting set objectives. The system was assessed for its ability to detect cases of ILI and outbreaks, classify ILI cases by strain, and monitor the occurrence of the disease in vulnerable groups.

 

Availability of materials

 

The materials used in the evaluation are available from the corresponding author and will be provided upon reasonable request.

 

Ethical considerations

 

Written permission was sought and granted by Ghana Health Service through the Regional Director of Health Services and the District Director of Health Services. We obtained consent from interviewees during the data collection period.

 

 

Background Characteristics of Respondents, Sunyani Municipality, 2020

 

In assessing the performance of the ILI surveillance system 22 key surveillance staff were interviewed. Some of the key surveillance staff interviewed included the ILI Municipal focal person, the Municipal Disease Control Officer, and the Municipal Health Information Officer. Only 27.3% (6/22) of the surveillance staff received ILI training. The most recent ILI training for those in the Sunyani Municipal Hospital sentinel site was in 2018 while that for the 3MRS sentinel site was in 2019.

 

Description of the ILI Surveillance System

 

Influenza surveillance in Sunyani municipality began in 2007 with the 3 Medical Reception Station as the only sentinel site. The municipality has two sentinel sites, the Sunyani Municipal Hospital and the 3 Medical Reception Station (MRS). In terms of the system´s operation in the municipality, clinicians at the Out Patient Department (OPD) of the various health facilities in the municipality identify suspect cases using the suspected case definition of ILI; these suspected cases are referred to either the Municipal hospital or the 3MRS. At the sentinel site, the disease control officer, nurse, or doctor completes a case investigation form and collects a specimen. The nasopharyngeal or oropharyngeal specimen is collected into a single cryovial with Viral Transport Media (VTM). The specimen is immediately refrigerated at 2 to 8 degree Celsius temperature. Specimens are transported in a triple packaging system and carried to the National Influenza Center (NIC) by courier service or hand-carried. At the NIC, specimens are refrigerated pending testing. The stored specimens are tested by real-time reverse transcription-polymerase chain reaction for influenza A and B viruses [10]. The test results are disseminated to the reporting sentinel site, the National Disease Surveillance Department (DSD), and the WHO. At the sentinel site, feedback is captured into the sentinel registers and the DHIMS 2 platform (Figure 1).

 

Case definition of ILI

 

The system operates using two case definitions for standardized case identification, including a suspected case definition and a confirmed case definition.

 

Suspected case definition: A person; child or adult with sudden onset of fever > 38 °C AND Cough or sore throat in the absence of other diagnoses.

 

Confirmed case definition: A case that meets the suspected case definition and is laboratory confirmed (laboratory results must be positive for influenza virus) [11].

 

ILI case detection by the surveillance system

 

A total of 1900 suspected cases of ILI were recorded by the two sentinel sites from 2015-2019 of which 227 were positive, giving a positivity rate of 11.9% (Table 2 ).

 

The utility of the ILI surveillance system in meeting its set objectives

 

For the 2015-2019 period evaluated, the ILI surveillance system was able to characterize the influenza viruses circulating in the municipality to be from influenza A and B lineages. The predominant influenza subtype was A(H3N2) at 45.0% (102/227). No outbreaks of ILI were detected over the period evaluated.

 

System attributes assessed

 

Simplicity: Reporting of ILI cases was done by the OPDs of the sentinel sites. Follow-ups for confirmed cases of ILI were averagely 5.0 days (±2). The case investigation forms were clear and comprehensive in terms of what data were collected. Of the total persons interviewed, 22.73% (5/22) had challenges filling out the case investigation forms, and 9.0% (2/22) could not fully outline the case definition. Collection of nasal or throat specimens for testing required specialized training. The sentinel sites did no laboratory testing of the specimen collected. Posters of the ILI case definition were only available at the public health unit of the 3MRS. An overall indicator assessment score of 2 was recorded for simplicity (Table 3).

 

Flexibility: Data obtained from the sentinel sites showed no change occurred in the ILI surveillance system operation. The ILI case definition was however used in the surveillance of Severe Acute Respiratory Infections (SARI). Flexibility was scored 3 based on the indicators assessed (Table 3).

 

Acceptability: Each sentinel site is expected to detect five suspected cases of ILI every week per the recommendations of the National Influenza Centre. Each of the sites is expected to submit an average of 260 suspected cases at the end of the year. Over the five years, the Municipal Hospital sentinel site only met this quota in 2017 and 2018. The 3MRS sentinel site met the case detection quota in only 2019. At the Municipal hospital sentinel site and the 3 MRS, out of 50 randomly sampled case investigation forms, 92.0% (46/50) were completed. Out of the 22 staff interviewed, 68.2% (15) were willing to operate the surveillance system. An overall indicator assessment score of 3 was recorded for acceptability (Table 3).

 

Sensitivity: The ILI surveillance system was able to capture cases of ILI throughout the period with no outbreaks detected (Table 3).

 

Stability: The NIC, the hub where ILI surveillance data is stored, has a robust data management policy. The WHO´s Flu-Net database also serves to store data from ILI surveillance across the globe. Neither of the two sentinel sites had operating computers with backup storage accessories. Both sentinel sites were detecting cases throughout the period evaluated. The heads of the two sentinel sites mentioned the constant availability of ILI surveillance logistics for their activities. The WHO, NAMRU-3, and CDC mainly provide funding for the program. An overall indicator assessment score of 3 was recorded for stability (Table 3).

 

Timeliness: Of 138 case investigation forms sampled, 93.0% (105/138) of suspected cases had a specimen collected within 10 days of signs and symptoms onset. The period of specimen collection at the facility until laboratory testing averaged 7±2 days. We could not assess the timeliness of report submission due to a lack of documentation of weekly reporting. Timeliness was scored 2 per the system´s performance on the above timeliness indicators (Table 3).

 

Representativeness: Females constituted 59.2% (1125/1900) of suspected cases. The majority 61.7% (140/227) of cases confirmed were between 20-30 years of age. The sentinel sites were located in only 2 of the six subdistricts in the municipality. An overall indicator assessment score of 3 was recorded for the representativeness of the ILI surveillance (Table 3).

 

Data quality: Of the 50 sampled case investigation forms 46 (92.0%) were completed. Sentinel registers were completed with all key parameters. Out of the 558 suspected ILI cases reported in the records of the two (2) sentinel sites in 2017, 8.7% (48) of these records could not be traced at the NIC. The data for all other years were however consistent at the sentinel sites and the NIC. An overall indicator assessment score of 3 was recorded for data quality (Table 3).

 

The usefulness of the ILI surveillance system.

 

Data generated by the ILI surveillance system in the Sunyani municipality allows for the estimation of the disease burden in the municipality and the country as a whole (Table 3). The data generated forms part of the data used by WHO in collaboration with the CDC in informing the choice of vaccines and antivirals used in influenza treatments.

 

Positive Predictive Value: Over the period evaluated, the lowest PPV of 2.4% (2/84) was recorded in 2015 and the highest PPV of 15.1% (86/569) was recorded in 2019. The overall PPV was 11.9% (227/1900) (Table 4).

 

 

We evaluated the ILI sentinel surveillance system to determine if it was meeting the objectives for which it was established. We found that the ILI surveillance system in the Sunyani municipality was able to detect and characterize influenza viruses in circulation A(H3N2), FLU B YAM, and FLU B VIC. Of the circulating strains identified, the predominant subtype in circulation was the influenza A(H3N2) virus. This is consistent with the findings of a study conducted in the Greater Accra Region of Ghana , which reported about 41.1% of all ILI cases detected were caused by A(H3N2) virus [2].

 

The evaluation further revealed majority of suspected cases were between 20-30 years. The finding is inconsistent with the findings of studies conducted in Georgia, South Africa, and Vietnam, where most ILI cases suspected were among children and adolescents [12-15].

 

The timeliness of case detection, laboratory confirmation, and result dissemination were poor while records of timeliness of report submission by either sentinel sites to the municipal health directorate could not be evaluated due to its unavailability. However, this is inconsistent with the findings of studies conducted in New Zealand and Ghana that rated ILI surveillance systems high in terms of timeliness [2,16]. The disparity in findings could be attributed to the proximity of the settings where these studies were conducted to the ILI testing centers compared to this current study. Furthermore, in a study conducted in Tunisia, the researchers stated the timeliness of an ILI surveillance system may be affected by the geographical distribution of sentinels [17]. Both sites had a universal specimen submission rate to the NIC with all stakeholders willing to continue operating the surveillance system. Although acceptable by all participating stakeholders, the system failed to meet the case detection quota for three of the five years evaluated. In a similar evaluation conducted in South Africa, many sentinel sites could not meet the sample submission quota [15].

 

In terms of representativeness, the two sentinel sites reported cases throughout all age groups and sex all year round, giving a 100% reporting rate. This is consistent with the findings of a study conducted in Tunisia, where the system was rated good in terms of representatives [17]. Siting the sentinel sites in only two of the six subdistricts in the municipality is a limitation affecting the system's representativeness. This is consistent with the findings of a study conducted in South Africa where public sentinel sites were set up in four out of nine provinces [15]. To increase reporting and ensure representation of all parts of the municipality, public sentinel sites should be set up in the other sub-districts or facilities should be selected and equipped to report cases of ILI. Data quality in terms of completion of case investigation forms was appreciably high. However, a slight discrepancy existed in the data acquired from the sentinel registers and the DHIMS II. This is consistent with the findings of a similar study in DR Congo and Madagascar that reported over 90% data quality level [18,19]. The stability of most surveillance systems depends on resources and funding available to run the system for the period together with how acceptable the system is to the stakeholders [20]. The absence of interruptions in the operation of the system and the availability of backups made it possible for the system to detect cases throughout the period evaluated. The reliance of the ILI surveillance system solely on donor funding might affect its stability. The PPV during the five years was low. This was a common finding considering the numerous respiratory tract infections, whether upper or lower, characterized by similar manifestations and partly because physicians treated suspected cases in the previous years based on clinical diagnosis and not laboratory confirmation. The timely provision of tools and regular training exercises by NIC is key in sustaining the system. Most of the surveillance staff at the sentinel sites showed a high interest in operating the system because of the regular training organized by the NIC and the timely provision of tools used in operating the system. The efforts of the laboratory component of the system are also commendable.

 

The evaluation further revealed that the ILI surveillance system in the municipality was useful. Estimating the burden of ILI in the municipality and the country as a whole is achieved with the support of data generated by the system. Also, through the data submitted to various global partners through the national influenza centre, the WHO in collaboration with the CDC are able to determine circulating viral strains to inform the choice of vaccines and antivirals used in influenza treatments. This is validated by the findings of a study conducted in Tunisia. The researchers reported the relevance of the system in making it possible to determine the disease burden and also inform the choice of antivirals [17].

 

 

The ILI surveillance system in Sunyani municipality was partially meeting its set objectives. The system showed good performance in terms of the attributes and indicators evaluated. The system was of good data quality, representative, flexible, acceptable, and sensitive in influenza case detection but complex. The system was lacking in terms of timeliness, stability, and PPV. The system was useful. The NIC should intensify the training of surveillance staff on filling ILI surveillance case investigation forms and specimen.

 

 

The authors declare no competing interest.

 

 

AGM conceptualized the study, implemented data acquisition, performed the analysis, and drafted the initial manuscript. CT, DB, MAO, DA, and EK edited and reviewed the manuscript. AGM, CT, DB, MAO, DA, EK read and approved the final manuscript.

 

 

We are grateful to the Sunyani Municipal Health Directorate and the two ILI sentinel sites for providing us with the required data.

 

 

Table 1: Performance of Influenza-like illnesses surveillance system on the quantitative and qualitative attributes assessed

Table 2: Suspected cases from the two sentinel sites from 2015-2019 in Sunyani municipality, N=1,900 surveillance system on the quantitative and qualitative attributes assessed

Table 3: Performance of Influenza-like illnesses surveillance system on the quantitative and qualitative attributes assessed

Table 4: Predictive Value Positive for ILI Surveillance System

Figure 1: Flow Chart of ILI Surveillance System, Sunyani Municipal, 2022

 

 

1. WHO Regional Office for Africa. WHO Ghana 2018 Annual Report [Internet]. Accra (Ghana): WHO; 2019 [cited 2024 Jun 20]. 53 p. Download WHO Ghana 2018 Annual Report.pdf.


2. Nuvey FS, Edu-Quansah EP, Kuma GK, Eleeza J, Kenu E, Sackey S, Ameme D, Abakar MF, Kreppel K, Ngandolo RB, Afari E, Bonfoh B.Evaluation of the sentinel surveillance system for influenza-like illnesses in the Greater Accra region, Ghana, 2018 . Schnur JM, editor. PLoS ONE [Internet]. 2019 Mar 14 [cited 2024 Jun 20];14(3):e0213627. https://doi.org/10.1371/journal.pone.0213627 PubMed | Google Scholar


3. WHO. Community case management during an influenza outbreak : a training package for community health workers [Internet]. Geneva (Switzerland): WHO; 2011 [cited 2024 Jun 20]. 35 p. Download 9789241501873_eng_flipbook.pdf.


4. WHO. Influenza (Seasonal) [Internet]. Geneva (Switzerland): WHO; 2023 Oct 3 [cited 2024 Jun 25].


5. Bonney JHK, Kronmann KC, Lindan CP, Asante IA, Parbie P, Aboagye J, Amankwah J, Odoom JK, Adjabeng M, Nzussouo NT, Ahadzie L, Barthel RV, Cornelius C, Amofah G, Oyofo B, Ampofo WK.Virological surveillance of influenza-like illness among children in Ghana, 2008-2010 . The Journal of Infectious Diseases [Internet]. 2012 Dec 15 [cited 2024 Jun 20];206(suppl_1):S108-13. https://doi.org/10.1093/infdis/jis577 Google Scholar


6. Ministry of Health (GH). Protocol for Human Influenza Surveillance in Ghana [Internet]. Accra (Ghana): Ministry of Health (GH); 2016 May [cited 2024 Jun 20]. 27 p. Download Ghana_National_Influenza_Sentinel_Surveillance Final Art work.pdf.


7. WHO. Global epidemiological surveillance standards for influenza [Internet]. Geneva (Switzerland): WHO; 2013 Aug 9 [cited 2024 Jun 20]; 73 p. Download 9789241506601-eng.pdf.


8. CityPopulation. Sunyani Municipal District, Ghana- Population Statistics, Charts, Map and Location [Internet]. Oldenburg (Germany):Thomas Brinkhoff; 2022 Apr 15 [cited 2024 Jun 20].


9. Appiah S, Adekoya AF, Bapuuroh C, Akowua-Kwakye C.A Knowledge-Base of Prevalent Diseases in Sunyani Municipality, Ethiopia Using Ontological Engineering . J Health Med Informat [Internet] 2019 Nov 7 [cited 2024 Jun 25]; 10 (4): 1000338. Download a-knowledgebase-of-prevalent-diseases-in-sunyani-municipality-ethiopia-.pdf. Google Scholar


10. MoH (NZ). National Laboratory Guidelines for Pandemic Influenza Collection and Handling of Human Specimens for Laboratory Diagnosis of Influenza with Pandemic Potential [Internet]. Wellington (NZ): MoH (NZ); 2006 Jun [cited 2024 Jun 26]. 29 p. Download national-laboratory-guidelines-pandemic-influenza.pdf.


11. WHO. WHO surveillance case definitions for ILI and SARI [Internet]. Geneva (Switzerland): WHO; 2014 Jan [cited 2024 Jun 25]; [about 2 screens].


12. MeŞe S, Uyanık A, Asar S, Ağaçfidan A, Yenen OŞ. Influenza surveillance data of influenza reference laboratory for 2015-2016 flu season. TMCD [Internet]. 2016 [cited 2024 Jun 26];46(4):172-180. https://doi.org/10.5222/TMCD.2016.172 Google Scholar


13. Chakhunashvili G, Wagner AL, Machablishvili A, Karseladze I, Tarkhan-Mouravi O, Zakhashvili K, Imnadze P, Boulton ML.Implementation of a sentinel surveillance system for influenza-like illness (Ili) and severe acute respiratory infection (Sari) in the country of Georgia, 2015-2016 . International Journal of Infectious Diseases [Internet]. 2017 Oct 10 [cited 2024 Jun 26];65:98-100. https://doi.org/10.1016/j.ijid.2017.09.028 English, Turkish. Only the abstract is in English. Google Scholar


14. Dong C, Kashon ML, Lowry D, Dordick JS, Reynolds SH, Rojanasakul Y, Sargent LM, Dinu CZ.Exposure to carbon nanotubes leads to changes in the cellular biomechanics. Adv Healthcare Materials [Internet]. 2013 Jan 18 [cited 2024 Jun 26];2(7):945-51. https://doi.org/10.1002/adhm.201200430 PubMed | Google Scholar


15. Budgell E, Cohen AL, McAnerney J, Walaza S, Madhi SA, Blumberg L, Dawood H, Kahn K, Tempia S, Venter M, Cohen C.Evaluation of two influenza surveillance systems in south africa . Costa C, editor. PLoS ONE [Internet]. 2015 Mar 30 [cited 2024 Jun 26];10(3):e0120226. https://doi.org/10.1371/journal.pone.0120226 PubMed | Google Scholar


16. Adnan M, Peterkin D, Lopez L, Mackereth G.Electronic sentinel surveillance of influenza-like illness: experience from a pilot study in new zealand . Appl Clin Inform [Internet]. 2017 Dec 20 [cited 2024 Jun 26];08(01):97-107. https://doi.org/10.4338/ACI-2016-06-RA-0103 PubMed | Google Scholar


17. Yazidi R, Aissi W, Bouguerra H, Nouira M, Kharroubi G, Maazaoui L, Zorraga M, Abdeddaiem N, Chlif S, El Moussi A, Ben Hadj Kacem MA, Snoussi MA, Ghawar W, Koubaa M, Polansky L, McCarron M, Boussarsar M, Menif K, Amine S, Ben Khelil J, Ben Jemaa M, Bettaieb J, Bouafif Ben Alaya N, Ben Salah A.Evaluation of the influenza-like illness surveillance system in Tunisia, 2012-2015 . BMC Public Health [Internet]. 2019 Jun 6[cited 2024 Jun 26];19(1):694. https://doi.org/10.1186/s12889-019-7035-3 PubMed | Google Scholar


18. Babakazo P, Kabamba-Tshilobo J, Wemakoy EO, Lubula L, Manya LK, Ilunga BK, Disasuani W, Nkwembe E, Kavunga-Membo H, Changachanga JC, Muhemedi S, Tamfum JJM, Tempia S.Evaluation of the influenza sentinel surveillance system in the Democratic Republic of Congo, 2012-2015 . BMC Public Health [Internet]. 2019 Dec 10 [cited 2024 Jun 26];19(1):1652. https://doi.org/10.1186/s12889-019-8008-2 PubMed | Google Scholar


19. Rakotoarisoa A, Randrianasolo L, Tempia S, Guillebaud J, Razanajatovo N, Randriamampionona L, Piola P, Halm A, Heraud JM.Evaluation of the influenza sentinel surveillance system in Madagascar, 2009-2014 . Bull World Health Organ [Internet]. 2017 Feb 21 [cited 2024 Jun 26];95(5):375-81. https://doi.org/10.2471/blt.16.171280 PubMed | Google Scholar


20. German RR, Lee LM, Horan JM, Milstein RL, Pertowski CA, Waller MN, Guidelines Working Group Centers for Disease Control and Prevention (CDC).Updated guidelines for evaluating public health surveillance systems: recommendations from the Guidelines Working Group . MMWR Recomm Rep [Internet]. 2001 Jul 27 [cited 2024 Jun 26];50(RR-13):1-35; quiz CE1-7. Google Scholar