Accuracy of Tuberculosis Routine Data and Nurses’ Views of the TB-HIV Information System in the Free State, South Africa

Article Outline

• Design and Methods
• Results
  • Inconsistency Between Facility-Level and Provincial-Level TB Data
  • Nurses’ Views of the TB-HIV Information System
• Discussion
• Limitations
• Conclusion
• Disclosures
• Acknowledgments
• References
• Biography
• Copyright

Key words: AIDS, health information system, HIV, South Africa, tuberculosis

Reliable data are a prerequisite for evidence-based decision making in health care policy (AbouZahr & Boerma, 2005). Accurate measurement is crucial in evaluating epidemic trends, as well as in planning and monitoring disease-specific service provision. On the basis of a systematic review of descriptive and comparative studies and previous reviews of health information technologies, Chaudhry et al. (2006) demonstrated the efficacy of information gathered using health information technologies, such as electronic health records, to improve both quality and efficiency of health care. Sound data are especially vital for the success of large-scale public sector health programs in developing countries where limited human and financial resources require their optimal use (Fraser et al., 2005).

Lippeveld (in AbouZahr & Boerma, 2005) defined a health information system (HIS) as an “integrated effort to collect, process, report and use health information and knowledge to influence policy-making, programme action and research” (p. 579). HISs are especially important when responses need to be urgent, as in the case of epidemic diseases such as tuberculosis (TB) and HIV infection. The scale-up of both TB and HIV treatment in resource-limited settings–such as South Africa–requires an integrated approach that bundles the respective TB and HIV information systems to combat the co-epidemic. Nurse respondents in the current study viewed an integrated approach as one in which, for example, integrated registers and tally sheets, inclusive of both TB and HIV information, are used. The necessity for integration is underscored by Phillips’s (2007) observation that 60% of HIV-infected persons in Africa will die as a result of TB (p. 77).

South Africa is experiencing an extraordinary TB epidemic thought to be attributable to the HIV epidemic, as patients are more susceptible to TB when their immune system is weakened by HIV (Singh, Upshur, & Padayatchi, 2007). In fact, among the 22 high-burden TB countries, South Africa displays the highest rate of HIV-incidence TB cases (73%; World Health Organization [WHO], 2009). Within South Africa, the Free State is one of the provinces hardest hit by the co-epidemic. In 2007, it recorded the third highest HIV prevalence (25.3%) and the fourth highest TB incidence (818/100,000) among the country’s nine provinces (Day & Gray, 2008). These statistics highlight the need for accurate TB and HIV surveillance.

However, accurate information is a health system resource that is often in short supply (Fraser et al., 2004). The quality of electronic records of patients has been questioned in both developed and developing countries. For example, in Finland, Vainiomäki, Kuusela, Vainiomäki, & Rautava (2008) found that “patient records are inadequate documents of consultations and below the standard of that country’s legislation” (p. 117). In the case of developing countries, the supply of accurate information is often not seen to be a priority because of limited human and financial resources.

In particular, the accuracy of recording and reporting systems for TB and TB-HIV programs is often questioned (Fraser et al., 2007, Kim et al., 2008, Scholten et al., 2008, Sprinson et al., 2006). The high burden of the co-epidemic forces the health personnel to concentrate efforts on patient care rather than data management. The evaluation of collaborative TB-HIV activities is especially challenging because of the lack of investment in monitoring and evaluation and the added complexity of sharing information between two largely vertical programs (Gunneberg, Reid, Williams, Floyd, & Nunn, 2008). The WHO (2008) warned that the integration of recording and reporting for TB into a multidisease information system, unless carefully managed, was likely to result in further deterioration in the quality of routinely collected data. In addition to the burden of integrating TB and HIV care and treatment, the number of people undergoing TB and antiretroviral treatment (ART) in the Free State are increasing, thus further increasing the burden on the public health care system.

Recent years have seen the Free State’s Department of Health’s ART program invest substantially in information systems for routine data recording and monitoring systems. Investments have included procurement and implementation of computer information technology systems, including the appointment and training of staff to enable and expedite the flow of routine information. However, based on the experience of developing a database and data collation system for monitoring and evaluating ART in the province, it has been found that data quality remained the most challenging aspect (Fairall et al., 2006). In addition, it is important to state that parallel investments in the recording and reporting system for TB program were latent. Therefore, there is a likelihood of similar or more difficult data quality challenges because of fewer financial, staff, and infrastructural investments made in the TB information system.

Few previous South African studies have researched the efficacy of public HISs. Such research is much needed because a paradox emerges: the massive burden of the combined TB and HIV epidemics requires accurate information to enable successful comprehensive policy formulation and implementation, but the burden of the co-epidemic inundates nurses, possibly causing them to consider collecting and entering data a lesser priority.

An effective and efficient information system is now more than ever a research and policy priority. This research set out to establish possible disparity between facility and provincial levels of TB program data, and how users and managers experience and perceive the functionality of the TB-HIV information system. More specifically, the objectives were as follows: (1) to compare information in the facility-based TB patient files with information submitted to and captured at the provincial corporate level, and (2) to determine whether nurses providing TB care (a) reported using TB-HIV data for planning purposes, (b) experienced problems in the daily operation of the information system, and (c) thought the information system could be improved for the clinical management of TB-HIV co-infected patients.

Back to Article Outline

Design and Methods 

The study followed an exploratory and descriptive design. Field work was conducted between November and December, 2007, in all the five districts of the Free State Province–Fezile Dabi, Lejweleputswa, Motheo, Thabo Mofutsanyana, and Xhariep. In each district, one mobile and one fixed clinic, one community health center (or large clinic), and one district hospital were purposively selected. Although budgetary restrictions limited the number of facilities to 20, they covered the whole spectrum of public sector primary-level facility types providing TB and HIV services, including small, medium, and large facilities, as well as those located in rural, peri-urban, and urban areas.

For an objective assessment, data were recorded on a random sample of TB and HIV patients (n = 20 per facility, total n = 400) selected directly from facility-based records (patient files or electronic registers) by means of a table of random numbers. For comparison between facility and provincial level data, the following seven data items were collected: file number, surname, name, treatment status (whether the patient was a new or a re-treatment case), treatment start date, treatment end date, and patient transfer status. These data were compared with the data recorded for the same patients at the provincial corporate level.

For a subjective assessment, semi-structured interviews were conducted with the nurse mainly responsible for TB program coordination at each facility (n = 20). The nurses’ participation in the interviews was subject to informed, voluntary, and written consent. Their views were elicited by asking them whether they used TB-HIV data for planning purposes, experienced problems in the daily operation of the information system, and thought the information system could be improved for the clinical management of TB-HIV co-infected patients.

TB patient data from the facility and province were captured in Excel and checked for inconsistencies. Responses to the open-ended questions were coded and captured in SPSS (Version 16) and frequencies were calculated. Data were subjected to descriptive analysis (i.e., calculation of frequencies and means). The study was approved by the Committee for Research Ethics of the Faculty of the Humanities, University of the Free State. All respondents participated on an informed voluntary basis and signed a form to this effect.

Back to Article Outline

Results 

Inconsistency Between Facility-Level and Provincial-Level TB Data 

For the objective assessment, of a total of 2,800 data entries, 21% (n = 592) inconsistency between data in hardcopy TB patient files (or, where available, electronic registers) and the data for the same patient recorded at the provincial level was measured (Table 1). Per data item, the highest discrepancy was observed in the TB treatment start (44%) and end (41%) dates. Inconsistencies regarding whether the patient had been transferred to another facility or not (30%) were also frequently observed, as was inconsistency in treatment status (12%). In contrast, smaller proportions (4%) of the patients’ file numbers and names (8%) and surnames (9%) were discrepantly (different names, surnames, and misspellings) recorded.

Table 1. Inconsistency Between Facility and Provincial TB Routine Data (N = 2,800 Entries)

Per patient, when comparing clinic-based data with provincial level data, one third (n = 128, 32%) of the 400 cases showed no inconsistency in any of the seven data items. In four of every 10 (41%) cases, one to two, and in just more than a quarter of the cases (27%), three to five, inconsistencies were observed. In 7% of the cases, four or more of the seven data items were discrepantly recorded.

Nurses’ Views of the TB-HIV Information System 

For purposes of the subjective assessment, we elicited TB program nurses’ opinions on three questions: First, for the question asking nurses whether they used the TB-HIV data for planning purposes, all the respondents answered affirmatively. They indicated that the TB-HIV data were used to monitor and evaluate performance and outcomes of the TB program (n = 9); provide feedback to managers, staff, and the community (clinic committees; n = 6); and to calculate resource requirements (n = 5). In fact, almost all of the respondents (n = 19) stated that they indeed found the TB-HIV data useful for planning purposes.

Second, for the question asking TB program nurses whether any problem areas existed in the daily operation of the TB-HIV information system, half of the respondents (n = 10) denied that they experienced such problems. The problems most frequently raised by the remaining 10 TB-coordinating nurses related to staff shortages (n = 4), lack of training (n = 4), too many forms to complete (n = 4), and difficulties experienced in keeping track of patients (n = 3).

Staff shortages were found to be the most important challenge in the daily operation of the TB-HIV information system: “If we could only have a data capturer it would help us a lot.” The nurses who viewed a lack of training as a problem particularly referred to the fact that many nurses had not (yet) undergone training in terms of the Practical Approach to Lung Health and HIV in South Africa (PALSA Plus). PALSA Plus is a multifaceted training program that combines symptom and sign-based guidelines with educational outreach and supervisory support to primary care nurses. Some of the respondents also suggested that PALSA Plus guidelines should be available in all consultation rooms in clinics. The problem of “too many forms” was closely related to the nurses’ suggestion that a single, integrated information system should be implemented. The problem of “difficulty in keeping track of patients” related to TB patients reportedly often providing incorrect or untraceable contact particulars.

Third, with regard to the question asking nurses to provide suggestions as to how the information system could be improved for the clinical management of TB and TB-HIV co-infected patients, the largest proportion (n = 6) of the nurses suggested that a single, integrated information system for the two programs should be implemented: “All information necessary for reporting should be on one tally sheet”; “All TB and HIV and AIDS information should be recorded in one register”; and “One form that is inclusive of all TB and HIV information is needed.”

Some TB nurses felt that their supervisors needed to provide more frequent feedback on the data submitted by facilities (n = 3): “Feedback on our TB data is necessary for us to be able to make graphs and for planning. Meetings with the TB coordinators in the district should be held at least quarterly.” and “Communication about the information should go from facility to district to province and vice versa.” It was also mentioned that all facility staff should undergo training on the operation and intricacies of the TB-HIV information system (n = 2): “If I [professional TB nurse] go on leave there is no one trained and able to maintain the TB data.” Two nurses were of the opinion that secrecy about TB patients’ HIV status should be eliminated (n = 2): “The secrecy surrounding HIV makes it difficult to appropriately manage [co-infected] patients.” The need for sufficient computers and/or electronic registers was also pointed out (n = 2): “We need computers and electronic registers with certain trusted nurses having passwords to access patients’ HIV information. The TB patients are hiding their HIV status.”

Back to Article Outline

Discussion 

The highest levels of discrepancy between the clinic- and provincial-level TB routine information were observed with respect to TB treatment start and end dates, whether the patient was transferred to another facility or not, and treatment status. All of these data items are essential for the calculation of smear conversion and treatment completion and cure rates. Although these could usually be resolved by cross-checking file and electronic data entries, the comparatively smaller proportions of discrepant patient file numbers, names, and surnames were important for administrative purposes and for tracking of patients, especially if they transferred from the clinic where they were initially registered to another clinic.

It is unfortunate that, as far as could be established, the wider norm for the types and levels of discrepancies in data recorded at different levels of the health system was not known or documented. Generally, research into the quality of TB surveillance is scarce. One of the few available studies that assessed the quality of TB surveillance found poor surveillance in one third of the cities in Brazil (Braga, 2007). The author also noted that other studies on the assessment of TB surveillance in Brazil could not be found, thus ruling out comparison. In a study to validate the surveillance system for TB in Botswana, Alpers et al. (2000) noted that the actual performance of the Botswana National TB Program with respect to sputum microscopy examination was much better than surveillance indicators suggested. An evaluation of the District Health Information System in rural South Africa found that 25% of the data were outside expected ranges, and no explanation was provided (Garrib et al., 2008).

Observed discrepancies between facility- and province-level data in the Free State cannot be assessed other than against the backdrop of severe resource shortages at the time of our cross-sectional survey. These shortages were graphically illustrated by the findings of the 2006 Health Information Report South Africa (Loveday, Smith, & Monticelli, 2006) with respect to the Free State: only 11% of HIS posts were filled, only 15% of HIS staff spent more than 80% of their work time doing HIS-related work, 24% had no e-mail access from their own computers, and 21% had no intranet access from their own computers.

In the subjective assessment in the current study, the nurses consistently reported using the TB-HIV routine data for planning purposes. The high rate of utilization of the data is positive since, according to Byskov and Ohlson (cited in Garrib et al., 2008), there has been a universal “culture of reporting” instead of a “culture of using” information at the district level (p. 551). In contrast, the fact that the data were in large measure discrepant between the facility and provincial levels probably meant that much incorrect data were factored into the planning process. However, half of the nurse respondents were seemingly not aware of this as they denied that any problems occurred in the daily operation of the information system.

For the question regarding how the TB-HIV information system in the Free State could be improved, the major recorded suggestion of the nurses was that a single integrated system should be implemented. This would clearly require policy change, as well as strong leadership and technical expertise to develop the system. If these could be mobilized, efficiency gains could be anticipated by the use of a common information system and a single set of records.

However, without sufficient staffing and training, any new system is likely to return discrepancies and problems similar to those identified in the current study. This is because 51.6% of professional nursing posts and 50.7% of all health professional posts in the Free State were vacant in 2008 (Day & Gray, 2008, pp. 359-360). Loveday et al. (2006) also reported that, at almost 90%, the Free State had the highest proportion of vacant information officer posts in the country.

Back to Article Outline

Limitations 

The limitations of this exploratory study should be noted. Although the research was conducted throughout the Free State, it was only possible to include 20 health facilities (i.e., 5% of all mobile and fixed clinics, community health centers, and district hospitals in the province) because of a limited budget. Hence, the extent to which the findings may be generalized is limited. However, the results raise important questions regarding the accuracy of routinely collected data and the value thereof for planning and policy implementation. A potential path for future research is to investigate not only the accuracy and use of TB-HIV routine information, but also to expand on nurses’ ideas on how program data are used to monitor and evaluate performance, what type of feedback is provided to managers, and how this, in turn, is used to plan for and manage TB and HIV services at the sub-district, district, and provincial levels.

It is also necessary to emphasize that data were gathered through a once-off cross-sectional survey and that the TB management program has since made efforts to improve the TB recording and reporting system. However, these efforts may have been thwarted by a lack of funds and staff shortages–particularly health information officers or data-capturing officials.

In late 2007, the Free State Department of Health implemented an upgraded and more user-friendly version of the “Electronic TB Register” (ETR; i.e., a user-friendly Epi-Info based software program) based on the WHO/International Union of Tuberculosis and Lung Disease format of recording and reporting. The ETR is a powerful tool for surveillance, management, and supervision for countries with well-functioning paper-based recording and reporting systems. Implementation of the ETR might thus have resulted in decreased disparity between facility and provincial level data.

Although implementation of ETR might have resulted in decreased disparity, this is still not the integrated TB-HIV information system that is required to aptly combat the TB-HIV co-epidemic. Again the lack of accompanying infrastructural and human resource improvements may have thwarted efforts to improve data quality. Indeed, a Malawian study found that improved data quality in ART clinics was associated with increased human resources (clerks) for record keeping (Makombe et al., 2008). In the words of Jacucci (2000): “Building local infrastructure and human resources capacity is certainly a key aspect for a sustainable information system” (p. 235), whereas Health Metrics Network (2008) stated that “…improved health outcomes cannot be achieved without strengthening health systems (including HISs) as a whole, rather than focusing on discrete, disease focused components” (p. 7).

Back to Article Outline

Conclusion 

Health program information systems should facilitate accountability and evaluation as a means to address public health challenges. At the time of our cross-sectional study in the Free State, it seemed that a large part of the information collected for planning, reporting, and for policy making and resource allocation in the Free State TB management program was in need of attention to rectify discrepancies between the facility and provincial levels. Data such as TB treatment start and end dates are used to calculate TB program outcomes. However, our interviews with front-line nurses serving the TB program showed that only half of them thought that the TB-HIV information system was problematic. Intensified efforts to assess quality and find ways to improve the TB-HIV information system are recommended. In view of the serious nurse shortages in the province, more recruitment, training, and deployment of dedicated data capturers in the TB program will be required. Follow-up research is also recommended to establish the potential effect of recent efforts by the province to improve the TB information system in the context of stringent financial, nurse, information officer, and training shortages. Most importantly, the need for integration of the TB-HIV information systems remains unresolved.

Back to Article Outline

Disclosures 

The study was commissioned and funded by the Free State Department of Health. Christo Heunis, Edwin Wouters, Gladys Kigozi, and Michelle Engelbrecht report no financial interests or potential conflicts of interest. Yolisa Tsibolane, Sonja van der Merwe, and Seipati Motlhanke are Free State Department of Health managers responsible for the TB management, TB programme surveillance, and TB-HIV/AIDS integration.

Back to Article Outline

Acknowledgments 

The authors thank Free State Department of Health for funding the study, Leona Smith for gathering data, and Hannes van Biljon and other CHSR&D colleagues for computerization of the data.

Back to Article Outline

References 

1. AbouZahr C, Boerma T. Health information systems: The foundations of public health. Bulletin of the World Health Organization. 2005;83:578–583
   • View In Article
   • MEDLINE
2. Alpers L, Chrouser K, Halabi S, Moeti T, Reingold A, Binkin N, et al. Validation of the surveillance system for tuberculosis in Botswana. International Journal of Tuberculosis and Lung Disease. 2000;4:737–743
   • View In Article
3. Braga JU. Tuberculosis surveillance and health information system in Brazil, 2001-2003. Revista Saúde Public. 2007;41(Suppl 1):1–11
   • View In Article
4. Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, et al Systematic review: Impact of health information technology on quality, efficiency, and costs of medical care. Annals of Internal Medicine. 2006;144:e12–e22
   • View In Article
5. Day C, Gray A. Health and related indicators. In:  Barron P,  Roma-Reardon J editor. South African Health Review 2008. Durban, South Africa: Health Systems Trust; 2008;p. 239–395
   • View In Article
6. Fairall, L., Staniland, G., Msimanga, M., Timmerman, V., Goedele, L., Bachmann, M., … Chapman, R. (2006). Development of an integrated database and data collation system for monitoring and evaluating the public sector antiretroviral treatment (ART) in the Free State province, South Africa. Paper presented at the 16th International AIDS Conference, Toronto, Canada.
   • View In Article
7. Fraser HSF, Allen C, Bailey C, Douglas G, Shin S, Blaya J. Information systems for patient follow-up and chronic management of HIV and tuberculosis: A life-saving technology in resource-poor areas. Journal of Medical Internet Research. 2007;9:e29
   • View In Article
8. Fraser HSF, Biondich P, Moodley D, Choi S, Mamlin BW, Szolovits P. Implementing electronic medical record systems in developing countries. Informatics in Primary Care. 2005;13:83–95
   • View In Article
   • MEDLINE
9. Fraser HSF, Jazayeri D, Nevil P, Karacaoglu Y, Farmer PE, Lyon E, et al An information system and medical record to support HIV treatment in rural Haiti. British Medical Journal. 2004;329:1142–1146
   • View In Article
10. Garrib A, Stoops N, McKenzie A, Dlamini L, Govender T, Rohde J, et al. An evaluation of the district health information system in rural South Africa. South African Medical Journal. 2008;98:549–552
    • View In Article
11. Gunneberg C, Reid A, Williams BG, Floyd K, Nunn P. Global monitoring of collaborative TB-HIV activities. International Journal of Tuberculosis and Lung Disease. 2008;12(3 Suppl 1):2–7
    • View In Article
12. Health Metrics Network. Framework and standards for country information health information systems. Geneva, Switzerland: World Health Organization; 2008;Retrieved from http://www.who.int/healthmetrics/documents/hmn_framework200803.pdf
    • View In Article
13. Jacucci E. Standardization of health information systems in South Africa: The challenge of local sustainability. Information Technology for Development. 2000;12:225–239
    • View In Article
14. Kim J, Langevin M, Wylie EL, McCarthy AE. The epidemiology of tuberculosis in Ottawa, Canada. International Journal of Tuberculosis and Lung Disease. 2008;12:1128–1133
    • View In Article
15. Loveday M, Smith J, Monticelli F. Health information audit report South Africa. Durban, South Africa: Health Systems Trust; 2006;Retrieved from http://www.hst.org.za/publications/765
    • View In Article
16. Makombe SD, Hochgesang M, Jahn A, Tweya H, Hedt B, Chuka S, et al Assessing the quality of data aggregated by antiretroviral treatment clinics in Malawi. Bulletin of the World Health Organization. 2008;86:241–320
    • View In Article
17. Phillips KD. A look at tuberculosis and its relationship to HIV/AIDS. Journal of the Association of Nurses in AIDS Care. 2007;18:75–78
    • View In Article
18. Scholten JN, de Vlas SJ, Zaleskis R. Under-reporting of HIV infection among cohorts of TB patients in the WHO European Region, 2003-2004. International Journal of Tuberculosis and Lung Disease. 2008;12:s85–s91
    • View In Article
19. Singh JA, Upshur R, Padayatchi N. XDR-TB in South Africa: No time for denial or complacency. PLoS Medicine. 2007;4:e50
    • View In Article
    • CrossRef
20. Sprinson JE, Lawton ES, Porco TC, Flood JM, Westenhouse JL. Assessing the validity of tuberculosis surveillance data in California. BMC Public Health. 2006;6:217;Retrieved from http://www.biomedcentral.com/1471-2458/6/217
    • View In Article
    • MEDLINE
    • CrossRef
21. Vainiomäki S, Kuusela M, Vainiomäki P, Rautava P. The quality of electronic records in Finnish primary healthcare needs to be improved. Scandinavian Journal of Primary Health Care. 2008;26:117–122
    • View In Article
    • CrossRef
22. World Health Organization. Global tuberculosis control: Surveillance, planning, financing. Geneva, Switzerland: World Health Organization; 2008;WHO/HTM/TB/2008.393. Retrieved from http://www.who.int/tb/publications/global_report/2008/en/index.html
    • View In Article
23. World Health Organization. Global tuberculosis control: Epidemiology, strategy, financing. Geneva, Switzerland: World Health Organization; 2009;WHO/HTM/TB/2009.411. Retrieved from http://www.who.int/tb/publications/global_report/2009/en/index.html
    • View In Article