August 2011 Critical Care Journal Club

Reference as : Raschke RA. August 2011 Critical Club Journal Club. Southwest J Pulm Crit Care 2011;3:37-39. (Click here for PDF version)

Two manuscripts recently appeared back-to-back in the New England Journal of Medicine. Both dealt with the use of cultures and barrier precautions to prevent new cases of resistant bacterial infection in the ICU, but the manuscripts came to different conclusions regarding the usefulness of these interventions.  Taken together, these articles raise an important question in regards to evidence-based practice: How do we synthesize research into practice when studies yield conflicting results?   

Huskins CW, and colleagues.  Intervention to reduce transmission of resistant bacteria in intensive care.  New Engl J Med 2011;364:1407.  (Click here for the abstract of the manuscript)

The first study was a randomized controlled trial carried out in 18 ICUs over a 6 month period.  Surveillance cultures were obtained to detect nasal carriage of methicillin resistant Staphylococcus aureus (MRSA) and gastrointestinal carriage of vancomycin resistant Enterococcus (VRE) in all the involved ICUs.  The ICUs themselves were the unit of randomization.  In the intervention ICUs, the results of the cultures were reported to the clinicians.  These units also received enhanced training on contact precaution procedures, and an aggregate report on compliance with universal gloving.  In the control ICUs, the reports of surveillance cultures were not disclosed.  Patients in either type of ICU could be placed into contact precautions based on results of clinical microbiology – otherwise, standard precautions were practiced.  The main outcome was the incidence of new events of colonization/infection with MRSA or VRE based solely on culture positivity.  No attempt was made to distinguish colonization from infection.

Eighteen ICUs entered the study, and over 9000 patients were admitted during the study period.  The presence of MRSA or VRE was recognized more commonly in the intervention ICUs because of the reports of surveillance cultures (although I could not quantitate this effect despite re-reading the results several times).  The use of barrier precautions in patients who were colonized or infected was correspondingly higher in the intervention ICUs [92% vs. 38% p<0.001].  However, the incidence of new colonization or infection events failed to improve [40.4 events per 1000 patient days vs. 35.6, p=0.35].  

This study showed that a bundle of interventions including surveillance cultures and expanded use of barrier precautions for MRSA and VRE failed to reduce spread of these organisms. 

Randomization by ICU unit avoids many of the possible sources of bias inherent in the second study we will discuss, but this study still has some flaws.  Studies with bundled interventions are difficult to interpret because there is no way of determining whether each individual bundle component contributed or detracted from the overall result. Approximately 20% of patients in this study were not eligible for analysis of the primary outcome because initial surveillance cultures were not obtained within 2 days of admission.  Still. this study provides the best designed analysis of this research question. 

There are some reasons why surveillance cultures and barrier precautions might not be effective.  Implementation of barrier precautions were often delayed pending results of surveillance cultures.  Universal gloving (used in 43% of the intervention patients) may not be as effective as full contact precautions, and compliance with full contact precautions was imperfect.  It may be that in order for these measures to work, they need to be carried out more rapidly and with better fidelity, and perhaps combined with other interventions to reduce bacterial colonization and contamination.    

Jain R, and colleagues.  Veterans Affairs initiative to prevent methiclllin-resistant Staphylococcus aureus infections.  New Engl J Med 2011;364:1419. (Click here for the abstract of the manuscript)

This retrospective observational study followed the impact of a bundle of MRSA disease control interventions launched in VA hospitals nationwide.  The bundle included nasal surveillance for MRSA, contact precautions for patients with MRSA, improved hand hygiene, and a positive “change in the institutional culture” in relation to improved infectious disease control.  These were implemented from March through October of 2007, and then results were followed through June 2010. 

The rate of MRSA surveillance cultures exceeded 90% once the bundle was implemented.   Compared to historical controls, the rate of healthcare-associated MRSA infections decreased from 1.64 to 0.62 infections per 1000 patient days (p <0.001) in the ICUs and from 0.47 to 0.26 infections per 1000 patient days in non ICU settings (p<0.001).  The incidence of healthcare-associated MRSA infections over time is nicely graphically displayed – it’s clear from the persistent reduction over more than 2 years after the intervention, that Hawthorne effect does not explain these results.  Internal validity also seems supported by subgroup analyses which show reductions in all types of MRSA infections. The VA system investigators should be congratulated for carrying out research of this magnitude. 

However, the design of this study is essentially observational.  It’s extremely large sample size does not compensate for the potential bias introduced by the use of historical controls.  Many other factors, besides those in the intervention bundle, might have reduced infection and colonization rates over the years the study was performed.  Possibilities include improved barrier precautions during central line placement, improved processes to prevent ventilator associated pneumonia, even changes in the epidemiology of the involved organisms.  The method by which the diagnosis of MRSA infections was determined was not adequately explained, and may have introduced detection bias.  The diagnosis seems to have been made by the same clinicians and nurses who were being held responsible for preventing spread of the organisms.  This situation might lead to biased reporting of MRSA infections. There were several months during the study in which not a single MRSA central line infection was reported at any VA in the nation.  This is a flaw that was avoided in the previous study by including any new case of MRSA in the outcome variable, without attempting to distinguish infections from colonization.  Although not reported in the manuscript, the standardized mortality rate of the VA ICUs did not decrease during the intervention period (Robbins RA, personal communication) despite hospital acquired infections with MRSA being a major cause of hospital mortality (1).    

The situation in which randomized and non-randomized studies have examined the same research questions has recurred in the medical literature.  Correlation is good in many cases, but nonrandomized trials are statistically more likely to demonstrate greater treatment effect (2).  Many well-recognized international consensus groups have agreed that randomized controlled trial(s) are required for high level evidence based practice.  It is my opinion that historically-controlled studies should be used only to generate hypotheses.  They do not have the inherent internal validity to allow firm conclusions, and they should certainly not form the basis of healthcare policy.  

On a personal note, I try to do my best to practice very careful infection control hygiene.  But I find it very difficult to be 100% compliant with strict contact precautions under all circumstances in the ICU, and this is part of human nature that will likely continue to degrade the effectiveness of infection control, even in highly motivated groups of clinicians.

Robert A. Raschke MD

Associate Editor, Critical Care Journal Club

References

1. Wenzel RP. Health Care–Associated Infections: Major Issues in the Early Years of the 21st Century. Clin Infect Dis 2007;45:S85-S88.
2. Ioannidis JPA, et al.  Comparison of Evidence of Treatment Effects in Randomized and Nonrandomized Studies. JAMA.   2001;286:821-30.