Clinical Performance of an Interactive Clinical Decision Support System for Assessment of Plasma Lactate in Hospitalized Patients with Organ Dysfunction

Robert A. Raschke, MD MS

Hargobind Khurana, MD 

Huw Owen-Reece, MBBS 

Robert H. Groves Jr, MD

Steven C. Curry, MD

Mary Martin, PharmD

Brenda Stoffer, RN BSN

Banner University Medical Center Phoenix

Phoenix, AZ USA

Abstract

Purpose: Elevated plasma lactate concentration can be a useful measure of tissue hypo-perfusion in acutely deteriorating patients, focusing attention on the need for urgent resuscitation. But lactate is not always assessed in a timely fashion in patients who have deteriorating vital signs. We hypothesized that an electronic medical record (EMR)-based decision support system could interact with clinicians to prompt assessment of plasma lactate in appropriate clinical situations in order to risk stratify a population of inpatients and identify those who are acutely deteriorating in real-time.

Methods: All adult patients admitted to our hospital over a three month period were monitored by an EMR-based lactate decision support system (lactate DSS) programmed to detect patients exhibiting acute organ dysfunction and engage the clinician in the decision to order a plasma lactate concentration. Inpatient mortality was determined for the five risk categories that this system generated, and chart review was performed on a high-risk subgroup to describe the spectrum of bedside events that triggered the system logic.

Results: The lactate DSS segregated inpatients into five strata with mortality rates of 0.8% (95%CI:0.6-1.0%); 2.7% (95%CI:1.0-4.4%); 7.9% (95%CI: 6.0-10.1%), 13.0% (95%CI: 9.0-17.8%) and 42.1% (95%CI: 32.0-52.4%), achieving a discriminant accuracy of 80% (95%CI:76-84%) by AUROC for predicting inpatient mortality. Classification into the two highest risk strata had a positive predictive value for detecting acute life-threatening clinical events of 54% (95%CI: 41.5-66.5%).

Conclusions: Our lactate decision support system is different than previously-described computerized “early warning systems”, because it engages the clinician in decision-making and incorporates clinical judgment in risk stratification. Our system has favorable operating characteristics for the prediction of inpatient mortality and real-time detection of acute life-threatening deterioration.

Introduction

Over 700,000 deaths occur annually in U.S. hospitals (1). Sepsis accounts directly for 37% and indirectly for 56% of these deaths (2). Other common causes of inpatient mortality such as acute hemorrhage and venous thromboembolism (3) share certain early clinical findings with sepsis, in that they may present with deterioration of vital signs and biochemical variables before life-threatening manifestations become obvious (4). Recognition of these findings provides an opportunity for early intervention, which has been shown to improve mortality (5,6). Studies have shown that failure to rapidly recognize acute clinical deterioration is one of the most common root causes of preventable inpatient mortality (4,8).

Early warning systems (EWSs) are a type of clinical decision support system (CDSS) utilized to provide surveillance of hospitalized patients in order to alert clinicians when a patient has findings associated with acute deterioration (19). These typically monitor for abnormal vital signs or laboratory evidence of organ dysfunction, but have included many other types of clinical and laboratory variables (20-23). Modern EWSs utilize logistic regression to weight up to 36 different independent variables and yield highly stratified risk scores (24-26).

We had previous experience developing a simple EWS that triggered when at least two systemic inflammatory response syndrome (SIRS) criteria plus at least one of 14 acute organ dysfunction (OD) parameters was detected. Although this system references SIRS it was found to be nonspecific for sepsis (27), and was subsequently employed in our healthcare system to identify patients deteriorating in real-time regardless of the cause. Subsequent research showed that our SIRS/OD alert system was triggered during the course of 19% of admissions, and that patients who triggered the alert had an odds ratio of 30.1 (95% CI: 26.1-34.5) for inpatient mortality (28). We hypothesized that this SIRS/OD alert system could be used to identify high risk patients who might be further risk-stratified by obtaining a plasma lactate concentration.

Elevated plasma lactate concentration is a particularly useful biochemical marker of acute decompensation. Hyperlactemia is pathophysiologically associated with acute tissue hypoperfusion, and clinically associated with organ dysfunction and mortality (7-11). Hyperlactemia is also associated with the need for urgent clinical interventions such as transfusion and urgent surgery in trauma patients (13,14), and resuscitation of medical patients with sepsis or other life-threatening illnesses (5,15). Lactate assessment is integral to the definition of sepsis (7,16), and an essential component of the Surviving Sepsis Campaign sepsis resuscitation bundle (6). Lactate assessment is integral to achieving sepsis bundle compliance as defined by the Centers for Medicare and Medicaid Services (CMS), which has mandated participating hospitals to report as a measure of quality of care. However, lactate is only ordered about half the time that it ought to be in patients with severe sepsis and septic shock (17,18). To our knowledge, only one previously reported EWS incorporates lactate assessment (29), but this system passively utilized lactate concentration results obtained on admission from the emergency room and was not used for surveillance during hospitalization.

We sought to use our SIRS/OD alert system to actively trigger lactate assessment to identify patients suffering from sepsis or any other life-threatening disease process requiring immediate intervention during hospitalization. We hypothesized that the resulting “lactate decision support system” (lactate DSS) would provide inpatient mortality risk stratification with high discriminant accuracy, and detect acute life-threatening events with high positive predictive value compared to contemporary EWSs.

A lactate DSS with these favorable characteristics could theoretically be used to guide emergent interventions in an effort to save lives, although it was not our aim at this time to perform an interventional trial. The specific aims of this study were to pilot an interactive lactate DSS in our healthcare system, and to calculate its discriminant accuracy for mortality risk stratification, and its positive predictive value as a real-time early warning system.

Methods

We prospectively studied a cohort of all adult inpatients admitted to Banner-University Medical Center - Phoenix, a 650-bed academic hospital in Phoenix Arizona, during the first quarter of 2014. Our research was part of an ongoing system-level patient safety project and was approved by our Institutional Review Board.

The decision support logic was developed at Banner Health using Discern Expert^® (Cerner Corporation, North Kansas City MO, USA). The lactate decision support system (lactate DSS) monitored each patient in our EMR for vital signs and laboratory results consistent with SIRS and organ dysfunction, using criteria derived from the standard definition of sepsis (5-7) (Table 1).

Table 1. Lactate DSS trigger logic

If criteria for SIRS and organ dysfunction overlapped in any eight-hour window, the lactate DSS was triggered to respond. An electronic notification was generated to the patient’s nurse and physician alerting them to the possibility of acute clinical deterioration suggested by SIRS and organ dysfunction, and recommending evaluation and resuscitation if appropriate. Decision support included automatic generation of an order for a STAT plasma lactate if one was not previously ordered by the clinician, interactively prompting the clinician to cancel it if they felt it was unnecessary.

Adult admissions during the three-month study period and subsequent inpatient mortality were enumerated using our hospital’s general financial database: MedSeries4^® (Siemens Corporation, Washington DC). Although some patients triggered the lactate DSS multiple times over the course of their hospital stay, only the first trigger event was included in our analysis.

Inpatient mortality rates with ninety-five percent confidence intervals were calculated for each of five subgroups: 1) patients who did not exhibit SIRS and organ dysfunction during their hospitalization and therefore did not trigger a lactate DSS response; 2) patients who triggered a lactate DSS response, for whom a DSS-generated lactate order was cancelled by their clinician; 3) patients who triggered the lactate DSS and had a lactate concentration <2.2 mmol/L (normal for our laboratory); 4) patients who triggered the lactate DSS and had an elevated lactate of 2.2-3.9 mmol/L; and 5) patients who triggered the lactate DSS and had a highly elevated lactate >4.0 mmol/L.

It was our hypothesis that mortality in patients who triggered the lactate DSS logic would be equivalent whether the clinician chose not to cancel a DSS-generated lactate order, or the clinician had already entered a lactate order themselves. Therefore, we classified patients into the subgroups above regardless of whether their lactate order was DSS-generated or entered independently by the clinician. In order to confirm the validity of this hypothesis, the mortality rate of all patients with any lactate concentration result (the sum of groups 3, 4 and 5 above), and mortality rates within each lactate concentration strata, were separately analyzed to determine if mortality depended on the method of lactate order entry.

Stratified likelihood ratios and the area under the receiver operating curve (AUROC) generated using the five subgroups described above were calculated to determine the discriminant accuracy of the lactate DSS for the outcome of inpatient mortality.

A subgroup analysis was performed of all study patients with an elevated lactate >2.2 mmol/L (above the upper limit of normal range at our laboratory) detected by a DSS-generated lactate order during the first six weeks of the study. These patients’ charts were reviewed in order to characterize the acute clinical events that triggered a lactate DSS response in this subgroup of patients. A physician researcher reviewed progress notes, laboratory and microbiology results at the time of system activation, and for 72 hours afterwards to make this determination. Patient were determined to be suffering an acute life-threatening clinical event if a new-onset or rapidly-progressive disease process was present at the time the lactate DSS was triggered that required emergent treatment with any one of the following: >1 L intravenous fluid resuscitation, vasopressor infusion, >2 units of packed red blood cell transfusion, endotracheal intubation, advanced cardiac life support, or emergent surgical intervention. Minor clinical events included any diagnosis that required initiation of treatment not included in the definition of acute life threatening clinical events above. False alerts were said to have occurred when no evidence was found that the patient was clinically deteriorating in temporal relationship to lactate DSS activation, or within 72 hours. The positive predictive value of the system was calculated for the real-time detection of acute life-threatening clinical events. Microsoft Research and VassarStats^® on-line statistical software were used for statistical calculations.

Results

8,867 adult patients were admitted during our three-month study period. One hundred and ninety-six of 8867 patients (2.2% 95%CI: 1.9-2.5%) died while in the hospital. Seventy percent (138/196) of these inpatient deaths occurred in the 16% (1400/8867) of patients who triggered a lactate DSS response.

Four hundred seventy-nine of 1400 patients who triggered the lactate DSS already had a clinician-ordered lactate. A DSS-generated order for plasma lactate was entered for the remaining 921 patients, but clinicians cancelled 337 of these. DSS-generated lactate orders were resulted for the remaining 584 patients. These patients were merged with 479 patients who had clinician –ordered lactates for the purposes of further analysis after confirmation that mortality did not depend on how the lactate was ordered (Figure 1).

Figure 1. Stratification of inpatients into five subgroups by the lactate DSS.

Patients who did not trigger the lactate DSS logic (n=7467) had a mortality rate of 0.78% (95%CI: 0.58-0.98). Patients who triggered the lactate DSS and for whom a DSS-generated lactate order was cancelled by the clinician (n=337) had mortality of 2.7% (95%CI: 1.0-4.4%). Patients who triggered the lactate DSS and had a lactate concentration in the normal range (< 2.2 mmol/L; n=721) had mortality of 7.9% (95%CI: 6.0-10.1%), and those with elevated lactates of 2.2-3.9 and >4.0 mmol/L (n=247 and n=95) had mortality rates of 13.0% (95%CI: 9.0-17.8%) and 42.1% (95%CI: 32.0-52.4%) respectively (Figure 2).

Figure 2. Inpatient mortality rates (Y-axis: Percent mortality) with 95% confidence intervals for five subgroups of patients stratified by lactate DSS.

The mortality of patients who triggered a lactate DSS response and for whom a lactate concentration was resulted did not depend on whether the order was DSS-generated or entered by the clinician (13.0% versus 12.1% (P=0.71)). Clinician-entered lactate orders were closely temporally related to the onset of organ dysfunction, preceding lactate DSS triggering by < six hours in 52%, <12 hours in 64%, and <24 hours in 75% of cases. Likelihood ratios for mortality in subgroups of patients with lactates <2.2, 2.2-3.9, and >4.0 mmol/L were 6.1 (95%CI: 5.4-6.9), 11.8 (95%CI: 9.5-14.7), and 32.4 (95%CI: 22.0-47.1) respectively.

Five-strata of mortality risk generated by the lactate DSS yielded an AUROC of 0.80 (95% CI: 0.76-0.84) (Figure 3).

Figure 3. Receiver-operating characteristic curve for mortality risk stratification by the lactate DSS.  

Focused chart review was performed on 61 patients who had elevated lactate (>2.2 mmol/L) detected by a DSS-generated lactate order. Thirty-three (54%) were experiencing acute life-threatening clinical events at the time the lactate DSS was triggered. These included 18 episodes of sepsis. Sepsis was due to pneumonia in nine patients, catheter-associated blood stream infection, bowel perforation, cellulitis, ascending cholangitis, endocarditis, liver abscess, cholecystitis, perianal abscess, or an unidentified source. Other acute life threatening clinical events included five cases of acute gastrointestinal hemorrhage, three of acute respiratory failure, and one each of post-operative bleeding, cardiogenic shock, acute liver failure, retroperitoneal bleeding, acute myocardial infarction, subdural hematoma, and cerebral dural sinus thrombosis. Twenty-one (64%) of these events occurred outside the intensive care unit. The positive predictive value of the detection of SIRS, organ dysfunction and elevated lactate by the lactate DSS for acute life-threatening clinical events was 54% (95%CI: 41.5-66.5%).

Ten minor clinical events included anemia, atrial fibrillation, post-op third spacing, transient mild hypotension associated with end stage liver disease, sedation related to narcotics, and dialysis disequilibrium. There were 18 false alerts among patients with SIRS, organ dysfunction and elevated lactate detected by the system. (18/61=29%).

Discussion

Our lactate DSS effectively segregated a population of adult inpatients into five subgroups with increasing inpatient mortality. Clinician engagement was critically important in achieving this result. About a quarter (337/1400) of patients who triggered the lactate DSS (simultaneously exhibited SIRS and organ dysfunction) were doing well enough in their clinician’s opinion that the DSS-generated lactate order was cancelled. Clinicians exercised good judgment in this regard, identifying a subgroup of patients with inpatient mortality rate not significantly higher than the overall mortality of all patients admitted during the study. This supports our decision to incorporate clinician judgment in our risk stratification method.

Approximately half of patients (721/1400) who triggered the lactate DSS turned out to have a normal lactate concentration, yet suffered inpatient mortality ten-times higher than patients who did not trigger the system. This likely represents the independent association between SIRS and organ dysfunction with the risk for mortality (27, 31,32).

One hundred twenty-nine patients over 3 months (14.5 per 1000 patient admissions) triggered the lactate DSS and were found to have an elevated lactate concentration because of a DSS-generated lactate order. These patients had >50% probability of experiencing an acute life-threatening clinical event at the time the lactate DSS was triggered, and subsequently suffered 50% inpatient mortality.

Our lactate DSS is consistent with the new definition of sepsis because it uses organ dysfunction in addition to SIRS criteria (7). As stated in the new definition of sepsis, “Nonspecific SIRS criteria such as pyrexia or neutrophilia will continue to aid in the general diagnosis of infection” (7). Although these criteria are nonspecific, they appear to be relatively sensitive for sepsis (7,27). Our lactate DSS has excellent discriminant accuracy for predicting inpatient mortality (AUROC=0.80). It is comparable to other criteria such SOFA (AUROC = 0.74) and the Logistic Organ Dysfunction System (AUROC=0.75).The five strata into which it segregates patients could further translate into a decision support-guided treatment protocol, directing appropriate real-time interventions such as those proposed in Table 2.

Table 2. Proposed stratified clinical response to lactate DSS.

* Our data indicate that RRT activation would occur about twice a week at our hospital.

Our lactate DSS is different than EWSs because it specifically prompts assessment of plasma lactate in patients exhibiting SIRS and organ dysfunction, rather than simply generating a warning. But a discussion of the operating characteristics of previously reported EWSs is useful for purposes of comparison. A review of 33 EWSs has reported AUROCs ranging from 0.66-0.78 (19). Several more recent EWSs reported AUROCs of 0.81-0.88 (23,24,26,33), but AUROC comparisons are confounded by lack of consensus regarding which clinical outcome to analyze. Authors have variously chosen 24-hour mortality, ICU transfer, and cardiac arrest, among other outcomes (20,23,24). Many EWSs yield highly stratified results, which may increase the AUROC by adding detail to the shape of the ROC curve, but this will not improve clinical discrimination unless each resulting strata has a distinct clinical response. If a EWS is simply used to activate a rapid response team (RRT), the clinically-achievable discriminant accuracy is best described by a polygonal AUROC derived from a single cutoff with two resulting strata (activate the RRT, or do not activate the RRT). This two-strata AUROC will invariably be lower than the highly stratified AUROC that many authors report (23,24,26,33). Our AUROC analysis is based on 5 strata, each of which could reasonably trigger a distinct clinical response (Table 2).

Our lactate decision support system has a positive predictive value (PPV) for acute life-threatening clinical events that is superior to that of our previous “sepsis alert” (27) and to those reported in several reviews of EWSs. One review of 39 EWSs reported PPVs ranging from 13.5-26.1% (34), and another review of 25 systems reported a median PPV of 36.7% with interquartile range 29.3-43.8% (34). PPV was not reported for several of the most elegant and well-studied EWSs (22,23,25,32). From the perspective of bedside clinicians and rapid response team members, the efficiency of an alert system is strongly influenced by the PPV, because a poor PPV translates to frequent false alerts. The PPV is of particularly concern when the pretest probability of the outcome of interest is low, as in the case of inpatient mortality (2% at our hospital). Bayes theory indicates that a test with relatively good AUROC will have a poor PPV if the pretest probability is low enough.

Our study has several limitations. Our sample size is small compared to many contemporary EWS studies. We did not have the resources to perform focused chart reviews on all study patients and therefore had to limit individual case analysis to a subgroup of study patients. Our simple treatment of vital sign abnormalities as markers of SIRS is not as elaborate as in many EWSs. Our study is only hypothesis-generating, whereas several EWSs are well validated (25,32). We cannot provide data on how our alert might change bedside interventions by clinicians. To our knowledge, no study to date has proven that using a computerized decision support system or EWS to trigger rapid clinical intervention actually improves patient outcomes.

Conclusions

We developed an automated decision-support system that prompts assessment of plasma lactate concentration in patients exhibiting SIRS and organ dysfunction. Our lactate decision support system is different than previously-described EWSs because it engages the clinician in decision-making and incorporates clinical judgment into risk stratification. This system has favorable operating characteristics for the prediction of inpatient mortality and for detecting acute life-threatening events in real time. We have proposed a stratified clinical response based on classification of patients into five subgroups by this system that requires further testing, but our current study was not designed to demonstrate a benefit on clinical outcomes. Our lactate DSS has the potential to improve sepsis bundle compliance by helping clinicians appropriately order lactate concentrations in patients deteriorating due to the onset of sepsis – a hypothesis we are currently investigating. It also has potential for easy generalizability, particularly to other healthcare systems that share the same EMR as ours, but requires further refinement and validation.

Author Contributions

All authors were involved in conceptualization, design and implementation of the decision support system described in this manuscript, and in preparation of the manuscript, and all approve of the content of the manuscript and vouch for the validity of the data. We list below additional contributions from several of the authors:

RAR: data analysis and interpretation, main author of initial draft of the manuscript.

HOW: data analysis and interpretation, contribution to discussion/conclusions

HK: directly in charge of design and pilot implementation team for the decision support system, data interpretation, contribution to discussion, conclusions

RHG: data interpretation, contribution to discussion, conclusions

SCC: data analysis and interpretation, contribution to discussion, conclusions. Manuscript editing.

MM: data collection and analysis

BS: data collection and analysis

References

1. Center for Disease Control. Trends in inpatient hospital deaths: National Hospital Discharge Survey: 2000:2010. NCHS Data Brief 118; 2013. [PubMed]
2. Liu V, Escobar G, Greene JD, Soule J, et al. Hospital deaths in patients with sepsis from two independent cohorts. JAMA. 2014;312:90-92. [CrossRef] [PubMed]
3. Nichols L, Chew B. Causes of sudden unexpected death of adult hospital patients. J Hosp Med. 2012;7:706-8. [CrossRef] [PubMed]
4. McGloin H, Adam SK, Singer M. Unexpected deaths and referrals to intensive care of patients on general wards. Are some cases potentially avoidable? J R Coll Physicians Lond. 1999;33:255-9. [PubMed]
5. Rivers E, Nguyen B, Havstad S, Ressler J, et al. Early goal-directed therapy in the treatment of severe sepsis and septic shock. NEJM. 2001;345:1368-77. [CrossRef] [PubMed]
6. Dellinger RP, Levy MM, Rhodes A, Annane D, et al. Surviving sepsis campaign: International guidelines for management of severe sepsis and septic shock: 2012. Crit Care Med. 2013;41:580. [CrossRef] [PubMed]
7. Singer M, Deutschman CS, Seymour CW, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016 Feb 23;315(8):801-10. [CrossRef] [PubMed]
8. National Patient Safety Agency. Safer care for the acutely ill patient: learning from serious incidents. 2007;Report # PSO/5. Available online at: http://www.nrls.npsa.nhs.uk/resources/?EntryId45=59828 (accessed 5/9/17).
9. Gultepe E, Green JP, Nguyen H, Adams J, et al. From vital signs to clinical outcomes for patients with sepsis: a machine learning basis for a clinical decision support system. J Am Med Inform Assoc. 2014;21:315-325. [CrossRef] [PubMed]
10. Jansen TC, van Bommel J, Woodward R, Mulder PG, Bakker J. Association between blood lactate levels, sequential organ failure assessment sub-scores, and 28-day mortality during early and late intensive care unit stay: a retrospective observational study. Crit Care Med. 2009;37:2369-74. [CrossRef] [PubMed]
11. Bakker J, Gris P, Coffernils M, Kahn RJ, Vincent JL. Serial blood lactate levels can predict the development of multiple organ failure following septic shock. Am J Surg. 1996;171:221-6. [CrossRef] [PubMed]
12. Jansen TC, van Bommel J, Bakker J. Blood lactate monitoring in critically ill patients: a systematic health technology assessment. Crit Care Med. 2009;37:2827-39. [CrossRef] [PubMed]
13. Guyette F, Suffoletto B, Castillo JL, Quintero J, Callaway C, Puyana JC. Prehospital serum lactate as a predictor of outcomes in trauma patients: A retrospective observational study. J Trauma. 2011;70:782–6. [CrossRef] [PubMed]
14. Vandromme MJ, Griffin RL, Weinberg JA, Rue LW 3rd, Kerby JD. Lactate is a better predictor than systolic blood pressure for determining blood requirement and mortality: Could prehospital measures improve trauma triage? J Am Coll Surg. 2010;210:861–9. [CrossRef] [PubMed]
15. Jansen TC, van Bommel J, Schoonderbeek FJ, Sleeswijk Visser SL, et al. Early lactate-guided therapy in intensive care unit patients: a multicenter, open-label, randomized controlled trial. Am J Respir Crit Care Med. 2010;182:752-61. [CrossRef] [PubMed]
16. Levy MM, Fink MP, Marshall JC, Abraham E, et al. International Sepsis Definitions Conference. Crit Care Med. 2003;31(4):1250. [CrossRef] [PubMed]
17. Gao R, Melody T, Daniels DF, Giles S and Fox S. The impact of compliance with 6-hour and 24-hour sepsis bundles on hospital mortality in patients with severe sepsis: a prospective observational study. Crit Care. 2005;9:R764–R770. [CrossRef] [PubMed]
18. Levy MM, Dellinger RP, Townsend SR, et al. The Surviving Sepsis Campaign: results of an international guideline-based performance improvement program targeting severe sepsis. Intensive Care Med. 2010; 36: 222–31. [CrossRef] [PubMed]
19. Smith GB, Prytherch DR, Schmidt PE, Featherstone PI. Review and performance evaluation of aggregate weighted "track and trigger" systems. Resuscitation. 2008;77:170-9. [CrossRef] [PubMed]
20. Hodgetts TJ, Kenward G, Vlachonikolis IG, Payne S, Castle N. The identification of risk factors for cardiac arrest and formulation of activation criteria to alert a medical emergency team. Resuscitation. 2002;54:125-31. [CrossRef] [PubMed]
21. Kho A, Rotz D, Alrahi K, Cardenas W, et al. Utility of commonly captured data from an HER to identify hospitalized patients at risk for clinical deterioration. AMIA 2007 symposium proceedings. 404-8.[CrossRef]
22. Howell MD, Donnino M, Clardy P, Talmor D, Shapiro NI. Occult hypoperfusion and mortality in patients with suspected infection. Intensive Care Med. 2007;33:1892-9. [CrossRef] [PubMed]
23. Escobar GJ, LaGuardia JC, Turk BJ, Ragins A, et al. Early detection of impending physiological deterioration among patients who are not in intensive care: Development of predictive models using data from an automated electronic medical record. J Hosp Med. 2012;7:388-95. [CrossRef] [PubMed]
24. Prytherch DR, Smith GB, Schmidt P, Featherstone PI. ViEWS – towards a national early warning score for detecting adult inpatient deterioration. Resuscitation. 2010;81:932-7. [CrossRef] [PubMed]
25. Bailey TC, Yixin C, Mao Y, Lu C, et al. A trial of a real-time alert for clinical deterioration in patients hospitalized on general medical wards. J Hosp Med. 2013;8:236-42. [CrossRef] [PubMed]
26. Churpek MM, Yuen TC, Winslow C, Robicsek AA, et al. Multicenter development and validation of a risk stratification tool for ward patients. Am J Resp Crit Care Med. 2014;190:649-55. [CrossRef] [PubMed]
27. Raschke RA, Owen-Reece H, Khurana H, Groves RH Jr, et al. Clinical performance of an automated systemic inflammatory response syndrome (SIRS)/organ dysfunction alert: a system-based patient safety project. Southwest J Pulm Crit Care. 2014;9:223-9. [CrossRef]
28. Khurana, H, Groves RH, Simons MP, Martin M, Stoffer B, et al. Real-time automated continuous sampling of electronic medical records predicts hospital mortality. Am J Med. 2016 Jul;129(7):688-698.e2. [CrossRef] [PubMed]
29. Jo S, Lee JB, Jin YH, Jeong TO, et al. Modified early warning score with rapid lactate level in critically ill medical patients: the ViEWS-L score. Emerg Med J. 2013;30:123-9. [CrossRef] [PubMed]
30. Rangel-Frausto MS, Pittet, D, Costigan M, et al. The natural history of the Systemic Inflammatory Response Syndrome (SIRS): A prospective study. JAMA. 1995;273:117-123. [CrossRef] [PubMed]
31. Matthew M, Churpek F, Zadravecz J, et al. Incidence and prognostic value of the Systemic Inflammatory Response Syndrome and organ dysfunctions in ward patients. Am J Resp Crit Care Med. 2015;192:958-64. [CrossRef] [PubMed]
32. Pittet D, Range-Frausto S, Tarara LN, Lin N, et al. Systemic inflammatory response syndrome, sepsis, severe sepsis and septic shock: incidence, morbidities and outcomes in surgical ICU patients. Intensive Care Med. 1995;21:302-9. [CrossRef] [PubMed]
33. Kellett J, Kim A. Validation of an abbreviated Vitalpac early warning score (ViEWS) in 75,419 consecutive admission to a Canadian regional hospital. Resuscitation. 2012;83:297-302. [CrossRef] [PubMed]
34. Smith GB, Prytherch DR, Schmidt PE, Featherstone PI, et al. A review and performance evaluation of single-parameter "track and trigger" systems. Resuscitation. 2008;79:11-21. [CrossRef] [PubMed]
35. Gao H, McDonnell A, Harrison DA, Moore T, et al. Systemic review and evaluation of physiological track and trigger warning systems for identifying at-risk patients on the ward. Intensive Care Med. 2007;33:667-79. [CrossRef] [PubMed]

Cite as: Raschke RA, Khurana H, Owen-Reece H, Groves RH Jr, Curry SC, Martin M, Stoffer B. Clinical performance of an interactive clinical decision support system for assessment of plasma lactate in hospitalized patients with organ dysfunction. Southwest J Pulm Crit Care. 2017;14:241-52. doi: https://doi.org/10.13175/swjpcc058-17 PDF