Dive Brief:
- Three studies published in Nature by researchers at Johns Hopkins University and machine learning startup Bayesian Health found that a sepsis early detection tool reduced relative deaths from sepsis by 18.2%. Sepsis is a life-threatening response to an infection that is estimated to cause about a third of hospital deaths, according to the Centers for Disease Control and Prevention.
- The prospective studies, which took place across five hospitals, found that Bayesian’s machine learning platform identified 82% of sepsis cases, and 38% of alerts were confirmed by a doctor. When an alert was confirmed by a doctor within three hours, patients received antibiotics nearly two hours faster than patients whose alert was addressed later, dismissed or never confirmed.
- Although sepsis prediction tools have been adopted at hundreds of hospitals, few prospective studies evaluate how they perform in the real world. A study published last year found that a widely used model developed by Epic Systems missed 67% of sepsis cases at one hospital, despite generating alerts for 18% of hospitalized patients.
Dive Insight:
Bayesian Health, a startup spun out of Johns Hopkins, tested its sepsis-detection platform across two academic hospitals and three community hospitals between 2018 and 2020.
At first, researchers ran the system in the background and measured how it performed against the current standard of care. Then, they deployed it in hospitals and measured provider adoption of the tool and how it affected patient outcomes.
The study measured the tool’s effectiveness at one pre-specified setting, but its performance could change under different alert settings, researchers cautioned.
Suchi Saria, CEO of Bayesian Health and director of Johns Hopkins’ machine learning, AI and healthcare lab, said in an interview that the accuracy of the tool was critical in providers’ willingness to adopt it.
Many hospitals that currently use sepsis-alerting tools either use a rule-based system, or simple predictive models. However, past studies have shown flaws with these systems, which can be too rigid or don’t detect sepsis until after a patient has started receiving antibiotic treatment.
“That has been a huge flaw in the field — people haven't rigorously measured things like sensitivity, precision, lead time, metrics that are necessary for being able to evaluate and understand system performance in order to see if it's going to be beneficial,” Saria said.
One of the challenges developers face is that predictive tools can become less accurate as patient populations change, as they go from one hospital to another, or over time as conditions change such as the start of the pandemic.
“The amount of data that's collected in the ED looks very different than the floor, than the ICU, so you really need systems that can adapt to different patient populations, adapt to different settings in which it's deployed and adapt to the variations in practice patterns that you might see as you go from one hospital to the next to the next,” Saria added.
Bayesian is working to integrate its predictive tool into more electronic medical record systems. It currently can integrate with Epic and Cerner’s workflows. Going forward, the company plans to partner with more systems to deploy its platform and expand into other health conditions.
