Welcome to Straight Talk about AI in Healthcare. In each post I explain a recent research paper in non-technical terms and highlight lessons for healthcare organizations. My focus is not necessarily the most inspirational work but the most practical insights: those that help you understand what you can do today and what should be on your radar for tomorrow.
Always feel free to reach out with any feedback and article suggestions, and if you are not on the mailing list, please subscribe!
In previous posts I described challenges encountered when applying deep learning, a family of methods that has revolutionized text and image analysis, to healthcare and biological data. Here, I want to discuss a recent successful application and how it gets around some of these challenges.
In this work, published in 2019, researchers from Google used five years of data from over 700,000 patients in the VA system to predict acute kidney injury (AKI, aka kidney failure) during hospitalization, a potentially life-threatening condition that in many cases is preventable with an early intervention.
The methodology was similar to a 2018 study by Google that used deep learning on data from two hospitals to build a model that predicts various outcomes like readmission and mortality. The main innovation in that study was designing a deep learning method that used data longitudinally, taking into account not just which diagnoses, procedures, and lab values were recorded, but also when.
The 2018 study achieved significant improvements relative to standard rules-based methods, but with two significant limitations: it was not clear how to intervene on high-risk patients, and, for each hospital, a new model had to be built from scratch, at significant effort and cost. It was therefore more of an academic proof-of-concept than a solution for real-world clinical settings. The 2019 work addresses both of these limitations.
First, the use case is very focused, and there are established interventions to mitigate AKI risk. The model can alert in real time when a patient is at risk of imminent AKI, identifying well over 50% of potential AKI cases. These represent just 2.5% of total hospitalizations, a reasonable number of cases to intervene on. By comparison, a traditional rules-based approach would require double this amount to achieve the same effectiveness. So using the model clearly leads to improved outcomes at lower cost.
Second, the model was built using data from a broad population and across many hospitals. This means that the results are more likely to be applicable for other institutions, and can already be applied at >130 VA centers. There is one very important caveat: less than 7% of the VA patient population are women. As a result, the model is less accurate for female patients.
So what can you learn from this study? First, the most important part of a machine learning project in healthcare is understanding how it fits in the end-to-end clinical workflow to ensure its actionability. A focused use-case and a clear, feasible intervention plan turned the methodology developed in 2018 into a potential real-world solution in 2019.
Second, this study helps to establish some key best practices for applying deep learning to EHR data. Interestingly, this work did not use free text from clinical notes due to privacy concerns. Instead, the longitudinal information turns out to be the most critical factor for accurate predictions. At the same time, the study also highlights the importance of ensuring model transferability across different care sites and diverse patient populations.
In the next post, I will discuss another successful application of deep learning, this time to biological data.
Not on the mailing list? Subscribe to the newsletter!