Models built on machine learning in health care can be victims
of their own success, according to researchers at the Icahn School
of Medicine and the University of
Michigan. Their study assessed the impact of implementing
predictive models on the subsequent performance of those and other
models. Their findings—that using the models to adjust how care is
delivered can alter the baseline assumptions that the models were
"trained" on, often for worse—were detailed in the October 9 online issue of Annals of Internal
Medicine: https://www.acpjournals.org/doi/10.7326/M23-0949.
NEW
YORK, Oct. 9, 2023 /PRNewswire-PRWeb/ -- Models
built on machine learning in health care can be victims of their
own success, according to researchers at the Icahn School of
Medicine and the University of
Michigan.
Their study assessed the impact of implementing predictive
models on the subsequent performance of those and other models.
Their findings—that using the models to adjust how care is
delivered can alter the baseline assumptions that the models were
"trained" on, often for worse—were detailed in the October 9 online issue of Annals of Internal
Medicine: https://www.acpjournals.org/doi/10.7326/M23-0949.
"We wanted to explore what happens when a machine learning model
is deployed in a hospital and allowed to influence physician
decisions for the overall benefit of patients," says first and
corresponding author Akhil Vaid, MD,
Clinical Instructor of Data-Driven and Digital Medicine (D3M), part
of the Department of Medicine at Icahn Mount Sinai. "For example,
we sought to understand the broader consequences when a patient is
spared from adverse outcomes like kidney damage or mortality. AI
models possess the capacity to learn and establish correlations
between incoming patient data and corresponding outcomes, but use
of these models, by definition, can alter these relationships.
Problems arise when these altered relationships are captured back
into medical records."
The study simulated critical care scenarios at two major health
care institutions, the Mount Sinai Health System in New York and Beth Israel Deaconess Medical
Center in Boston, analyzing
130,000 critical care admissions. The researchers investigated
three key scenarios:
- Model retraining after initial use
Current practice suggests retraining models to address performance
degradation over time. Retraining can improve performance initially
by adapting to changing conditions, but the Mount Sinai study shows it can paradoxically
lead to further degradation by disrupting the learned relationships
between presentation and outcome.
- Creating a new model after one has already been in use
Following a model's predictions can save patients from adverse
outcomes such as sepsis. However, death may follow sepsis, and the
model effectively works to pre-vent both. Any new models developed
in the future for prediction of death will now also be subject to
upset relationships as before. Since we do not know the ex-act
relationships between all possible outcomes, any data from patients
with ma-chine-learning influenced care may be inappropriate to use
in training further mod-els.
- Concurrent use of two predictive models
If two models make simultaneous predictions, using one set of
predictions renders the oth-er obsolete. Therefore, predictions
should be based on freshly gathered data, which can be costly or
impractical.
"Our findings reinforce the complexities and challenges of
maintaining predictive model performance in active clinical use,"
says co-senior author Karandeep
Singh, MD, Associate Professor of Learning Health Sciences,
Internal Medicine, Urology, and Information at the University of Michigan. "Model performance can fall
dramatically if patient populations change in their makeup.
However, agreed-upon corrective measures may fall apart completely
if we do not pay attention to what the models are doing—or more
properly, what they are learning from."
"We should not view predictive models as unreliable," says
co-senior author Girish Nadkarni,
MD, MPH, Irene and Dr. Arthur M. Fishberg Professor of Medicine at
Icahn Mount Sinai, Director of The Charles Bronfman Institute of
Personalized Medicine, and System Chief of Data-Driven and Digital
Medicine. "Instead, it's about recognizing that these tools require
regular maintenance, understanding, and contextualization.
Neglecting their performance and impact monitoring can undermine
their effectiveness. We must use predictive models thoughtfully,
just like any other medical tool. Learning health systems must pay
heed to the fact that indiscriminate use of, and updates to, such
models will cause false alarms, unnecessary testing, and increased
costs."
"We recommend that health systems promptly implement a system to
track individuals impacted by machine learning predictions, and
that the relevant governmental agencies issue guidelines," says Dr.
Vaid. "These findings are equally applicable outside of health care
settings and extend to predictive models in general. As such, we
live in a model-eat-model world where any naively deployed model
can disrupt the function of current and future models, and
eventually render itself useless."
The paper is titled "Implications of the Use of Artificial
Intelligence Predictive Models in Health Care Settings: A
Simulation Study."
The remaining authors are Ashwin
Sawant, MD; Mayte
Suarez-Farinas, PhD; Juhee
Lee, MD; Sanjeev Kaul, MD;
Patricia Kovatch, BS; Robert Freeman, RN; Joy
Jiang, BS; Pushkala
Jayaraman, MS; Zahi Fayad,
PhD; Edgar Argulian, MD; Stamatios
Lerakis, MD; Alexander W Charney, MD, PhD; Fei Wang, PhD; Matthew
Levin, MD, PhD; Benjamin
Glicksberg, PhD; Jagat
Narula, MD, PhD; and Ira
Hofer, MD.
The work was supported by Clinical and translational award for
infrastructure UL1TR004419.
-####-
About the Icahn School of Medicine at Mount Sinai
The Icahn School of Medicine at Mount
Sinai is internationally renowned for its outstanding
research, educational, and clinical care programs. It is the sole
academic partner for the eight- member hospitals* of the Mount
Sinai Health System, one of the largest academic health systems in
the United States, providing care
to a large and diverse patient population.
Ranked 14th nationwide in National Institutes of Health (NIH)
funding and among the 99th percentile in research dollars per
investigator according to the Association of American Medical
Colleges, Icahn Mount Sinai has a talented, productive, and
successful faculty. More than 3,000 full-time scientists,
educators, and clinicians work within and across 44 academic
departments and 36 multidisciplinary institutes, a structure that
facilitates tremendous collaboration and synergy. Our emphasis on
translational research and therapeutics is evident in such diverse
areas as genomics/big data, virology, neuroscience, cardiology,
geriatrics, as well as gastrointestinal and liver diseases.
Icahn Mount Sinai offers highly competitive MD, PhD, and
Master's degree programs, with current enrollment of approximately
1,300 students. It has the largest graduate medical education
program in the country, with more than 2,000 clinical residents and
fellows training throughout the Health System. In addition, more
than 550 postdoctoral research fellows are in training within the
Health System.
A culture of innovation and discovery permeates every Icahn
Mount Sinai program. Mount Sinai's
technology transfer office, one of the largest in the country,
partners with faculty and trainees to pursue optimal
commercialization of intellectual property to ensure that
Mount Sinai discoveries and
innovations translate into healthcare products and services that
benefit the public.
Icahn Mount Sinai's commitment to breakthrough science and
clinical care is enhanced by academic affiliations that supplement
and complement the School's programs.
Through the Mount Sinai Innovation Partners (MSIP), the Health
System facilitates the real-world application and commercialization
of medical breakthroughs made at Mount
Sinai. Additionally, MSIP develops research partnerships
with industry leaders such as Merck & Co., AstraZeneca, Novo
Nordisk, and others.
The Icahn School of Medicine at Mount
Sinai is located in New York
City on the border between the Upper East Side and East
Harlem, and classroom teaching takes place on a campus facing
Central Park. Icahn Mount Sinai's location offers many
opportunities to interact with and care for diverse communities.
Learning extends well beyond the borders of our physical campus, to
the eight hospitals of the Mount Sinai Health System, our academic
affiliates, and globally.
- Mount Sinai Health System member hospitals: The Mount Sinai
Hospital; Mount Sinai Beth Israel; Mount Sinai Brooklyn; Mount
Sinai Morningside; Mount Sinai Queens; Mount Sinai South Nassau;
Mount Sinai West; and New York Eye and Ear Infirmary of
Mount Sinai.
Media Contact
Karin Eskenazi, Mount Sinai, 332-257-1538,
karin.eskenazi@mssm.edu, mountsinai.org
View original content to download
multimedia:https://www.prweb.com/releases/what-is-the-impact-of-predictive-ai-in-the-health-care-setting-findings-underscore-the-need-to-track-individuals-affected-by-machine-learning-predictions-301951235.html
SOURCE Mount Sinai
