Machine Learning Faces a Reckoning in Health Research

Megan Scudellari, Machine Learning Faces a Reckoning in Health Research, IEEE Spectrum, 29 Mar 2021.

In a paper describing her team’s analysis of 511 other papers, Ghassemi’s team reported that machine learning papers in healthcare were reproducible far less often than in other machine learning subfields. The group’s findings were published this week in the journal Science Translational Medicine. And in a systematic review published in Nature Machine Intelligence, 85 percent of studies using machine learning to detect COVID-19 in chest scans failed a reproducibility and quality check, and none of the models was near ready for use in clinics, the authors say.

“We were surprised at how far the models are from being ready for deployment,” says Derek Driggs, co-author of the paper from the lab of Carola-Bibiane Schönlieb at the University of Cambridge. “There were many flaws that should not have existed.”