[Skip to Content]
[Skip to Content Landing]

Artificial Intelligence in Breast Cancer ScreeningEvaluation of FDA Device Regulation and Future Recommendations

To identify the key insights or developments described in this article
1 Credit CME
Abstract

Importance  Contemporary approaches to artificial intelligence (AI) based on deep learning have generated interest in the application of AI to breast cancer screening (BCS). The US Food and Drug Administration (FDA) has approved several next-generation AI products indicated for BCS in recent years; however, questions regarding their accuracy, appropriate use, and clinical utility remain.

Objectives  To describe the current FDA regulatory process for AI products, summarize the evidence used to support FDA clearance and approval of AI products indicated for BCS, consider the advantages and limitations of current regulatory approaches, and suggest ways to improve the current system.

Evidence Review  Premarket notifications and other publicly available documents used for FDA clearance and approval of AI products indicated for BCS from January 1, 2017, to December 31, 2021.

Findings  Nine AI products indicated for BCS for identification of suggestive lesions and mammogram triage were included. Most of the products had been cleared through the 510(k) pathway, and all clearances were based on previously collected retrospective data; 6 products used multicenter designs; 7 products used enriched data; and 4 lacked details on whether products were externally validated. Test performance measures, including sensitivity, specificity, and area under the curve, were the main outcomes reported. Most of the devices used tissue biopsy as the criterion standard for BCS accuracy evaluation. Other clinical outcome measures, including cancer stage at diagnosis and interval cancer detection, were not reported for any of the devices.

Conclusions and Relevance  The findings of this review suggest important gaps in reporting of data sources, data set type, validation approach, and clinical utility assessment. As AI-assisted reading becomes more widespread in BCS and other radiologic examinations, strengthened FDA evidentiary regulatory standards, development of postmarketing surveillance, a focus on clinically meaningful outcomes, and stakeholder engagement will be critical for ensuring the safety and efficacy of these products.

Sign in to take quiz and track your certificates

Buy This Activity

JN Learning™ is the home for CME and MOC from the JAMA Network. Search by specialty or US state and earn AMA PRA Category 1 Credit(s)™ from articles, audio, Clinical Challenges and more. Learn more about CME/MOC

CME Disclosure Statement: Unless noted, all individuals in control of content reported no relevant financial relationships. If applicable, all relevant financial relationships have been mitigated.

Article Information

Accepted for Publication: September 13, 2022.

Published Online: November 7, 2022. doi:10.1001/jamainternmed.2022.4969

Corresponding Author: Ilana B. Richman, MD, MHS, Yale School of Medicine, 367 Cedar St, Room 301a, New Haven, CT 06511 (ilana.richman@yale.edu).

Author Contributions: Mr Potnis and Dr Richman had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Potnis, Ross, Gross, Richman.

Acquisition, analysis, or interpretation of data: Potnis, Ross, Aneja, Richman.

Drafting of the manuscript: Potnis, Richman.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Richman.

Obtained funding: Richman.

Supervision: Gross, Richman.

Conflict of Interest Disclosures: Mr Potnis reported grants from the US National Institutes of Health (NIH) and grants from the American Society of Hematology outside the submitted work. Dr Ross reported grants from the US Food and Drug Administration, Johnson & Johnson, the Medical Devices Innovation Consortium, Agency for Healthcare Research and Quality (AHRQ), grants from the NIH and Arnold Ventures outside the submitted work; and serving as an expert witness at the request of Relator’s attorneys in a qui tam suit alleging violations of the False Claims Act and Anti-Kickback Statute against Biogen, all outside the submitted work. Dr Aneja reported grants from the US National Science Foundation, from the US National Cancer Institute (NCI), Patterson Family Trust, American Society of Clinical Oncology, the Radiation Society of North America, the American Cancer Society, and AHRQ, and nonfinancial support from Amazon, all outside the submitted work. Dr Gross reported grants from Johnson & Johnson, funding from AstraZeneca, and personal fees from Genentech outside the submitted work. Dr Richman reported funding from the NCI and salary support from the Centers for Medicare & Medicaid Services outside of the submitted work. No other disclosures were reported.

Funding/Support: Dr Richman received funding for this study from the US National Institutes of Health (No. K08 CA248725).

Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

References
1.
Freeman  K , Geppert  J , Stinton  C ,  et al.  Use of artificial intelligence for image analysis in breast cancer screening programmes: systematic review of test accuracy.   BMJ. 2021;374:n1872. doi:10.1136/bmj.n1872 PubMedGoogle ScholarCrossref
2.
Lamb  LR , Lehman  CD , Gastounioti  A , Conant  EF , Bahl  M .  Artificial Intelligence (AI) for screening mammography, from the AJR Special Series on AI Applications.   AJR Am J Roentgenol. 2022;219(3):369-380. doi:10.2214/AJR.21.27071 PubMedGoogle ScholarCrossref
3.
Anderson  AW , Marinovich  ML , Houssami  N ,  et al.  Independent external validation of artificial intelligence algorithms for automated interpretation of screening mammography: a systematic review.   J Am Coll Radiol. 2022;19(2 Pt A):259-273. doi:10.1016/j.jacr.2021.11.008 PubMedGoogle ScholarCrossref
4.
Houssami  N , Kirkpatrick-Jones  G , Noguchi  N , Lee  CI .  Artificial Intelligence (AI) for the early detection of breast cancer: a scoping review to assess AI’s potential in breast screening practice.   Expert Rev Med Devices. 2019;16(5):351-362. doi:10.1080/17434440.2019.1610387PubMedGoogle ScholarCrossref
5.
Wu  E , Wu  K , Daneshjou  R , Ouyang  D , Ho  DE , Zou  J .  How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals.   Nat Med. 2021;27(4):582-584. doi:10.1038/s41591-021-01312-x PubMedGoogle ScholarCrossref
6.
Benjamens  S , Dhunnoo  P , Meskó  B .  The state of artificial intelligence-based FDA-approved medical devices and algorithms: an online database.   NPJ Digit Med. 2020;3(1):118. doi:10.1038/s41746-020-00324-0 PubMedGoogle ScholarCrossref
7.
Meyers  PH , Nice  CM  Jr , Becker  HC , Nettleton  WJ  Jr , Sweeney  JW , Meckstroth  GR .  Automated computer analysis of radiographic images.   Radiology. 1964;83(6):1029-1034. doi:10.1148/83.6.1029 PubMedGoogle ScholarCrossref
8.
Winsberg  F , Elkin  M , Josiah Macy  J , Bordaz  V , Weymouth  W .  Detection of radiographic abnormalities in mammograms by means of optical scanning and computer analysis.   Radiology. 1967;89(2):211-215. doi:10.1148/89.2.211Google ScholarCrossref
9.
Muralidhar  GS , Haygood  TM , Stephens  TW , Whitman  GJ , Bovik  AC , Markey  MK .  Computer-aided detection of breast cancer—have all bases been covered?   Breast Cancer (Auckl). 2008;2:5-9.PubMedGoogle Scholar
10.
US Food and Drug Administration. M1000 ImageChecker: premarket approval. 2022. Accessed February 5, 2022. https://www.accessdata.fda.gov/scrIpts/cdrh/cfdocs/cfpma/pma.cfm?id=P970058
11.
Fenton  JJ , Foote  SB , Green  P , Baldwin  L-M .  Diffusion of computer-aided mammography after mandated Medicare coverage.   Arch Intern Med. 2010;170(11):987-989. doi:10.1001/archinternmed.2010.104 PubMedGoogle ScholarCrossref
12.
Geras  KJ , Mann  RM , Moy  L .  Artificial intelligence for mammography and digital breast tomosynthesis: current concepts and future perspectives.   Radiology. 2019;293(2):246-259. doi:10.1148/radiol.2019182627 PubMedGoogle ScholarCrossref
13.
Gross  CP , Long  JB , Ross  JS ,  et al.  The cost of breast cancer screening in the Medicare population.   JAMA Intern Med. 2013;173(3):220-226. doi:10.1001/jamainternmed.2013.1397 PubMedGoogle ScholarCrossref
14.
Lehman  CD , Wellman  RD , Buist  DS , Kerlikowske  K , Tosteson  AN , Miglioretti  DL ; Breast Cancer Surveillance Consortium.  Diagnostic accuracy of digital screening mammography with and without computer-aided detection.   JAMA Intern Med. 2015;175(11):1828-1837. doi:10.1001/jamainternmed.2015.5231 PubMedGoogle ScholarCrossref
15.
Fenton  JJ , Taplin  SH , Carney  PA ,  et al.  Influence of computer-aided detection on performance of screening mammography.   N Engl J Med. 2007;356(14):1399-1409. doi:10.1056/NEJMoa066099 PubMedGoogle ScholarCrossref
16.
Soffer  S , Ben-Cohen  A , Shimon  O , Amitai  MM , Greenspan  H , Klang  E .  Convolutional neural networks for radiologic images: a radiologist’s guide.   Radiology. 2019;290(3):590-606. doi:10.1148/radiol.2018180547 PubMedGoogle ScholarCrossref
17.
Kohli  A , Jha  S .  Why CAD failed in mammography.   J Am Coll Radiol. 2018;15(3 Pt B):535-537. doi:10.1016/j.jacr.2017.12.029 PubMedGoogle ScholarCrossref
18.
Chan  HP , Samala  RK , Hadjiiski  LM .  CAD and AI for breast cancer-recent development and challenges.   Br J Radiol. 2020;93(1108):20190580. doi:10.1259/bjr.20190580 PubMedGoogle ScholarCrossref
19.
Joel  MZ , Umrao  S , Chang  E ,  et al.  Using adversarial images to assess the robustness of deep learning models trained on diagnostic images in oncology.   JCO Clin Cancer Inform. 2022;6:e2100170. doi:10.1200/CCI.21.00170 PubMedGoogle ScholarCrossref
20.
Kann  BH , Hicks  DF , Payabvash  S ,  et al.  Multi-institutional validation of deep learning for pretreatment identification of extranodal extension in head and neck squamous cell carcinoma.   J Clin Oncol. 2020;38(12):1304-1311. doi:10.1200/JCO.19.02031 PubMedGoogle ScholarCrossref
21.
US Food and Drug Administration. Software as a Medical Device (SaMD). 2018. Accessed November 18, 2021.https://www.fda.gov/medical-devices/digital-health-center-excellence/software-medical-device-samd
22.
US Food and Drug Administration. Overview of device regulation. 2020. Accessed November 18, 2021. https://www.fda.gov/medical-devices/device-advice-comprehensive-regulatory-assistance/overview-device-regulation
23.
US Food and Drug Administration. PMA Clinical Studies. 2020. Accessed January 22, 2022. https://www.fda.gov/medical-devices/premarket-approval-pma/pma-clinical-studies
24.
US Food and Drug Administration. De Novo Classification Request. 2022. Accessed July 30, 2022. https://www.fda.gov/medical-devices/premarket-submissions-selecting-and-preparing-correct-submission/de-novo-classification-request
25.
US Food and Drug Administration. The 510(k) Program: Evaluating Substantial Equivalence in Premarket Notifications [510(k)]. 2014. Accessed August 1, 2022. https://www.fda.gov/media/82395/download
26.
US Food and Drug Administration. Clinical Performance Assessment: Considerations for Computer-Assisted Detection Devices Applied to Radiology Images and Radiology Device Data in Premarket Notification (510(k)) Submissions. 2020. Accessed August 1, 2022. https://www.fda.gov/media/77642/download
27.
Pew Charitable Trusts. How FDA Regulates Artificial Intelligence in Medical Products. 2021. Accessed October 11, 2022. https://www.pewtrusts.org/en/research-and-analysis/issue-briefs/2021/08/how-fda-regulates-artificial-intelligence-in-medical-products
28.
US Food and Drug Administration. Digital Health Software Precertification (Pre-Cert) Program. 2021. Accessed August 3, 2022. https://www.fda.gov/medical-devices/digital-health-center-excellence/digital-health-software-precertification-pre-cert-program
29.
US Food and Drug Administration. Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) Action Plan. 2021. Accessed February 5, 2022. https://www.fda.gov/files/medical%20devices/published/US-FDA-Artificial-Intelligence-and-Machine-Learning-Discussion-Paper.pdf
30.
US Food and Drug Administration. Artificial Intelligence and Machine Learning in Software as a Medical Device. 2021. Accessed February 5, 2022. https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-and-machine-learning-software-medical-device
31.
American College of Radiology. AI Central. 2022. Accessed January 28, 2022. https://aicentral.acrdsi.org/
32.
Adhikari  S , Normand  S-L , Bloom  J , Shahian  D , Rose  S .  Revisiting performance metrics for prediction with rare outcomes.   Stat Methods Med Res. 2021;30(10):2352-2366. doi:10.1177/09622802211038754 PubMedGoogle ScholarCrossref
33.
Freeman  K , Geppert  J , Stinton  C ,  et al. Use of artificial intelligence for image analysis in breast cancer screening: rapid review and evidence map. UK National Screening Committee; 2022. Accessed on October 11, 2022. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/1057279/UK_NSC_evidence_summary_-_the_use_of_AI_for_mammographic_image_analysis_in_breast_cancer_screening.pdf
34.
Marinovich  ML , Wylie  E , Lotter  W ,  et al.  Artificial intelligence (AI) to enhance breast cancer screening: protocol for population-based cohort study of cancer detection.   BMJ Open. 2022;12(1):e054005. doi:10.1136/bmjopen-2021-054005 PubMedGoogle ScholarCrossref
35.
US Food and Drug Administration. Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device. 2019. Accessed August 1, 2022. https://www.fda.gov/media/122535/download
36.
Schaffter  T , Buist  DSM , Lee  CI ,  et al; and the DM DREAM Consortium.  Evaluation of combined artificial intelligence and radiologist assessment to interpret screening mammograms.   JAMA Netw Open. 2020;3(3):e200265-e200265. doi:10.1001/jamanetworkopen.2020.0265 PubMedGoogle ScholarCrossref
37.
Lee  RS , Gimenez  F , Hoogi  A , Miyake  KK , Gorovoy  M , Rubin  DL .  A curated mammography data set for use in computer-aided detection and diagnosis research.   Sci Data. 2017;4(1):170177. doi:10.1038/sdata.2017.177 PubMedGoogle ScholarCrossref
38.
McKinney  SM , Sieniek  M , Godbole  V ,  et al.  International evaluation of an AI system for breast cancer screening.   Nature. 2020;577(7788):89-94. doi:10.1038/s41586-019-1799-6 PubMedGoogle ScholarCrossref
39.
Kaushal  A , Altman  R , Langlotz  C .  Geographic distribution of us cohorts used to train deep learning algorithms.   JAMA. 2020;324(12):1212-1213. doi:10.1001/jama.2020.12067 PubMedGoogle ScholarCrossref
40.
US Food and Drug Administration. Diversity Plans to Improve Enrollment of Participants from Underrepresented Racial and Ethnic Populations in Clinical Trials Guidance for Industry; 2022. Accessed October 11, 2022. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/diversity-plans-improve-enrollment-participants-underrepresented-racial-and-ethnic-populations
41.
Kerlikowske  K , Su  Y-R , Sprague  BL ,  et al.  Association of screening with digital breast tomosynthesis vs digital mammography with risk of interval invasive and advanced breast cancer.   JAMA. 2022;327(22):2220-2230. doi:10.1001/jama.2022.7672 PubMedGoogle ScholarCrossref
42.
Lee  C , McCaskill-Stevens  W .  Tomosynthesis Mammographic Imaging Screening Trial (TMIST): an invitation and opportunity for the national medical association community to shape the future of precision screening for breast cancer.   J Natl Med Assoc. 2020;112(6):613-618. doi:10.1016/j.jnma.2020.05.021 PubMedGoogle ScholarCrossref
43.
McCarthy  AM , Barlow  WE , Conant  EF ,  et al; PROSPR Consortium.  Breast cancer with a poor prognosis diagnosed after screening mammography with negative results.   JAMA Oncol. 2018;4(7):998-1001. doi:10.1001/jamaoncol.2018.0352 PubMedGoogle ScholarCrossref
44.
Niraula  S , Biswanger  N , Hu  P , Lambert  P , Decker  K .  Incidence, characteristics, and outcomes of interval breast cancers compared with screening-detected breast cancers.   JAMA Netw Open. 2020;3(9):e2018179-e2018179. doi:10.1001/jamanetworkopen.2020.18179 PubMedGoogle ScholarCrossref
45.
Jatoi  I , Pinsky  PF .  Breast cancer screening trials: endpoints and overdiagnosis.   J Natl Cancer Inst. 2021;113(9):1131-1135. doi:10.1093/jnci/djaa140 PubMedGoogle ScholarCrossref
46.
Institute of Medicine. Medical Devices and the Public's Health: The FDA 510(k) Clearance Process at 35 Years. 2011. Accessed October 11, 2022. https://nap.nationalacademies.org/catalog/13150/medical-devices-and-the-publics-health-the-fda-510k-clearance
AMA CME Accreditation Information

Credit Designation Statement: The American Medical Association designates this Journal-based CME activity activity for a maximum of 1.00  AMA PRA Category 1 Credit(s)™. Physicians should claim only the credit commensurate with the extent of their participation in the activity.

Successful completion of this CME activity, which includes participation in the evaluation component, enables the participant to earn up to:

  • 1.00 Medical Knowledge MOC points in the American Board of Internal Medicine's (ABIM) Maintenance of Certification (MOC) program;;
  • 1.00 Self-Assessment points in the American Board of Otolaryngology – Head and Neck Surgery’s (ABOHNS) Continuing Certification program;
  • 1.00 MOC points in the American Board of Pediatrics’ (ABP) Maintenance of Certification (MOC) program;
  • 1.00 Lifelong Learning points in the American Board of Pathology’s (ABPath) Continuing Certification program; and
  • 1.00 credit toward the CME [and Self-Assessment requirements] of the American Board of Surgery’s Continuous Certification program

It is the CME activity provider's responsibility to submit participant completion information to ACCME for the purpose of granting MOC credit.

Close
Want full access to the AMA Ed Hub?
After you sign up for AMA Membership, make sure you sign in or create a Physician account with the AMA in order to access all learning activities on the AMA Ed Hub
Buy this activity
Close
Want full access to the AMA Ed Hub?
After you sign up for AMA Membership, make sure you sign in or create a Physician account with the AMA in order to access all learning activities on the AMA Ed Hub
Buy this activity
Close
With a personal account, you can:
  • Access free activities and track your credits
  • Personalize content alerts
  • Customize your interests
  • Fully personalize your learning experience
Education Center Collection Sign In Modal Right
Close

Name Your Search

Save Search
With a personal account, you can:
  • Access free activities and track your credits
  • Personalize content alerts
  • Customize your interests
  • Fully personalize your learning experience
Close
Close

Lookup An Activity

or

My Saved Searches

You currently have no searches saved.

Close

My Saved Courses

You currently have no courses saved.

Close