Deep Generative Models for High-Resolution Synthetic Retinal Image Generation of AMD | Macular Diseases | JN Learning | AMA Ed Hub [Skip to Content]
[Skip to Content Landing]

Assessment of Deep Generative Models for High-Resolution Synthetic Retinal Image Generation of Age-Related Macular Degeneration

Educational Objective
To develop deep learning techniques for synthesizing high-resolution realistic fundus images serving as proxy data sets for use by retinal specialists and deep learning machines.
1 Credit CME
Key Points

Question  Can deep learning be used to synthesize fundus images of age-related macular degeneration (AMD) that appear realistic to retinal specialists?

Findings  In this study of fundus images from 4613 study participants from the Age-Related Eye Disease Study and 133 821 fundus images, the ability of 2 retinal specialists to distinguish real from synthetic fundus images of varying stages of AMD was close to chance, and their diagnostic accuracy similar for real and synthetic images. Machines trained with only synthetic images showed performance nearing that resulting from training on real images.

Meaning  These findings suggest that deep-learning–synthesized fundus images of AMD are realistic and could be used for education of humans across various levels of expertise and for machine training.

Abstract

Importance  Deep learning (DL) used for discriminative tasks in ophthalmology, such as diagnosing diabetic retinopathy or age-related macular degeneration (AMD), requires large image data sets graded by human experts to train deep convolutional neural networks (DCNNs). In contrast, generative DL techniques could synthesize large new data sets of artificial retina images with different stages of AMD. Such images could enhance existing data sets of common and rare ophthalmic diseases without concern for personally identifying information to assist medical education of students, residents, and retinal specialists, as well as for training new DL diagnostic models for which extensive data sets from large clinical trials of expertly graded images may not exist.

Objective  To develop DL techniques for synthesizing high-resolution realistic fundus images serving as proxy data sets for use by retinal specialists and DL machines.

Design, Setting, and Participants  Generative adversarial networks were trained on 133 821 color fundus images from 4613 study participants from the Age-Related Eye Disease Study (AREDS), generating synthetic fundus images with and without AMD. We compared retinal specialists’ ability to diagnose AMD on both real and synthetic images, asking them to assess image gradability and testing their ability to discern real from synthetic images. The performance of AMD diagnostic DCNNs (referable vs not referable AMD) trained on either all-real vs all-synthetic data sets was compared.

Main Outcomes and Measures  Accuracy of 2 retinal specialists (T.Y.A.L. and K.D.P.) for diagnosing and distinguishing AMD on real vs synthetic images and diagnostic performance (area under the curve) of DL algorithms trained on synthetic vs real images.

Results  The diagnostic accuracy of 2 retinal specialists on real vs synthetic images was similar. The accuracy of diagnosis as referable vs nonreferable AMD compared with certified human graders for retinal specialist 1 was 84.54% (error margin, 4.06%) on real images vs 84.12% (error margin, 4.16%) on synthetic images and for retinal specialist 2 was 89.47% (error margin, 3.45%) on real images vs 89.19% (error margin, 3.54%) on synthetic images. Retinal specialists could not distinguish real from synthetic images, with an accuracy of 59.50% (error margin, 3.93%) for retinal specialist 1 and 53.67% (error margin, 3.99%) for retinal specialist 2. The DCNNs trained on real data showed an area under the curve of 0.9706 (error margin, 0.0029), and those trained on synthetic data showed an area under the curve of 0.9235 (error margin, 0.0045).

Conclusions and Relevance  Deep learning–synthesized images appeared to be realistic to retinal specialists, and DCNNs achieved diagnostic performance on synthetic data close to that for real images, suggesting that DL generative techniques hold promise for training humans and machines.

Sign in to take quiz and track your certificates

Buy This Activity

JN Learning™ is the home for CME and MOC from the JAMA Network. Search by specialty or US state and earn AMA PRA Category 1 CME Credit™ from articles, audio, Clinical Challenges and more. Learn more about CME/MOC

Article Information

Accepted for Publication: October 23, 2018.

Published Online: January 10, 2019. doi:10.1001/jamaophthalmol.2018.6156

Correction: This article was corrected on February 14, 2019, to fix errors in the text, Table 2, and the supplement.

Corresponding Author: Neil M. Bressler, MD, Wilmer Eye Institute, Johns Hopkins University, 600 N Wolfe St, Maumenee 752, Baltimore, MD 21287-9227 (nmboffice@jhmi.edu).

Author Contributions: Dr Burlina and Mr Joshi had full access to all the data in the study and take full responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Burlina, Joshi.

Study concept and design: Bressler.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Burlina, Joshi, Pacheco, Liu.

Critical revision of the manuscript for important intellectual content: Burlina, Liu, Bressler.

Statistical analysis: Burlina, Joshi.

Obtained funding: Burlina, Bressler.

Administrative, technical, or material support: Burlina, Joshi, Bressler.

Supervision: Burlina.

Conflict of Interest: Dr Burlina reported a patent to a system and method for detecting and classifying severity of retinal disease issued and a patent to a system and method for automated detection of age-related macular degeneration issued. Dr Bressler reported grants from Bayer, Genentech/Roche, Novartis, the National Institutes of Health, and Samsung Bioepis outside the submitted work and a patent for automated detection of retinal diseases issued. No other disclosures were reported.

Funding/Support: This work was supported in part by award R21EY024310 from the National Eye Institute (Drs Burlina and Bressler), the Johns Hopkins Applied Physics Laboratory, the James P. Gills Professorship, and unrestricted research funds to the Johns Hopkins University School of Medicine Retina Division for Macular Degeneration and Related Diseases Research.

Role of the Funder/Sponsor: The National Eye Institute and Johns Hopkins University had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: Dr Bressler is the editor of JAMA Ophthalmology, but he was not involved in the review process or the acceptance of the manuscript.

Additional Information: Answer to the Figure: The real images with referable age-related macular degeneration (AMD) (ie, with the intermediate or advanced stage of AMD as defined in the Age-Related Eye Disease Study) are A, C, and F. The synthetic images with referable AMD are B, D, and E.

References
1.
Gulshan  V, Peng  L, Coram  M,  et al.  Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs.  JAMA. 2016;316(22):2402-2410. doi:10.1001/jama.2016.17216PubMedGoogle ScholarCrossref
2.
Quellec  G, Charrière  K, Boudi  Y, Cochener  B, Lamard  M.  Deep image mining for diabetic retinopathy screening.  Med Image Anal. 2017;39:178-193. doi:10.1016/j.media.2017.04.012PubMedGoogle ScholarCrossref
3.
Gargeya  R, Leng  T.  Automated identification of diabetic retinopathy using deep learning.  Ophthalmology. 2017;124(7):962-969. doi:10.1016/j.ophtha.2017.02.008PubMedGoogle ScholarCrossref
4.
Ting  DSW, Cheung  CY, Lim  G,  et al.  Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes.  JAMA. 2017;318(22):2211-2223. doi:10.1001/jama.2017.18152PubMedGoogle ScholarCrossref
5.
De Fauw  J, Ledsam  JR, Romera-Paredes  B,  et al.  Clinically applicable deep learning for diagnosis and referral in retinal disease.  Nat Med. 2018;24(9):1342-1350. doi:10.1038/s41591-018-0107-6PubMedGoogle ScholarCrossref
6.
Brown  JM, Campbell  JP, Beers  A,  et al; Imaging and Informatics in Retinopathy of Prematurity (i-ROP) Research Consortium.  Automated diagnosis of plus disease in retinopathy of prematurity using deep convolutional neural networks.  JAMA Ophthalmol. 2018;136(7):803-810. doi:10.1001/jamaophthalmol.2018.1934PubMedGoogle ScholarCrossref
7.
Burlina  P, Pacheco  KD, Joshi  N, Freund  DE, Bressler  NM.  Comparing humans and deep learning performance for grading AMD: A study in using universal deep features and transfer learning for automated AMD analysis.  Comput Biol Med. 2017;82:80-86. doi:10.1016/j.compbiomed.2017.01.018PubMedGoogle ScholarCrossref
8.
Burlina  P, Freund  DE, Joshi  N, Wolfson  Y, Bressler  NM.  Detection of age-related macular degeneration via deep learning. In: 2016 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, ISBI 2016 - Proceedings. Piscataway, NJ: IEEE; 2016:184-188.
9.
Burlina  PM, Joshi  N, Pekala  M, Pacheco  KD, Freund  DE, Bressler  NM.  Automated grading of age-related macular degeneration from color fundus images using deep convolutional neural networks.  JAMA Ophthalmol. 2017;135(11):1170-1176. doi:10.1001/jamaophthalmol.2017.3782PubMedGoogle ScholarCrossref
10.
Burlina  P, Joshi  N, Pacheco  KD, Freund  DE, Kong  J, Bressler  NM.  Utility of deep learning methods for referability classification of age-related macular degeneration.  JAMA Ophthalmol. 2018;136(11):1305-1307. doi:10.1001/jamaophthalmol.2018.3799PubMedGoogle ScholarCrossref
11.
Grassmann  F, Mengelkamp  J, Brandl  C,  et al.  A deep learning algorithm for prediction of age-related eye disease study severity scale for age-related macular degeneration from color fundus photography.  Ophthalmology. 2018;125(9):1410-1420. doi:10.1016/j.ophtha.2018.02.037PubMedGoogle ScholarCrossref
12.
Burlina  PM, Joshi  N, Pacheco  KD, Freund  DE, Kong  J, Bressler  NM.  Use of deep learning for detailed severity characterization and estimation of 5-year risk among patients with age-related macular degeneration [published online September 14, 2018].  JAMA Ophthalmol. doi:10.1001/jamaophthalmol.2018.4118Google Scholar
13.
Kankanahalli  S, Burlina  PM, Wolfson  Y, Freund  DE, Bressler  NM.  Automated classification of severity of age-related macular degeneration from fundus photographs.  Invest Ophthalmol Vis Sci. 2013;54(3):1789-1796. doi:10.1167/iovs.12-10928PubMedGoogle ScholarCrossref
14.
Burlina  P, Freund  DE, Dupas  B, Bressler  N.  Automatic screening of age-related macular degeneration and retinal abnormalities.  Conf Proc IEEE Eng Med Biol Soc. 2011;2011:3962-3966. PubMedGoogle Scholar
15.
Ting  DSW, Liu  Y, Burlina  P, Xu  X, Bressler  NM, Wong  TY.  AI for medical imaging goes deep.  Nat Med. 2018;24(5):539-540. doi:10.1038/s41591-018-0029-3PubMedGoogle ScholarCrossref
16.
Goodfellow  I, Pouget-Abadie  J, Mirza  M,  et al.  Generative adversarial nets.  Adv Neural Inf Process Syst. 2014;2672-2680.Google Scholar
17.
Karras  T, Aila  T, Laine  S, Lehtinen  J. Progressive growing of GANs for improved quality, stability, and variation. Preprint. Published online October 27, 2017. arXiv 1710:10196.
18.
Velez-Montoya  R, Oliver  SC, Olson  JL, Fine  SL, Quiroz-Mercado  H, Mandava  N.  Current knowledge and trends in age-related macular degeneration: genetics, epidemiology, and prevention.  Retina. 2014;34(3):423-441. doi:10.1097/IAE.0000000000000036PubMedGoogle ScholarCrossref
19.
Bressler  NM.  Age-related macular degeneration is the leading cause of blindness....  JAMA. 2004;291(15):1900-1901. doi:10.1001/jama.291.15.1900PubMedGoogle ScholarCrossref
20.
 Age-Related Eye Disease Study Research Group. The age-related eye disease study system for classifying age-related macular degeneration from stereoscopic color fundus photographs: the age-related eye disease study report number 6.  Am J Ophthalmol. 2001;132(5):668-681. PubMedGoogle Scholar
21.
Bressler  NM, Bressler  SB, Congdon  NG,  et al; Age-Related Eye Disease Study Research Group.  Potential public health impact of Age-Related Eye Disease Study results: AREDS report no. 11.  Arch Ophthalmol. 2003;121(11):1621-1624. doi:10.1001/archopht.121.11.1621PubMedGoogle ScholarCrossref
22.
Age-Related Eye Disease Study Research Group.  The Age-Related Eye Disease Study (AREDS): design implications. AREDS report no. 1.  Control Clin Trials. 1999;20(6):573-600. doi:10.1016/S0197-2456(99)00031-8PubMedGoogle ScholarCrossref
23.
He  K, Zhang  X, Ren  S, Sun  J.  Deep residual learning for image recognition.  CVPR. 2016:771-778.Google Scholar
24.
Costa  P, Galdran  A, Meyer  MI,  et al.  End to end adversarial retinal image synthesis.  IEEE Trans Med Imaging. 2018;37(3):781-791. doi:10.1109/TMI.2017.2759102PubMedGoogle ScholarCrossref
25.
Yi  X, Babyn  P.  Sharpness-aware low-dose CT denoising using conditional generative adversarial network.  J Digit Imaging. 2018;31(5):655-669. doi:10.1007/s10278-018-0056-0PubMedGoogle ScholarCrossref
26.
Nie  D, Trullo  R, Lian  J,  et al. Medical Image Synthesis with Deep Convolutional Adversarial Networks. IEEE Transactions on Biomedical Engineering, 2018: https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=8310638. Accessed November 19, 2018.
27.
Schlegl  T, Seeböck  P, Waldstein  SM, Schmidt-Erfurth  U, Langs  G. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In: Niethammer M, Styner M, Aylward S, et al, eds. International Conference on Information Processing in Medical Imaging; 25th International Conference, IPMI 2017, Boone, NC, USA, June 25-30, 2017, Proceedings. New York, NY: Springer International Publishing; 2017:146-157.
28.
Mahapatra  D, Bhavna  A, Suman  S, Rahil  G. Deformable medical image registration using generative adversarial networks. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018). Piscataway, NJ: IEEE; 2018:1449-1453.
29.
Baur  C, Albarqouni  S, Navab  N. MelanoGANs: high resolution skin lesion synthesis with GANs. Preprint. Published online April 12, 2018. arXiv. 1804.04338.
30.
Pekala  M, Joshi  N, Freund  DE, Bressler  NM, Cabrera Debuc  D, Burlina  P. Deep learning based retinal OCT segmentation. Preprint. Published online January 29, 2018. arXiv. 1801.
31.
Jain  AK, Arun  R, Salil  P.  An introduction to biometric recognition.  IEEE Trans Circ Syst Video Tech. 2004;14(1):4-20. doi:10.1109/TCSVT.2003.818349Google ScholarCrossref
32.
Bowles  N. After Cambridge Analytica, privacy experts get to say ‘I told you so.’ New York Times. April 12, 2018. https://www.nytimes.com/2018/04/12/technology/privacy-researchers-facebook.html. Accessed July 24, 2018.
If you are not a JN Learning subscriber, you can either:
Subscribe to JN Learning for one year
Buy this activity
jn-learning_Modal_Multimedia_LoginSubscribe_Purchase
Close
If you are not a JN Learning subscriber, you can either:
Subscribe to JN Learning for one year
Buy this activity
jn-learning_Modal_Multimedia_LoginSubscribe_Purchase
Close
With a personal account, you can:
  • Access free activities and track your credits
  • Personalize content alerts
  • Customize your interests
  • Fully personalize your learning experience
Education Center Collection Sign In Modal Right
Close

Name Your Search

Save Search
Close
With a personal account, you can:
  • Track your credits
  • Personalize content alerts
  • Customize your interests
  • Fully personalize your learning experience
jn-learning_Modal_SaveSearch_NoAccess_Purchase
Close

Lookup An Activity

or

Close

My Saved Searches

You currently have no searches saved.

Close
With a personal account, you can:
  • Access free activities and track your credits
  • Personalize content alerts
  • Customize your interests
  • Fully personalize your learning experience
Education Center Collection Sign In Modal Right
Close