Want to take quizzes and track your credits?
In recent years, many new clinical diagnostic tools have been developed using complicated machine learning methods. Irrespective of how a diagnostic tool is derived, it must be evaluated using a 3-step process of deriving, validating, and establishing the clinical effectiveness of the tool. Machine learning–based tools should also be assessed for the type of machine learning model used and its appropriateness for the input data type and data set size. Machine learning models also generally have additional prespecified settings called hyperparameters, which must be tuned on a data set independent of the validation set. On the validation set, the outcome against which the model is evaluated is termed the reference standard. The rigor of the reference standard must be assessed, such as against a universally accepted gold standard or expert grading.
Sign in to take quiz and track your certificates
JN Learning™ is the home for CME and MOC from the JAMA Network. Search by specialty or US state and earn AMA PRA Category 1 CME Credit™ from articles, audio, Clinical Challenges and more. Learn more about CME/MOC
Corresponding Author: Yun Liu, PhD, Google Health, 3400 Hillview Ave, Palo Alto, CA 94304 (firstname.lastname@example.org).
Author Contributions: Dr Liu had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Drs Liu and Chen contributed equally to this article.
Concept and design: All authors.Acquisition, analysis, or interpretation of data: Liu, Chen, Peng.Drafting of the manuscript: Liu, Chen.Critical revision of the manuscript for important intellectual content: All authors.Administrative, technical, or material support: Liu, Chen, Peng.Supervision: Liu, Chen, Peng.Other - machine learning expertise: Liu, Chen, Krause.
Conflict of Interest Disclosures: Dr Liu reported holding a patent in machine learning to each of the following: analyzing retinal fundus photographs and another for analyzing histopathology slides (status pending or granted), analyzing skin conditions (status pending), and analyzing physiological signals (status granted); and being an employee of Google and holding Alphabet stock (part of the compensation package). Dr Chen reported being an employee of Google and holding Alphabet stock (part of the compensation package). Dr Krause reported receipt of personal fees from Stanford University outside the submitted work and being an employee of Google and holding Alphabet stock (part of the compensation package). Dr Peng reported holding a patent to the each of the following: predicting cardiovascular risk factors in retinal fundus photographs using deep learning, fundus imagery machine learning systems, health predictions from histopathology slides, and pathology heatmap predictions (status pending for all); and being an employee of Google and holding Alphabet stock (part of the compensation package).
You currently have no searches saved.