[Skip to Content]
[Skip to Content Landing]

Comparison of International Classification of Diseases and Related Health Problems, Tenth Revision Codes With Electronic Medical Records Among Patients With Symptoms of Coronavirus Disease 2019

Educational Objective
To understand how to use International Statistical Classification of Diseases and Related Health Problems, Tenth Revision (ICD-10) codes to properly record COVID-19 symptoms
1 Credit CME
Key Points

Question  Do International Statistical Classification of Diseases and Related Health Problems, Tenth Revision (ICD-10) codes accurately capture presenting symptoms of fever, cough, and dyspnea among patients being tested for coronavirus disease 2019 (COVID-19)?

Findings  In this cohort study, an electronic medical record review of 2201 patients tested for COVID-19 between March 10 and April 6, 2020, found that ICD-10 codes had poor sensitivity and negative predictive value for capturing fever, cough, and dyspnea.

Meaning  These findings suggest that symptom-specific ICD-10 codes do not accurately capture COVID-19–related symptoms and should not be used to populate symptoms in electronic medical record–based cohorts.


Importance  International Statistical Classification of Diseases and Related Health Problems, Tenth Revision (ICD-10) codes are used to characterize coronavirus disease 2019 (COVID-19)–related symptoms. Their accuracy is unknown, which could affect downstream analyses.

Objective  To compare the performance of fever-, cough-, and dyspnea-specific ICD-10 codes with medical record review among patients tested for COVID-19.

Design, Setting, and Participants  This cohort study included patients who underwent quantitative reverse transcriptase–polymerase chain reaction testing for severe acute respiratory syndrome coronavirus 2 at University of Utah Health from March 10 to April 6, 2020. Data analysis was performed in April 2020.

Main Outcomes and Measures  The sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of ICD-10 codes for fever (R50*), cough (R05*), and dyspnea (R06.0*) were compared with manual medical record review. Performance was calculated overall and stratified by COVID-19 test result, sex, age group (<50, 50-64, and >64 years), and inpatient status. Bootstrapping was used to generate 95% CIs, and Pearson χ2 tests were used to compare different subgroups.

Results  Among 2201 patients tested for COVD-19, the mean (SD) age was 42 (17) years; 1201 (55%) were female, 1569 (71%) were White, and 282 (13%) were Hispanic or Latino. The prevalence of fever was 66% (1444 patients), that of cough was 88% (1930 patients), and that of dyspnea was 64% (1399 patients). For fever, the sensitivity of ICD-10 codes was 0.26 (95% CI, 0.24-0.29), specificity was 0.98 (95% CI, 0.96-0.99), PPV was 0.96 (95% CI, 0.93-0.97), and NPV was 0.41 (95% CI, 0.39-0.43). For cough, the sensitivity of ICD-10 codes was 0.44 (95% CI, 0.42-0.46), specificity was 0.88 (95% CI, 0.84-0.92), PPV was 0.96 (95% CI, 0.95-0.97), and NPV was 0.18 (95% CI, 0.16-0.20). For dyspnea, the sensitivity of ICD-10 codes was 0.24 (95% CI, 0.22-0.26), specificity was 0.97 (95% CI, 0.96-0.98), PPV was 0.93 (95% CI, 0.90-0.96), and NPV was 0.42 (95% CI, 0.40-0.44). ICD-10 code performance was better for inpatients than for outpatients for fever (χ2 = 41.30; P < .001) and dyspnea (χ2 = 14.25; P = .003) but not for cough (χ2 = 5.13; P = .16).

Conclusions and Relevance  These findings suggest that ICD-10 codes lack sensitivity and have poor NPV for symptoms associated with COVID-19. This inaccuracy has implications for any downstream data model, scientific discovery, or surveillance that relies on these codes.

Sign in to take quiz and track your certificates

Buy This Activity
Our websites may be periodically unavailable between 12:00am CT March 25, 2023 and 4:00pm CT March 26, 2023 for regularly scheduled maintenance.

JN Learning™ is the home for CME and MOC from the JAMA Network. Search by specialty or US state and earn AMA PRA Category 1 Credit(s)™ from articles, audio, Clinical Challenges and more. Learn more about CME/MOC

CME Disclosure Statement: Unless noted, all individuals in control of content reported no relevant financial relationships. If applicable, all relevant financial relationships have been mitigated.

Article Information

Accepted for Publication: July 11, 2020.

Published: August 14, 2020. doi:10.1001/jamanetworkopen.2020.17703

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Crabb BT et al. JAMA Network Open.

Corresponding Author: Rashmee U. Shah, MD, MS, Division of Cardiovascular Medicine, University of Utah School of Medicine, 30 N 1900 E, Room 4A100, Salt Lake City, UT 84132 (rashmee.shah@utah.edu).

Author Contributions: Mr Crabb and Dr Shah had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Crabb, West, Brown, Shah.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Crabb, Martin, West, Brown, Leung, Shah.

Critical revision of the manuscript for important intellectual content: Crabb, Lyons, Bale, Martin, Berger, Mann, West, Peacock, Leung, Shah.

Statistical analysis: Crabb, Martin, Brown, Shah.

Administrative, technical, or material support: Berger, West, Peacock.

Supervision: Crabb, Mann, Leung, Shah.

Conflict of Interest Disclosures: Dr Shah reported receiving honoraria from the American College of Cardiology outside the submitted work. No other disclosures were reported.

Funding/Support: Dr Leung is supported by grant R01AI135114 from the National Institute of Allergy and Infectious Diseases. Dr Shah is supported by grant K08HL136850 from the National Heart Lung and Blood Institute and a donation from Women As One. REDCap, used in this study, is supported by the University of Utah Center for Clinical and Translational Science, with funding in part from the National Center for Research Resources and the National Center for Advancing Translational Sciences (grant 8UL1TR000105, formerly UL1RR025764).

Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Lee  P . COVID-19 interoperability project: logica. Published March 25, 2020. Accessed April 23, 2020. https://www.logicahealth.org/covid-19-interoperability-project/
World Health Organization.  International Statistical Classification of Diseases, Tenth Revision (ICD-10). World Health Organization; 1992.
Horsky  J , Drucker  EA , Ramelson  HZ .  Accuracy and completeness of clinical coding using ICD-10 for ambulatory visits.   AMIA Annu Symp Proc. 2018;2017:912-920.PubMedGoogle Scholar
Shah  RU , Mukherjee  R , Zhang  Y ,  et al.  Impact of different electronic cohort definitions to identify patients with atrial fibrillation from the electronic medical record.   J Am Heart Assoc. 2020;9(5):e014527. doi:10.1161/JAHA.119.014527PubMedGoogle Scholar
von Elm  E , Altman  DG , Egger  M , Pocock  SJ , Gøtzsche  PC , Vandenbroucke  JP ; STROBE Initiative.  The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies.   Lancet. 2007;370(9596):1453-1457. doi:10.1016/S0140-6736(07)61602-XPubMedGoogle ScholarCrossref
Guan  W-J , Ni  Z-Y , Hu  Y ,  et al; China Medical Treatment Expert Group for Covid-19.  Clinical characteristics of coronavirus disease 2019 in China.   N Engl J Med. 2020;382(18):1708-1720. doi:10.1056/NEJMoa2002032PubMedGoogle ScholarCrossref
Richardson  S , Hirsch  JS , Narasimhan  M ,  et al; Northwell COVID-19 Research Consortium.  Presenting characteristics, comorbidities, and outcomes among 5700 patients hospitalized with COVID-19 in the New York City area.   JAMA. 2020;323(20):2052-2059. doi:10.1001/jama.2020.6775PubMedGoogle ScholarCrossref
Huang  C , Wang  Y , Li  X ,  et al.  Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China.   Lancet. 2020;395(10223):497-506. doi:10.1016/S0140-6736(20)30183-5PubMedGoogle ScholarCrossref
Harris  PA , Taylor  R , Thielke  R , Payne  J , Gonzalez  N , Conde  JG .  Research electronic data capture (REDCap): a metadata-driven methodology and workflow process for providing translational research informatics support.   J Biomed Inform. 2009;42(2):377-381. doi:10.1016/j.jbi.2008.08.010PubMedGoogle ScholarCrossref
US Department of Health & Human Services. National COVID cohort collaborative (N3C). Accessed April 14, 2020. https://covid.cd2h.org/N3C
Observational Health Data Sciences and Informatics. OMOP common data model. Accessed October 7, 2019. https://www.ohdsi.org/data-standardization/the-common-data-model/
Centers for Disease Control and Prevention. U.S. influenza surveillance system: purpose and methods. Published February 24, 2020. Accessed April 24, 2020. https://www.cdc.gov/flu/weekly/overview.htm
Molnar  AO , van Walraven  C , McArthur  E , Fergusson  D , Garg  AX , Knoll  G .  Validation of administrative database codes for acute kidney injury in kidney transplant recipients.   Can J Kidney Health Dis. 2016;3(1):18. doi:10.1186/s40697-016-0108-7PubMedGoogle Scholar
Chang  TE , Lichtman  JH , Goldstein  LB , George  MG .  Accuracy of ICD-9-CM codes by hospital characteristics and stroke severity: Paul Coverdell National Acute Stroke Program.   J Am Heart Assoc. 2016;5(6):e003056. doi:10.1161/JAHA.115.003056PubMedGoogle Scholar
Coronavirus Utah.gov. State of Utah releases “Healthy Together” beta app. Accessed April 24, 2020. https://coronavirus.utah.gov/state-of-utah-releases-healthy-together-beta-app/
Biber  J , Ose  D , Reese  J ,  et al.  Patient reported outcomes: experiences with implementation in a university health care setting.   J Patient Rep Outcomes. 2018;2:34. doi:10.1186/s41687-018-0059-0PubMedGoogle ScholarCrossref
Joseph  A . Lancet, New England Journal retract Covid-19 studies, including one that raised safety concerns about malaria drugs. STAT. Published June 4, 2020. Accessed June 17, 2020. https://www.statnews.com/2020/06/04/lancet-retracts-major-covid-19-paper-that-raised-safety-concerns-about-malaria-drugs/
Roumie  CL , Mitchel  E , Gideon  PS , Varas-Lorenzo  C , Castellsague  J , Griffin  MR .  Validation of ICD-9 codes with a high positive predictive value for incident strokes resulting in hospitalization using Medicaid health data.   Pharmacoepidemiol Drug Saf. 2008;17(1):20-26. doi:10.1002/pds.1518PubMedGoogle ScholarCrossref
Yasaitis  LC , Berkman  LF , Chandra  A .  Comparison of self-reported and Medicare claims-identified acute myocardial infarction.   Circulation. 2015;131(17):1477-1485. doi:10.1161/CIRCULATIONAHA.114.013829PubMedGoogle ScholarCrossref
AMA CME Accreditation Information

Credit Designation Statement: The American Medical Association designates this Journal-based CME activity activity for a maximum of 1.00  AMA PRA Category 1 Credit(s)™. Physicians should claim only the credit commensurate with the extent of their participation in the activity.

Successful completion of this CME activity, which includes participation in the evaluation component, enables the participant to earn up to:

  • 1.00 Medical Knowledge MOC points in the American Board of Internal Medicine's (ABIM) Maintenance of Certification (MOC) program;;
  • 1.00 Self-Assessment points in the American Board of Otolaryngology – Head and Neck Surgery’s (ABOHNS) Continuing Certification program;
  • 1.00 MOC points in the American Board of Pediatrics’ (ABP) Maintenance of Certification (MOC) program;
  • 1.00 Lifelong Learning points in the American Board of Pathology’s (ABPath) Continuing Certification program; and
  • 1.00 CME points in the American Board of Surgery’s (ABS) Continuing Certification program

It is the CME activity provider's responsibility to submit participant completion information to ACCME for the purpose of granting MOC credit.

Want full access to the AMA Ed Hub?
After you sign up for AMA Membership, make sure you sign in or create a Physician account with the AMA in order to access all learning activities on the AMA Ed Hub
Buy this activity
Want full access to the AMA Ed Hub?
After you sign up for AMA Membership, make sure you sign in or create a Physician account with the AMA in order to access all learning activities on the AMA Ed Hub
Buy this activity
With a personal account, you can:
  • Access free activities and track your credits
  • Personalize content alerts
  • Customize your interests
  • Fully personalize your learning experience
Education Center Collection Sign In Modal Right

Name Your Search

Save Search
With a personal account, you can:
  • Access free activities and track your credits
  • Personalize content alerts
  • Customize your interests
  • Fully personalize your learning experience

Lookup An Activity


My Saved Searches

You currently have no searches saved.


My Saved Courses

You currently have no courses saved.