Comparison of ICD-10 Codes With Electronic Medical Records Among Patients With Symptoms of COVID-19 | Electronic Health Records | JN Learning | AMA Ed Hub [Skip to Content]
Access to paid content on this site is currently suspended due to excessive activity being detected from your IP address 18.206.177.17. Please contact the publisher to request reinstatement.
[Skip to Content Landing]

Comparison of International Classification of Diseases and Related Health Problems, Tenth Revision Codes With Electronic Medical Records Among Patients With Symptoms of Coronavirus Disease 2019

Educational Objective
To understand how to use International Statistical Classification of Diseases and Related Health Problems, Tenth Revision (ICD-10) codes to properly record COVID-19 symptoms
1 Credit CME
Key Points

Question  Do International Statistical Classification of Diseases and Related Health Problems, Tenth Revision (ICD-10) codes accurately capture presenting symptoms of fever, cough, and dyspnea among patients being tested for coronavirus disease 2019 (COVID-19)?

Findings  In this cohort study, an electronic medical record review of 2201 patients tested for COVID-19 between March 10 and April 6, 2020, found that ICD-10 codes had poor sensitivity and negative predictive value for capturing fever, cough, and dyspnea.

Meaning  These findings suggest that symptom-specific ICD-10 codes do not accurately capture COVID-19–related symptoms and should not be used to populate symptoms in electronic medical record–based cohorts.

Abstract

Importance  International Statistical Classification of Diseases and Related Health Problems, Tenth Revision (ICD-10) codes are used to characterize coronavirus disease 2019 (COVID-19)–related symptoms. Their accuracy is unknown, which could affect downstream analyses.

Objective  To compare the performance of fever-, cough-, and dyspnea-specific ICD-10 codes with medical record review among patients tested for COVID-19.

Design, Setting, and Participants  This cohort study included patients who underwent quantitative reverse transcriptase–polymerase chain reaction testing for severe acute respiratory syndrome coronavirus 2 at University of Utah Health from March 10 to April 6, 2020. Data analysis was performed in April 2020.

Main Outcomes and Measures  The sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of ICD-10 codes for fever (R50*), cough (R05*), and dyspnea (R06.0*) were compared with manual medical record review. Performance was calculated overall and stratified by COVID-19 test result, sex, age group (<50, 50-64, and >64 years), and inpatient status. Bootstrapping was used to generate 95% CIs, and Pearson χ2 tests were used to compare different subgroups.

Results  Among 2201 patients tested for COVD-19, the mean (SD) age was 42 (17) years; 1201 (55%) were female, 1569 (71%) were White, and 282 (13%) were Hispanic or Latino. The prevalence of fever was 66% (1444 patients), that of cough was 88% (1930 patients), and that of dyspnea was 64% (1399 patients). For fever, the sensitivity of ICD-10 codes was 0.26 (95% CI, 0.24-0.29), specificity was 0.98 (95% CI, 0.96-0.99), PPV was 0.96 (95% CI, 0.93-0.97), and NPV was 0.41 (95% CI, 0.39-0.43). For cough, the sensitivity of ICD-10 codes was 0.44 (95% CI, 0.42-0.46), specificity was 0.88 (95% CI, 0.84-0.92), PPV was 0.96 (95% CI, 0.95-0.97), and NPV was 0.18 (95% CI, 0.16-0.20). For dyspnea, the sensitivity of ICD-10 codes was 0.24 (95% CI, 0.22-0.26), specificity was 0.97 (95% CI, 0.96-0.98), PPV was 0.93 (95% CI, 0.90-0.96), and NPV was 0.42 (95% CI, 0.40-0.44). ICD-10 code performance was better for inpatients than for outpatients for fever (χ2 = 41.30; P < .001) and dyspnea (χ2 = 14.25; P = .003) but not for cough (χ2 = 5.13; P = .16).

Conclusions and Relevance  These findings suggest that ICD-10 codes lack sensitivity and have poor NPV for symptoms associated with COVID-19. This inaccuracy has implications for any downstream data model, scientific discovery, or surveillance that relies on these codes.

Sign in to take quiz and track your certificates

Buy This Activity

JN Learning™ is the home for CME and MOC from the JAMA Network. Search by specialty or US state and earn AMA PRA Category 1 CME Credit™ from articles, audio, Clinical Challenges and more. Learn more about CME/MOC

Article Information

Accepted for Publication: July 11, 2020.

Published: August 14, 2020. doi:10.1001/jamanetworkopen.2020.17703

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Crabb BT et al. JAMA Network Open.

Corresponding Author: Rashmee U. Shah, MD, MS, Division of Cardiovascular Medicine, University of Utah School of Medicine, 30 N 1900 E, Room 4A100, Salt Lake City, UT 84132 (rashmee.shah@utah.edu).

Author Contributions: Mr Crabb and Dr Shah had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Crabb, West, Brown, Shah.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Crabb, Martin, West, Brown, Leung, Shah.

Critical revision of the manuscript for important intellectual content: Crabb, Lyons, Bale, Martin, Berger, Mann, West, Peacock, Leung, Shah.

Statistical analysis: Crabb, Martin, Brown, Shah.

Administrative, technical, or material support: Berger, West, Peacock.

Supervision: Crabb, Mann, Leung, Shah.

Conflict of Interest Disclosures: Dr Shah reported receiving honoraria from the American College of Cardiology outside the submitted work. No other disclosures were reported.

Funding/Support: Dr Leung is supported by grant R01AI135114 from the National Institute of Allergy and Infectious Diseases. Dr Shah is supported by grant K08HL136850 from the National Heart Lung and Blood Institute and a donation from Women As One. REDCap, used in this study, is supported by the University of Utah Center for Clinical and Translational Science, with funding in part from the National Center for Research Resources and the National Center for Advancing Translational Sciences (grant 8UL1TR000105, formerly UL1RR025764).

Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

References
1.
Lee  P . COVID-19 interoperability project: logica. Published March 25, 2020. Accessed April 23, 2020. https://www.logicahealth.org/covid-19-interoperability-project/
2.
World Health Organization.  International Statistical Classification of Diseases, Tenth Revision (ICD-10). World Health Organization; 1992.
3.
Horsky  J , Drucker  EA , Ramelson  HZ .  Accuracy and completeness of clinical coding using ICD-10 for ambulatory visits.   AMIA Annu Symp Proc. 2018;2017:912-920.PubMedGoogle Scholar
4.
Shah  RU , Mukherjee  R , Zhang  Y ,  et al.  Impact of different electronic cohort definitions to identify patients with atrial fibrillation from the electronic medical record.   J Am Heart Assoc. 2020;9(5):e014527. doi:10.1161/JAHA.119.014527PubMedGoogle Scholar
5.
von Elm  E , Altman  DG , Egger  M , Pocock  SJ , Gøtzsche  PC , Vandenbroucke  JP ; STROBE Initiative.  The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies.   Lancet. 2007;370(9596):1453-1457. doi:10.1016/S0140-6736(07)61602-XPubMedGoogle ScholarCrossref
6.
Guan  W-J , Ni  Z-Y , Hu  Y ,  et al; China Medical Treatment Expert Group for Covid-19.  Clinical characteristics of coronavirus disease 2019 in China.   N Engl J Med. 2020;382(18):1708-1720. doi:10.1056/NEJMoa2002032PubMedGoogle ScholarCrossref
7.
Richardson  S , Hirsch  JS , Narasimhan  M ,  et al; Northwell COVID-19 Research Consortium.  Presenting characteristics, comorbidities, and outcomes among 5700 patients hospitalized with COVID-19 in the New York City area.   JAMA. 2020;323(20):2052-2059. doi:10.1001/jama.2020.6775PubMedGoogle ScholarCrossref
8.
Huang  C , Wang  Y , Li  X ,  et al.  Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China.   Lancet. 2020;395(10223):497-506. doi:10.1016/S0140-6736(20)30183-5PubMedGoogle ScholarCrossref
9.
Harris  PA , Taylor  R , Thielke  R , Payne  J , Gonzalez  N , Conde  JG .  Research electronic data capture (REDCap): a metadata-driven methodology and workflow process for providing translational research informatics support.   J Biomed Inform. 2009;42(2):377-381. doi:10.1016/j.jbi.2008.08.010PubMedGoogle ScholarCrossref
10.
US Department of Health & Human Services. National COVID cohort collaborative (N3C). Accessed April 14, 2020. https://covid.cd2h.org/N3C
11.
Observational Health Data Sciences and Informatics. OMOP common data model. Accessed October 7, 2019. https://www.ohdsi.org/data-standardization/the-common-data-model/
12.
Centers for Disease Control and Prevention. U.S. influenza surveillance system: purpose and methods. Published February 24, 2020. Accessed April 24, 2020. https://www.cdc.gov/flu/weekly/overview.htm
13.
Molnar  AO , van Walraven  C , McArthur  E , Fergusson  D , Garg  AX , Knoll  G .  Validation of administrative database codes for acute kidney injury in kidney transplant recipients.   Can J Kidney Health Dis. 2016;3(1):18. doi:10.1186/s40697-016-0108-7PubMedGoogle Scholar
14.
Chang  TE , Lichtman  JH , Goldstein  LB , George  MG .  Accuracy of ICD-9-CM codes by hospital characteristics and stroke severity: Paul Coverdell National Acute Stroke Program.   J Am Heart Assoc. 2016;5(6):e003056. doi:10.1161/JAHA.115.003056PubMedGoogle Scholar
15.
Coronavirus Utah.gov. State of Utah releases “Healthy Together” beta app. Accessed April 24, 2020. https://coronavirus.utah.gov/state-of-utah-releases-healthy-together-beta-app/
16.
Biber  J , Ose  D , Reese  J ,  et al.  Patient reported outcomes: experiences with implementation in a university health care setting.   J Patient Rep Outcomes. 2018;2:34. doi:10.1186/s41687-018-0059-0PubMedGoogle ScholarCrossref
17.
Joseph  A . Lancet, New England Journal retract Covid-19 studies, including one that raised safety concerns about malaria drugs. STAT. Published June 4, 2020. Accessed June 17, 2020. https://www.statnews.com/2020/06/04/lancet-retracts-major-covid-19-paper-that-raised-safety-concerns-about-malaria-drugs/
18.
Roumie  CL , Mitchel  E , Gideon  PS , Varas-Lorenzo  C , Castellsague  J , Griffin  MR .  Validation of ICD-9 codes with a high positive predictive value for incident strokes resulting in hospitalization using Medicaid health data.   Pharmacoepidemiol Drug Saf. 2008;17(1):20-26. doi:10.1002/pds.1518PubMedGoogle ScholarCrossref
19.
Yasaitis  LC , Berkman  LF , Chandra  A .  Comparison of self-reported and Medicare claims-identified acute myocardial infarction.   Circulation. 2015;131(17):1477-1485. doi:10.1161/CIRCULATIONAHA.114.013829PubMedGoogle ScholarCrossref
If you are not a JN Learning subscriber, you can either:
Subscribe to JN Learning for one year
Buy this activity
jn-learning_Modal_Multimedia_LoginSubscribe_Purchase
Close
If you are not a JN Learning subscriber, you can either:
Subscribe to JN Learning for one year
Buy this activity
jn-learning_Modal_Multimedia_LoginSubscribe_Purchase
Close
With a personal account, you can:
  • Access free activities and track your credits
  • Personalize content alerts
  • Customize your interests
  • Fully personalize your learning experience
Education Center Collection Sign In Modal Right
Close

Name Your Search

Save Search
Close
With a personal account, you can:
  • Track your credits
  • Personalize content alerts
  • Customize your interests
  • Fully personalize your learning experience
jn-learning_Modal_SaveSearch_NoAccess_Purchase
Close

Lookup An Activity

or

Close

My Saved Searches

You currently have no searches saved.

Close
With a personal account, you can:
  • Access free activities and track your credits
  • Personalize content alerts
  • Customize your interests
  • Fully personalize your learning experience
Education Center Collection Sign In Modal Right
Close