[Skip to Content]
[Skip to Content Landing]

Development and Validation of a Clinical Risk Score to Predict the Occurrence of Critical Illness in Hospitalized Patients With COVID-19

Educational Objective
To understand the development of a tool with epidemiological and clinical associations used to predict critical illness in patients infected with COVID-19
1 Credit CME
Key Points

Question  What epidemiological and clinical characteristics are associated with the development of critical illness among patients with novel coronavirus disease 2019 (COVID-19)? Can these characteristics be used to predict which patients admitted to the hospital with COVID-19 will need admission to an intensive care unit, mechanical ventilation, or will die?

Findings  In this study with a development cohort of 1590 patients and a validation cohort of 710 patients, a risk score was developed and validated to predict development of critical illness. We identified 10 independent predictors and developed a risk score (COVID-GRAM) that predicts development of critical illness. The risk score predictors included: chest radiography abnormality, age, hemoptysis, dyspnea, unconsciousness, number of comorbidities, cancer history, neutrophil-to-lymphocyte ratio, lactate dehydrogenase, and direct bilirubin.

Meaning  The COVID risk score may help identify patients with COVID-19 who may subsequently develop critical illness.

Abstract

Importance  Early identification of patients with novel coronavirus disease 2019 (COVID-19) who may develop critical illness is of great importance and may aid in delivering proper treatment and optimizing use of resources.

Objective  To develop and validate a clinical score at hospital admission for predicting which patients with COVID-19 will develop critical illness based on a nationwide cohort in China.

Design, Setting, and Participants  Collaborating with the National Health Commission of China, we established a retrospective cohort of patients with COVID-19 from 575 hospitals in 31 provincial administrative regions as of January 31, 2020. Epidemiological, clinical, laboratory, and imaging variables ascertained at hospital admission were screened using Least Absolute Shrinkage and Selection Operator (LASSO) and logistic regression to construct a predictive risk score (COVID-GRAM). The score provides an estimate of the risk that a hospitalized patient with COVID-19 will develop critical illness. Accuracy of the score was measured by the area under the receiver operating characteristic curve (AUC). Data from 4 additional cohorts in China hospitalized with COVID-19 were used to validate the score. Data were analyzed between February 20, 2020 and March 17, 2020.

Main Outcomes and Measures  Among patients with COVID-19 admitted to the hospital, critical illness was defined as the composite measure of admission to the intensive care unit, invasive ventilation, or death.

Results  The development cohort included 1590 patients. the mean (SD) age of patients in the cohort was 48.9 (15.7) years; 904 (57.3%) were men. The validation cohort included 710 patients with a mean (SD) age of 48.2 (15.2) years, and 382 (53.8%) were men and 172 (24.2%). From 72 potential predictors, 10 variables were independent predictive factors and were included in the risk score: chest radiographic abnormality (OR, 3.39; 95% CI, 2.14-5.38), age (OR, 1.03; 95% CI, 1.01-1.05), hemoptysis (OR, 4.53; 95% CI, 1.36-15.15), dyspnea (OR, 1.88; 95% CI, 1.18-3.01), unconsciousness (OR, 4.71; 95% CI, 1.39-15.98), number of comorbidities (OR, 1.60; 95% CI, 1.27-2.00), cancer history (OR, 4.07; 95% CI, 1.23-13.43), neutrophil-to-lymphocyte ratio (OR, 1.06; 95% CI, 1.02-1.10), lactate dehydrogenase (OR, 1.002; 95% CI, 1.001-1.004) and direct bilirubin (OR, 1.15; 95% CI, 1.06-1.24). The mean AUC in the development cohort was 0.88 (95% CI, 0.85-0.91) and the AUC in the validation cohort was 0.88 (95% CI, 0.84-0.93). The score has been translated into an online risk calculator that is freely available to the public (http://118.126.104.170/)

Conclusions and Relevance  In this study, a risk score based on characteristics of COVID-19 patients at the time of admission to the hospital was developed that may help predict a patient’s risk of developing critical illness.

Sign in to take quiz and track your certificates

Buy This Activity
Our websites may be periodically unavailable between 12:00am CT March 25, 2023 and 4:00pm CT March 26, 2023 for regularly scheduled maintenance.

JN Learning™ is the home for CME and MOC from the JAMA Network. Search by specialty or US state and earn AMA PRA Category 1 Credit(s)™ from articles, audio, Clinical Challenges and more. Learn more about CME/MOC

CME Disclosure Statement: Unless noted, all individuals in control of content reported no relevant financial relationships. If applicable, all relevant financial relationships have been mitigated.

Article Information

Corresponding Author: Jianxing He, MD, PhD (drjianxing.he@gmail.com), and Nan-Shan Zhong, MD, Guangzhou Institute of Respiratory Health (nanshan@vip.163.com), China State Key Laboratory of Respiratory Disease, National Clinical Research Center for Respiratory Disease, The First Affiliated Hospital of Guangzhou Medical University, 151 Yanjiang Rd, Guangzhou, Guangdong 510120, China.

Accepted for Publication: April 20, 2020.

Published Online: May 12, 2020. doi:10.1001/jamainternmed.2020.2033

Author Contributions: Drs W. Liang and He had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Drs W. Liang, H. Liang, Ou, B. Chen, A. Chen, and C. Li are joint first authors.

Concept and design: W. Liang, H. Liang, A. Chen, Xu, G. Chen, H. Guo, Zhang, Zhong, He.

Acquisition, analysis, or interpretation of data: W. Liang, Ou, B. Chen, C. Li, Y. Li, Guan, Sang, Lu, Xu, J. Guo, Z. Chen, Zhao, S. Li, Zhang, Zhong.

Drafting of the manuscript: W. Liang, H. Liang, B. Chen, C. Li, G. Chen, Zhao, Zhong, He.

Critical revision of the manuscript for important intellectual content: W. Liang, Ou, A. Chen, Y. Li, Guan, Sang, Lu, Xu, H. Guo, J. Guo, Z. Chen, S. Li, Zhang, Zhong.

Statistical analysis: W. Liang, H. Liang, Ou, B. Chen, H. Guo, Z. Chen, He.

Obtained funding: W. Liang, A. Chen, Xu.

Administrative, technical, or material support: Y. Li, G. Chen, J. Guo, Z. Chen, S. Li, Zhong.

Supervision: Sang, Lu, Zhang, Zhong.

Other - Construction of the online calculator: B. Chen.

Conflict of Interest Disclosures: None reported.

Funding/Support: This study is supported by China National Science Foundation (Grant No. 81871893), Key Project of Guangzhou Scientific Research Project (Grant No. 201804020030), High-level university construction project of Guangzhou medical university (Grant No. 20182737, 201721007, 201715907, 2017160107); National key R & D Program (Grant No. 2017YFC0907903 & 2017YFC0112704) and the Guangdong high level hospital construction “reaching peak” plan.

Role of the Funder/Sponsor: The funding organizations had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

The China Medical Treatment Expert Group for COVID-19: Zong-jiu Zhang, MD, Ya-hui Jiao, MD, Bin Du, MD, Xin-qiang Gao, MD and Tao Wei, MD (National Health Commission), Yu-fei Duan, MD and Zhi-ling Zhao, MD (Health Commission of Guangdong Province), Yi-min Li, MD, Zi-jing Liang, MD, Nuo-fu Zhang, MD, Shi-yue Li, MD, Qing-hui Huang, MD, Wen-xi Huang, MD, and Ming Li, MD (Guangzhou Institute of Respiratory Health), Zheng Chen, MD, Dong Han, MD, Li Li, MD, Zheng Chen, MD, Zhi-ying Zhan, MD, Jin-jian Chen, MD, Li-jun Xu, MD, Xiao-han Xu, MD (State Key Laboratory of Organ Failure Research, Department of Biostatistics, Guangdong Provincial Key Laboratory of Tropical Disease Research, School of Public Health, Southern Medical University); Li-qiang Wang, MD, Wei-peng Cai, MD, Zi-sheng Chen, MD (the sixth affiliated hospital of Guangzhou medical university), Chang-xing Ou, MD, Xiao-min Peng, MD, Si-ni Cui, MD, Yuan Wang, MD, Mou Zeng, MD, Xin Hao, MD, Qi-hua He, MD, Jing-pei Li, MD, Xu-kai Li, MD, Wei Wang, MD, Li-min Ou, MD, Ya-lei Zhang, MD, Jing-wei Liu, MD, Xin-guo Xiong, MD, Wei-juna Shi, MD, San-mei Yu, MD, Run-dong Qin, MD, Si-yang Yao, MD, Bo-meng Zhang, MD, Xiao-hong Xie, MD, Zhan-hong Xie, MD, Wan-di Wang, MD, Xiao-xian Zhang, MD, Hui-yin Xu, MD, Zi-qing Zhou, MD, Ying Jiang, MD, Ni Liu, MD, Jing-jing Yuan, MD, Zheng Zhu, MD, Jie-xia Zhang, MD, Hong-hao Li, MD, Wei-hua Huang, MD, Lu-lin Wang, MD, Jie-ying Li, MD, Li-fen Gao, MD, Jia-bo Gao, MD, Cai-chen Li, MD, Xue-wei Chen, MD, Jia-bo Gao, MD, Ming-shan Xue, MD, Shou-xie Huang, MD, Jia-man Tang, MD, Wei-li Gu, MD, Jin-lin Wang, MD (Guangzhou Institute of Respiratory Health).

Acknowledgement: Special thanks to Ling Sang, MD, supporting Wuhan Jinyintan Hospital; Yuan-da Xu, MD, supporting Wuhan Union Hospital; Ai-lan Chen MD, supporting Wuhan Hankou Hospital; Guo-qiang Chen, MD, and Hai-yan Guo, MD, Foshan Hospital; and Jun Guo, MD, Daye Hospital; and the China Medical Treatment Expert Group for COVID-19 for providing validation data. See further acknowledgements in the eAppendix in the Supplement.

References
1.
Zhou  F , Yu  T , Du  R ,  et al.  Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study.   Lancet. 2020;395(10229):1054-1062. doi:10.1016/S0140-6736(20)30566-3PubMedGoogle ScholarCrossref
2.
Wu  Z , McGoogan  JM .  Characteristics of and important lessons from the coronavirus disease 2019 (COVID-19) outbreak in China: summary of a report of 72 314 cases from the Chinese Center for Disease Control and Prevention.   JAMA. 2020;(Feb):24. doi:10.1001/jama.2020.2648PubMedGoogle Scholar
3.
Wu  C , Chen  X , Cai  Y ,  et al.  Risk factors associated with acute respiratory distress syndrome and death in patients with coronavirus disease 2019 pneumonia in Wuhan, China.   JAMA Intern Med. 2020;(Mar):13. doi:10.1001/jamainternmed.2020.0994PubMedGoogle Scholar
4.
Guan  WJ , Ni  ZY , Hu  Y ,  et al; China Medical Treatment Expert Group for Covid-19.  Clinical characteristics of coronavirus disease 2019 in China.   N Engl J Med. 2020;(Feb):28. doi:10.1056/NEJMoa2002032PubMedGoogle Scholar
5.
Metlay  JP , Waterer  GW , Long  AC ,  et al.  Diagnosis and treatment of adults with community-acquired pneumonia. an official clinical practice guideline of the American Thoracic Society and Infectious Diseases Society of America.   Am J Respir Crit Care Med. 2019;200(7):e45-e67. doi:10.1164/rccm.201908-1581STPubMedGoogle ScholarCrossref
6.
Gao  HN , Lu  HZ , Cao  B ,  et al.  Clinical findings in 111 cases of influenza A (H7N9) virus infection.   N Engl J Med. 2013;368(24):2277-2285. doi:10.1056/NEJMoa1305584PubMedGoogle ScholarCrossref
7.
Iasonos  A , Schrag  D , Raj  GV , Panageas  KS .  How to build and interpret a nomogram for cancer prognosis.   J Clin Oncol. 2008;26(8):1364-1370. doi:10.1200/JCO.2007.12.9791PubMedGoogle ScholarCrossref
8.
Neill  AM , Martin  IR , Weir  R ,  et al.  Community acquired pneumonia: aetiology and usefulness of severity criteria on admission.   Thorax. 1996;51(10):1010-1016. doi:10.1136/thx.51.10.1010PubMedGoogle ScholarCrossref
9.
Xu  J , Long  JS .  Confidence intervals for predicted outcomes in regression models for categorical outcomes.   Stata Journal. 2005;5(4):537-559. doi:10.1177/1536867X0500500405Google ScholarCrossref
10.
Liang  W , Guan  W , Chen  R ,  et al.  Cancer patients in SARS-CoV-2 infection: a nationwide analysis in China.   Lancet Oncol. 2020;21(3):335-337. doi:10.1016/S1470-2045(20)30096-6PubMedGoogle ScholarCrossref
AMA CME Accreditation Information

Credit Designation Statement: The American Medical Association designates this Journal-based CME activity activity for a maximum of 1.00  AMA PRA Category 1 Credit(s)™. Physicians should claim only the credit commensurate with the extent of their participation in the activity.

Successful completion of this CME activity, which includes participation in the evaluation component, enables the participant to earn up to:

  • 1.00 Medical Knowledge MOC points in the American Board of Internal Medicine's (ABIM) Maintenance of Certification (MOC) program;;
  • 1.00 Self-Assessment points in the American Board of Otolaryngology – Head and Neck Surgery’s (ABOHNS) Continuing Certification program;
  • 1.00 MOC points in the American Board of Pediatrics’ (ABP) Maintenance of Certification (MOC) program;
  • 1.00 Lifelong Learning points in the American Board of Pathology’s (ABPath) Continuing Certification program; and
  • 1.00 CME points in the American Board of Surgery’s (ABS) Continuing Certification program

It is the CME activity provider's responsibility to submit participant completion information to ACCME for the purpose of granting MOC credit.

Close
Want full access to the AMA Ed Hub?
After you sign up for AMA Membership, make sure you sign in or create a Physician account with the AMA in order to access all learning activities on the AMA Ed Hub
Buy this activity
Close
Want full access to the AMA Ed Hub?
After you sign up for AMA Membership, make sure you sign in or create a Physician account with the AMA in order to access all learning activities on the AMA Ed Hub
Buy this activity
Close
With a personal account, you can:
  • Access free activities and track your credits
  • Personalize content alerts
  • Customize your interests
  • Fully personalize your learning experience
Education Center Collection Sign In Modal Right
Close

Name Your Search

Save Search
With a personal account, you can:
  • Access free activities and track your credits
  • Personalize content alerts
  • Customize your interests
  • Fully personalize your learning experience
Close
Close

Lookup An Activity

or

My Saved Searches

You currently have no searches saved.

Close

My Saved Courses

You currently have no courses saved.

Close