[Skip to Content]
[Skip to Content Landing]

Development and Validation of a Clinical Risk Score to Predict the Occurrence of Critical Illness in Hospitalized Patients With COVID-19

Educational Objective
To understand the development of a tool with epidemiological and clinical associations used to predict critical illness in patients infected with COVID-19
1 Credit CME
Key Points

Question  What epidemiological and clinical characteristics are associated with the development of critical illness among patients with novel corona virus disease-2019 (COVID-19)? Can these characteristics be used to predict which patients admitted to the hospital with COVID-19 will need admission to an intensive care unit, mechanical ventilation, or will die?

Findings  In this study with a development cohort of 1590 patients and a validation cohort of 710 patients, a risk score was developed and validated to predict development of critical illness. We identified 10 independent predictors and developed a risk score (COVID-GRAM) that predicts development of critical illness. The risk score predictors included: chest radiography abnormality, age, hemoptysis, dyspnea, unconsciousness, number of comorbidities, cancer history, neutrophil-to-lymphocyte ratio, lactate dehydrogenase, and direct bilirubin.

Meaning  The COVID risk score may help identify patients with COVID-19 who may subsequently develop critical illness.

Abstract

Importance  Early identification of patients with novel corona virus disease 2019 (COVID-19) who may develop critical illness is of great importance and may aid in delivering proper treatment and optimizing use of resources.

Objective  To develop and validate a clinical score at hospital admission for predicting which patients with COVID-19 will develop critical illness based on a nationwide cohort in China.

Design, Setting, and Participants  Collaborating with the National Health Commission of China, we established a retrospective cohort of patients with COVID-19 from 575 hospitals in 31 provincial administrative regions as of January 31, 2020. Epidemiological, clinical, laboratory, and imaging variables ascertained at hospital admission were screened using Least Absolute Shrinkage and Selection Operator (LASSO) and logistic regression to construct a predictive risk score (COVID-GRAM). The score provides an estimate of the risk that a hospitalized patient with COVID-19 will develop critical illness. Accuracy of the score was measured by the area under the receiver operating characteristic curve (AUC). Data from 4 additional cohorts in China hospitalized with COVID-19 were used to validate the score. Data were analyzed between February 20, 2020 and March 17, 2020.

Main Outcomes and Measures  Among patients with COVID-19 admitted to the hospital, critical illness was defined as the composite measure of admission to the intensive care unit, invasive ventilation, or death.

Results  The development cohort included 1590 patients. the mean (SD) age of patients in the cohort was 48.9 (15.7) years; 904 (57.3%) were men. The validation cohort included 710 patients with a mean (SD) age of 48.2 (15.2) years, and 382 (53.8%) were men and 172 (24.2%). From 72 potential predictors, 10 variables were independent predictive factors and were included in the risk score: chest radiographic abnormality (OR, 3.39; 95% CI, 2.14-5.38), age (OR, 1.03; 95% CI, 1.01-1.05), hemoptysis (OR, 4.53; 95% CI, 1.36-15.15), dyspnea (OR, 1.88; 95% CI, 1.18-3.01), unconsciousness (OR, 4.71; 95% CI, 1.39-15.98), number of comorbidities (OR, 1.60; 95% CI, 1.27-2.00), cancer history (OR, 4.07; 95% CI, 1.23-13.43), neutrophil-to-lymphocyte ratio (OR, 1.06; 95% CI, 1.02-1.10), lactate dehydrogenase (OR, 1.002; 95% CI, 1.001-1.004) and direct bilirubin (OR, 1.15; 95% CI, 1.06-1.24). The mean AUC in the development cohort was 0.88 (95% CI, 0.85-0.91) and the AUC in the validation cohort was 0.88 (95% CI, 0.84-0.93). The score has been translated into an online risk calculator that is freely available to the public (http://118.126.104.170/)

Conclusions and Relevance  In this study, a risk score based on characteristics of COVID-19 patients at the time of admission to the hospital was developed that may help predict a patient’s risk of developing critical illness.

Sign in to take quiz and track your certificates

Buy This Activity

JN Learning™ is the home for CME and MOC from the JAMA Network. Search by specialty or US state and earn AMA PRA Category 1 CME Credit™ from articles, audio, Clinical Challenges and more. Learn more about CME/MOC

Article Information

Corresponding Author: Jianxing He, MD, PhD (drjianxing.he@gmail.com), and Nan-Shan Zhong, MD, Guangzhou Institute of Respiratory Health (nanshan@vip.163.com), China State Key Laboratory of Respiratory Disease, National Clinical Research Center for Respiratory Disease, The First Affiliated Hospital of Guangzhou Medical University, 151 Yanjiang Rd, Guangzhou, Guangdong 510120, China.

Accepted for Publication: April 20, 2020.

Published Online: May 12, 2020. doi:10.1001/jamainternmed.2020.2033

Author Contributions: Drs W. Liang and He had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Drs W. Liang, H. Liang, Ou, B. Chen, A. Chen, and C. Li are joint first authors.

Concept and design: W. Liang, H. Liang, A. Chen, Xu, G. Chen, H. Guo, Zhang, Zhong, He.

Acquisition, analysis, or interpretation of data: W. Liang, Ou, B. Chen, C. Li, Y. Li, Guan, Sang, Lu, Xu, J. Guo, Z. Chen, Zhao, S. Li, Zhang, Zhong.

Drafting of the manuscript: W. Liang, H. Liang, B. Chen, C. Li, G. Chen, Zhao, Zhong, He.

Critical revision of the manuscript for important intellectual content: W. Liang, Ou, A. Chen, Y. Li, Guan, Sang, Lu, Xu, H. Guo, J. Guo, Z. Chen, S. Li, Zhang, Zhong.

Statistical analysis: W. Liang, H. Liang, Ou, B. Chen, H. Guo, Z. Chen, He.

Obtained funding: W. Liang, A. Chen, Xu.

Administrative, technical, or material support: Y. Li, G. Chen, J. Guo, Z. Chen, S. Li, Zhong.

Supervision: Sang, Lu, Zhang, Zhong.

Other - Construction of the online calculator: B. Chen.

Conflict of Interest Disclosures: None reported.

Funding/Support: This study is supported by China National Science Foundation (Grant No. 81871893), Key Project of Guangzhou Scientific Research Project (Grant No. 201804020030), High-level university construction project of Guangzhou medical university (Grant No. 20182737, 201721007, 201715907, 2017160107); National key R & D Program (Grant No. 2017YFC0907903 & 2017YFC0112704) and the Guangdong high level hospital construction “reaching peak” plan.

Role of the Funder/Sponsor: The funding organizations had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

The China Medical Treatment Expert Group for COVID-19: Zong-jiu Zhang, MD, Ya-hui Jiao, MD, Bin Du, MD, Xin-qiang Gao, MD and Tao Wei, MD (National Health Commission), Yu-fei Duan, MD and Zhi-ling Zhao, MD (Health Commission of Guangdong Province), Yi-min Li, MD, Zi-jing Liang, MD, Nuo-fu Zhang, MD, Shi-yue Li, MD, Qing-hui Huang, MD, Wen-xi Huang, MD, and Ming Li, MD (Guangzhou Institute of Respiratory Health), Zheng Chen, MD, Dong Han, MD, Li Li, MD, Zheng Chen, MD, Zhi-ying Zhan, MD, Jin-jian Chen, MD, Li-jun Xu, MD, Xiao-han Xu, MD (State Key Laboratory of Organ Failure Research, Department of Biostatistics, Guangdong Provincial Key Laboratory of Tropical Disease Research, School of Public Health, Southern Medical University); Li-qiang Wang, MD, Wei-peng Cai, MD, Zi-sheng Chen, MD (the sixth affiliated hospital of Guangzhou medical university), Chang-xing Ou, MD, Xiao-min Peng, MD, Si-ni Cui, MD, Yuan Wang, MD, Mou Zeng, MD, Xin Hao, MD, Qi-hua He, MD, Jing-pei Li, MD, Xu-kai Li, MD, Wei Wang, MD, Li-min Ou, MD, Ya-lei Zhang, MD, Jing-wei Liu, MD, Xin-guo Xiong, MD, Wei-juna Shi, MD, San-mei Yu, MD, Run-dong Qin, MD, Si-yang Yao, MD, Bo-meng Zhang, MD, Xiao-hong Xie, MD, Zhan-hong Xie, MD, Wan-di Wang, MD, Xiao-xian Zhang, MD, Hui-yin Xu, MD, Zi-qing Zhou, MD, Ying Jiang, MD, Ni Liu, MD, Jing-jing Yuan, MD, Zheng Zhu, MD, Jie-xia Zhang, MD, Hong-hao Li, MD, Wei-hua Huang, MD, Lu-lin Wang, MD, Jie-ying Li, MD, Li-fen Gao, MD, Jia-bo Gao, MD, Cai-chen Li, MD, Xue-wei Chen, MD, Jia-bo Gao, MD, Ming-shan Xue, MD, Shou-xie Huang, MD, Jia-man Tang, MD, Wei-li Gu, MD, Jin-lin Wang, MD (Guangzhou Institute of Respiratory Health).

Acknowledgement: Special thanks to Ling Sang, MD, supporting Wuhan Jinyintan Hospital; Yuan-da Xu, MD, supporting Wuhan Union Hospital; Ai-lan Chen MD, supporting Wuhan Hankou Hospital; Guo-qiang Chen, MD, and Hai-yan Guo, MD, Foshan Hospital; and Jun Guo, MD, Daye Hospital; and the China Medical Treatment Expert Group for COVID-19 for providing validation data. See further acknowledgements in the eAppendix in the Supplement.

References
1.
Zhou  F , Yu  T , Du  R ,  et al.  Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study.   Lancet. 2020;395(10229):1054-1062. doi:10.1016/S0140-6736(20)30566-3PubMedGoogle ScholarCrossref
2.
Wu  Z , McGoogan  JM .  Characteristics of and important lessons from the coronavirus disease 2019 (COVID-19) outbreak in China: summary of a report of 72 314 cases from the Chinese Center for Disease Control and Prevention.   JAMA. 2020;(Feb):24. doi:10.1001/jama.2020.2648PubMedGoogle Scholar
3.
Wu  C , Chen  X , Cai  Y ,  et al.  Risk factors associated with acute respiratory distress syndrome and death in patients with coronavirus disease 2019 pneumonia in Wuhan, China.   JAMA Intern Med. 2020;(Mar):13. doi:10.1001/jamainternmed.2020.0994PubMedGoogle Scholar
4.
Guan  WJ , Ni  ZY , Hu  Y ,  et al; China Medical Treatment Expert Group for Covid-19.  Clinical characteristics of coronavirus disease 2019 in China.   N Engl J Med. 2020;(Feb):28. doi:10.1056/NEJMoa2002032PubMedGoogle Scholar
5.
Metlay  JP , Waterer  GW , Long  AC ,  et al.  Diagnosis and treatment of adults with community-acquired pneumonia. an official clinical practice guideline of the American Thoracic Society and Infectious Diseases Society of America.   Am J Respir Crit Care Med. 2019;200(7):e45-e67. doi:10.1164/rccm.201908-1581STPubMedGoogle ScholarCrossref
6.
Gao  HN , Lu  HZ , Cao  B ,  et al.  Clinical findings in 111 cases of influenza A (H7N9) virus infection.   N Engl J Med. 2013;368(24):2277-2285. doi:10.1056/NEJMoa1305584PubMedGoogle ScholarCrossref
7.
Iasonos  A , Schrag  D , Raj  GV , Panageas  KS .  How to build and interpret a nomogram for cancer prognosis.   J Clin Oncol. 2008;26(8):1364-1370. doi:10.1200/JCO.2007.12.9791PubMedGoogle ScholarCrossref
8.
Neill  AM , Martin  IR , Weir  R ,  et al.  Community acquired pneumonia: aetiology and usefulness of severity criteria on admission.   Thorax. 1996;51(10):1010-1016. doi:10.1136/thx.51.10.1010PubMedGoogle ScholarCrossref
9.
Xu  J , Long  JS .  Confidence intervals for predicted outcomes in regression models for categorical outcomes.   Stata Journal. 2005;5(4):537-559. doi:10.1177/1536867X0500500405Google ScholarCrossref
10.
Liang  W , Guan  W , Chen  R ,  et al.  Cancer patients in SARS-CoV-2 infection: a nationwide analysis in China.   Lancet Oncol. 2020;21(3):335-337. doi:10.1016/S1470-2045(20)30096-6PubMedGoogle ScholarCrossref
If you are not a JN Learning subscriber, you can either:
Subscribe to JN Learning for one year
Buy this activity
jn-learning_Modal_LoginSubscribe_Purchase
Close
If you are not a JN Learning subscriber, you can either:
Subscribe to JN Learning for one year
Buy this activity
jn-learning_Modal_LoginSubscribe_Purchase
Close
With a personal account, you can:
  • Access free activities and track your credits
  • Personalize content alerts
  • Customize your interests
  • Fully personalize your learning experience
Education Center Collection Sign In Modal Right
Close

Name Your Search

Save Search
Close
With a personal account, you can:
  • Track your credits
  • Personalize content alerts
  • Customize your interests
  • Fully personalize your learning experience
jn-learning_Modal_SaveSearch_NoAccess_Purchase
Close

Lookup An Activity

or

Close

My Saved Searches

You currently have no searches saved.

Close
With a personal account, you can:
  • Access free activities and track your credits
  • Personalize content alerts
  • Customize your interests
  • Fully personalize your learning experience
Education Center Collection Sign In Modal Right
Close