[Skip to Content]
[Skip to Content Landing]

Assessment of a Crisis Standards of Care Scoring System for Resource Prioritization and Estimated Excess Mortality by Race, Ethnicity, and Socially Vulnerable Area During a Regional Surge in COVID-19

Educational Objective
To identify the key insights or developments described in this article
1 Credit CME
Key Points

Question  Is a crisis standards of care scoring system designed to allocate scarce resources in the COVID-19 pandemic associated with inequities in resource allocation by race?

Findings  In this cohort study of 498 adults admitted to the intensive care unit and preemptively scored during a COVID-19 surge, nearly twice the proportion of Black patients were scored in the lowest priority group compared with all other patients, a significant difference.

Meaning  These findings suggest that a scoring system designed to maximize lives and life-years saved in the setting of resource scarcity during the COVID-19 pandemic may result in racial inequities in prioritization.


Importance  Crisis standards of care (CSOC) scores designed to allocate scarce resources during the COVID-19 pandemic could exacerbate racial disparities in health care.

Objective  To analyze the association of a CSOC scoring system with resource prioritization and estimated excess mortality by race, ethnicity, and residence in a socially vulnerable area.

Design, Setting, and Participants  This retrospective cohort analysis included adult patients in the intensive care unit during a regional COVID-19 surge from April 13 to May 22, 2020, at 6 hospitals in a health care network in greater Boston, Massachusetts. Participants were scored by acute severity of illness using the Sequential Organ Failure Assessment score and chronic severity of illness using comorbidity and life expectancy scores, and only participants with complete scores were included. The score was ordinal, with cutoff points suggested by the Massachusetts guidelines.

Exposures  Race, ethnicity, Social Vulnerability Index.

Main Outcomes and Measures  The primary outcome was proportion of patients in the lowest priority score category stratified by self-reported race. Secondary outcomes were discrimination and calibration of the score overall and by race, ethnicity, and neighborhood Social Vulnerability Index. Projected excess deaths were modeled by race, using the priority scoring system and a random lottery.

Results  Of 608 patients in the intensive care unit during the study period, 498 had complete data and were included in the analysis; this population had a median (IQR) age of 67 (56-75) years, 191 (38.4%) female participants, 79 (15.9%) Black participants, and 225 patients (45.7%) with COVID-19. The area under the receiver operating characteristic curve for the priority score was 0.79 and was similar across racial groups. Black patients were more likely than others to be in the lowest priority group (12 [15.2%] vs 34 [8.1%]; P = .046). In an exploratory simulation model using the score for ventilator allocation, with only those in the highest priority group receiving ventilators, there were 43.9% excess deaths among Black patients (18 of 41 patients) and 28.6% (58 of 203 patients among all others (P = .05); when the highest and intermediate priority groups received ventilators, there were 4.9% (2 of 41 patients) excess deaths among Black patients and 3.0% (6 of 203) among all others (P = .53). A random lottery resulted in more excess deaths than the score.

Conclusions and Relevance  In this study, a CSOC priority score resulted in lower prioritization of Black patients to receive scarce resources. A model using a random lottery resulted in more estimated excess deaths overall without improving equity by race. CSOC policies must be evaluated for their potential association with racial disparities in health care.

Sign in to take quiz and track your certificates

Buy This Activity

JN Learning™ is the home for CME and MOC from the JAMA Network. Search by specialty or US state and earn AMA PRA Category 1 Credit(s)™ from articles, audio, Clinical Challenges and more. Learn more about CME/MOC

CME Disclosure Statement: Unless noted, all individuals in control of content reported no relevant financial relationships. If applicable, all relevant financial relationships have been mitigated.

Article Information

Accepted for Publication: December 26, 2021.

Published: March 15, 2022. doi:10.1001/jamanetworkopen.2022.1744

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2022 Riviello ED et al. JAMA Network Open.

Corresponding Author: Elisabeth D. Riviello, MD, MPH, Division of Pulmonary, Critical Care, and Sleep Medicine, Beth Israel Deaconess Medical Center, Harvard Medical School, 330 Brookline Ave, Boston, MA 02215 (eriviell@bidmc.harvard.edu).

Author Contributions: Ms Dechen and Dr O’Donoghue had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Mr Sontag and Dr Stevens share senior authorship.

Concept and design: Riviello, Hayes, Mosenthal, Talmor, Sontag, Stevens.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Riviello, Dechen, O’Donoghue, Cocchi, Talmor, Sontag, Stevens.

Critical revision of the manuscript for important intellectual content: Riviello, Dechen, O’Donoghue, Cocchi, Hayes, Molina, Moraco, Mosenthal, Rosenblatt, Walsh, Stevens.

Statistical analysis: Dechen, O’Donoghue, Stevens.

Administrative, technical, or material support: Dechen, Cocchi, Hayes, Molina, Mosenthal, Talmor, Sontag, Stevens.

Supervision: Riviello, Mosenthal.

Conflict of Interest Disclosures: None reported.

Funding/Support: Dr Stevens was supported by grant number K08HS024288 from the Agency for Healthcare Research and Quality.

Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: The content is solely the responsibility of the authors and does not represent the official views of the Agency for Healthcare Research and Quality.

Additional Contributions: We would like to thank Emilie Downing, BS, Director of Market Analytics and Intelligence at Beth Israel Lahey Health, who refined the data collection tool and performed initial analyses for the hospital system Crisis Standards of Care team. We would also like to thank Karla Pollick, MHA, Administrative Director for the Center for Healthcare Delivery Science at Beth Israel Deaconess Medical Center, who provided project management and institutional review board coordination for the study. We would also like to thank Kimberly Clark Ross, BA, Associate Director, Philanthropy Vendor Systems and Compliance at Beth Israel Lahey Health, for her role in coordinating the Crisis Standards of Care leadership team and data collection team. These individuals were not compensated for their time beyond their usual salaries.

Kliff  S , Satariano  A , Silver-Greenberg  J , Kulish  N . There aren’t enough ventilators to cope with the coronavirus. The New York Times. March 18, 2020. Accessed February 1, 2022. https://www.nytimes.com/2020/03/18/business/coronavirus-ventilator-shortage.html
Cleveland Manchanda  EC , Sanky  C , Appel  JM .  Crisis standards of care in the USA: a systematic review and implications for equity amidst COVID-19.   J Racial Ethn Health Disparities. 2021;8(4):824-836.PubMedGoogle ScholarCrossref
Emanuel  EJ , Persad  G , Upshur  R ,  et al.  Fair allocation of scarce medical resources in the time of COVID-19.   N Engl J Med. 2020;382(21):2049-2055. doi:10.1056/NEJMsb2005114PubMedGoogle ScholarCrossref
Maves  RC , Downar  J , Dichter  JR ,  et al; ACCP Task Force for Mass Critical Care.  Triage of scarce critical care resources in COVID-19 an implementation guide for regional allocation: an expert panel report of the Task Force for Mass Critical Care and the American College of Chest Physicians.   Chest. 2020;158(1):212-225. doi:10.1016/j.chest.2020.03.063PubMedGoogle ScholarCrossref
Sprung  CL , Joynt  GM , Christian  MD , Truog  RD , Rello  J , Nates  JL .  Adult ICU triage during the coronavirus disease 2019 pandemic: who will live and who will die? recommendations to improve survival.   Crit Care Med. 2020;48(8):1196-1202. doi:10.1097/CCM.0000000000004410PubMedGoogle ScholarCrossref
Antommaria  AHM , Gibb  TS , McGuire  AL ,  et al; Task Force of the Association of Bioethics Program Directors.  Ventilator triage policies during the COVID-19 pandemic at US hospitals associated with members of the Association of Bioethics Program Directors.   Ann Intern Med. 2020;173(3):188-194. doi:10.7326/M20-1738PubMedGoogle ScholarCrossref
Gershengorn  HB , Holt  GE , Rezk  A ,  et al.  Assessment of disparities associated with a crisis standards of care resource allocation algorithm for patients in 2 US hospitals during the COVID-19 pandemic.   JAMA Netw Open. 2021;4(3):e214149. doi:10.1001/jamanetworkopen.2021.4149PubMedGoogle Scholar
Gold  JAW , Rossen  LM , Ahmad  FB ,  et al.  Race, ethnicity, and age trends in persons who died from COVID-19—United States, May-August 2020.   MMWR Morb Mortal Wkly Rep. 2020;69(42):1517-1521. doi:10.15585/mmwr.mm6942e1PubMedGoogle ScholarCrossref
Price-Haywood  EG , Burton  J , Fort  D , Seoane  L .  Hospitalization and mortality among Black patients and White patients with COVID-19.   N Engl J Med. 2020;382(26):2534-2543. doi:10.1056/NEJMsa2011686PubMedGoogle ScholarCrossref
Cleveland Manchanda  E , Couillard  C , Sivashanker  K .  Inequity in crisis standards of care.   N Engl J Med. 2020;383(4):e16. doi:10.1056/NEJMp2011359PubMedGoogle Scholar
Galiatsatos  P , Kachalia  A , Belcher  HME ,  et al.  xsHealth equity and distributive justice considerations in critical care resource allocation.   Lancet Respir Med. 2020;8(8):758-760. doi:10.1016/S2213-2600(20)30277-0PubMedGoogle ScholarCrossref
Reid  L .  Triage of critical care resources in COVID-19: a stronger role for justice.   J Med Ethics. 2020;46(8):526-530. doi:10.1136/medethics-2020-106320PubMedGoogle ScholarCrossref
Bebinger  M . After uproar, Mass. revises guidelines on who gets an ICU bed or ventilator amid COVID-19 surge. Updated April 22, 2020. Accessed February 1, 2022. https://www.wbur.org/commonhealth/2020/04/20/mass-guidelines-ventilator-covid-coronavirus
White  DB , Lo  B .  A framework for rationing ventilators and critical care beds during the COVID-19 pandemic.   JAMA. 2020;323(18):1773-1774. doi:10.1001/jama.2020.5046PubMedGoogle ScholarCrossref
Donadee  C , Rudd  KE .  Mortality prediction models: another barrier to racial equity in a pandemic.   Am J Respir Crit Care Med. 2021;204(2):120-121. doi:10.1164/rccm.202103-0809EDPubMedGoogle Scholar
Ashana  DC , Anesi  GL , Liu  VX ,  et al.  Equitably allocating resources during crises: racial differences in mortality prediction models.   Am J Respir Crit Care Med. 2021;204(2):178-186. doi:10.1164/rccm.202012-4383OCPubMedGoogle Scholar
Arasteh  K .  Prevalence of comorbidities and risks associated with COVID-19 among Black and Hispanic populations in New York City: an examination of the 2018 New York City Community Health Survey.   J Racial Ethn Health Disparities. 2021;8(4):863-869.PubMedGoogle ScholarCrossref
Miller  WD , Han  X , Peek  ME , Charan Ashana  D , Parker  WF .  Accuracy of the Sequential Organ Failure Assessment score for in-hospital mortality by race and relevance to crisis standards of care.   JAMA Netw Open. 2021;4(6):e2113891. doi:10.1001/jamanetworkopen.2021.13891PubMedGoogle Scholar
The Commonwealth of Massachusetts. Crisis standards of care planning guidance for the COVID-19 pandemic. April 7, 2020. Accessed February 1, 2022. https://mcusercontent.com/b6fca240156200bca3e026661/files/62a0c5ee-f1fc-4cd0-ae26-1f77884e9508/CSC_April_7_2020.pdf
Rosen  A . As coronavirus cases rise, some healthcare workers urge Mass. to change ventilator guidelines. Boston Globe. April 12, 2020. Accessed May 12, 2021. https://www.bostonglobe.com/2020/04/12/metro/state-announces-70-more-deaths-due-coronavirus-2615-new-cases/?p1=BGSearch_Overlay_Results
Santiago  J , Rollins  R , Pressley  A . Bold action needed to battle coronavirus in communities of color. Boston Globe. April 10, 2020. Accessed May 12, 2021. https://www.bostonglobe.com/2020/04/10/opinion/bold-action-needed-battle-coronavirus-communities-color/
Gavin  C . Mass. lawmakers say the state’s ventilator, bed rationing guidelines prioritize whites over patients of color. Boston.com. April 10, 2020. Accessed May 12, 2021. https://www.boston.com/news/local-news/2020/04/10/massachusetts-lawmakers-ventilator-bed-guidelines-race-coronavirus
Atkins Stohr  K . Pressley says ICU, ventilator guidelines negatively affect minorities and wants Baker to rescind them. April 14, 2020. Accessed May 12, 2021. https://www.wbur.org/commonhealth/2020/04/14/ayanna-pressley-critical-care-state-guidance-governor-letter
The Commonwealth of Massachusetts. Crisis standards of care planning guidance for the COVID-19 pandemic. April 7, 2020; revised April 20, 2020. Accessed February 1, 2022. https://www.mass.gov/doc/statewide-advisory-committee-recommendations-for-standards-of-care/download
Vincent  JL , Moreno  R , Takala  J ,  et al.  The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure: on behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine.   Intensive Care Med. 1996;22(7):707-710. doi:10.1007/BF01709751PubMedGoogle ScholarCrossref
Agency for Toxic Substances and Disease Registry. CDC/ATSDR SVI data and documentation download. Accessed May 12, 2021. https://www.atsdr.cdc.gov/placeandhealth/svi/data_documentation_download.html
White  DB , Angus  DC .  A proposed lottery system to allocate scarce COVID-19 medications: promoting fairness and generating knowledge.   JAMA. 2020;324(4):329-330. doi:10.1001/jama.2020.11464PubMedGoogle ScholarCrossref
Schmidt  H , Roberts  DE , Eneanya  ND .  Rationing, racism and justice: advancing the debate around ‘colourblind’ COVID-19 ventilator allocation.   J Med Ethics. 2022;48(2):126-130.PubMedGoogle ScholarCrossref
White  DB , Lo  B .  Mitigating inequities and saving lives with ICU triage during the COVID-19 pandemic.   Am J Respir Crit Care Med. 2021;203(3):287-295. doi:10.1164/rccm.202010-3809CPPubMedGoogle ScholarCrossref
Aoyama  K , D’Souza  R , Pinto  R ,  et al.  Risk prediction models for maternal mortality: a systematic review and meta-analysis.   PLoS One. 2018;13(12):e0208563. doi:10.1371/journal.pone.0208563PubMedGoogle Scholar
Klinger  A , Mueller  A , Sutherland  T ,  et al.  Predicting mortality in adults with suspected infection in a Rwandan hospital: an evaluation of the adapted MEWS, qSOFA and UVA scores.   BMJ Open. 2021;11(2):e040361. doi:10.1136/bmjopen-2020-040361PubMedGoogle Scholar
Wunsch  H , Hill  AD , Bosch  N ,  et al.  Comparison of 2 triage scoring guidelines for allocation of mechanical ventilators.   JAMA Netw Open. 2020;3(12):e2029250. doi:10.1001/jamanetworkopen.2020.29250PubMedGoogle Scholar
Kendi  IX .  How to Be an Antiracist. One World; 2019.
AMA CME Accreditation Information

Credit Designation Statement: The American Medical Association designates this Journal-based CME activity activity for a maximum of 1.00  AMA PRA Category 1 Credit(s)™. Physicians should claim only the credit commensurate with the extent of their participation in the activity.

Successful completion of this CME activity, which includes participation in the evaluation component, enables the participant to earn up to:

  • 1.00 Medical Knowledge MOC points in the American Board of Internal Medicine's (ABIM) Maintenance of Certification (MOC) program;;
  • 1.00 Self-Assessment points in the American Board of Otolaryngology – Head and Neck Surgery’s (ABOHNS) Continuing Certification program;
  • 1.00 MOC points in the American Board of Pediatrics’ (ABP) Maintenance of Certification (MOC) program;
  • 1.00 Lifelong Learning points in the American Board of Pathology’s (ABPath) Continuing Certification program; and
  • 1.00 CME points in the American Board of Surgery’s (ABS) Continuing Certification program

It is the CME activity provider's responsibility to submit participant completion information to ACCME for the purpose of granting MOC credit.

Want full access to the AMA Ed Hub?
After you sign up for AMA Membership, make sure you sign in or create a Physician account with the AMA in order to access all learning activities on the AMA Ed Hub
Buy this activity
Want full access to the AMA Ed Hub?
After you sign up for AMA Membership, make sure you sign in or create a Physician account with the AMA in order to access all learning activities on the AMA Ed Hub
Buy this activity
With a personal account, you can:
  • Access free activities and track your credits
  • Personalize content alerts
  • Customize your interests
  • Fully personalize your learning experience
Education Center Collection Sign In Modal Right

Name Your Search

Save Search
With a personal account, you can:
  • Access free activities and track your credits
  • Personalize content alerts
  • Customize your interests
  • Fully personalize your learning experience

Lookup An Activity


My Saved Searches

You currently have no searches saved.


My Saved Courses

You currently have no courses saved.