[Skip to Content]
[Skip to Content Landing]

Reliability and Validity of an Instrument of COVID-19 Patient-Reported Symptoms in Outpatients

Educational Objective
To identify the key insights or developments described in this article
1 Credit CME
Key Points

Question  Is the Symptoms Evolution of COVID-19 (SE-C19) instrument a valid and reliable tool, able to detect symptom changes in outpatients with laboratory-confirmed COVID-19?

Findings  In this diagnostic/prognostic study of SE-C19 using 657 outpatients with COVID-19 randomized to the NCT04425629 trial, 19 of 23 items from the SE-C19 instrument were identified as valid and reliable to measure disease-related symptoms.

Meaning  These findings suggest that the SE-C19 instrument is a valid and reliable method for identifying symptom resolution among outpatients with COVID-19 and may be useful in the clinical research context.

Abstract

Importance  Patient-reported outcome instruments are key in assessing COVID-19–related symptoms and associated burden. However, a valid and reliable instrument to assess symptom severity and progression among outpatients with COVID-19 is not yet available.

Objectives  To assess the extent to which the Symptoms Evolution of COVID-19 (SE-C19) instrument is valid, reliable, and able to detect symptom changes in outpatients with COVID-19, as well as to establish a definition of symptom resolution.

Design, Setting, and Participants  In this diagnostic/prognostic study, psychometric properties of SE-C19 were assessed in participants recruited into an ongoing, adaptive, phase 1/2/3, randomized, double-blind, placebo-controlled clinical trial, during 2020 to 2022. Adult outpatients with symptomatic COVID-19 were randomized 1:1:1 to receive 2.4 g or 8.0 g intravenous casirivimab and imdevimab or placebo, in outpatient centers at 114 sites, from 2 countries (US and Mexico).

Main Outcomes and Measures  Reliability, validity, and sensitivity to change of the SE-C19 were assessed. SE-C19 and Patient Global Impression of Severity (PGIS) were administered daily from predose at day 1 to day 29.

Results  Analysis was conducted on 657 adult outpatients (342 female patients [52.1%], 562 White patients [85.5%]), and 337 non-Hispanic patients [51.3%]. At baseline, patients reported a mean (SD) of 6.6 (3.9) symptoms (ie, rated as at least mild) with a mean (SD) of 3.8 (3.3) of these symptoms being rated as moderate or severe. Stable patients according to PGIS showed scores with intraclass correlation values indicating moderate-to-good test-retest reliability (ie, 0.50-0.90). At baseline, 20 item scores (87%) varied significantly across PGIS-defined groups, supporting the validity of the SE-C19. A symptom-resolution end point was defined after excluding the item sneezing due to its low ability to discriminate severity levels, and excluding confusion, rash, and vomiting, due to their low prevalence in this population. Symptom resolution required complete absence of all remaining items, except cough, fatigue, and headache, which could be mild or moderate in severity. A total of 19 of 23 items from the SE-C19 instrument were identified as valid and reliable to measure disease-related symptoms in outpatients with COVID-19.

Conclusions and Relevance  This study identified 19 items that are valid and reliable to measure disease-related symptoms in outpatients with COVID-19, and proposed a definition of symptom resolution for potential use in future clinical trials.

Sign in to take quiz and track your certificates

Buy This Activity

JN Learning™ is the home for CME and MOC from the JAMA Network. Search by specialty or US state and earn AMA PRA Category 1 Credit(s)™ from articles, audio, Clinical Challenges and more. Learn more about CME/MOC

CME Disclosure Statement: Unless noted, all individuals in control of content reported no relevant financial relationships. If applicable, all relevant financial relationships have been mitigated.

Article Information

Accepted for Publication: September 12, 2022.

Published: October 28, 2022. doi:10.1001/jamanetworkopen.2022.39053

Open Access: This is an open access article distributed under the terms of the CC-BY-NC-ND License. © 2022 Rofail D et al. JAMA Network Open.

Corresponding Author: Diana Rofail, PhD, Regeneron Pharmaceuticals, Inc, One Rockwood Rd, Sleepy Hollow, NY 10591 (diana.rofail@regeneron.com).

Author Contributions: Drs Rofail and Griffiths had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Rofail, Griffiths, Hussein, Sivapalasingam, Rodriguez, Mastey, Gwaltney.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Rofail, Griffiths, Podolanczuk, Gwaltney.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Griffiths, Flore, Rodriguez, Gwaltney.

Obtained funding: Rofail.

Administrative, technical, or material support: Rofail, Hussein, Sivapalasingam.

Supervision: Rofail, Hussein, Podolanczuk, Rodriguez, Mastey.

Conflict of Interest Disclosures: Dr Rofail reported being a Regeneron Pharmaceuticals, Inc employee, and a former Roche employee and current stockholder. Dr Griffiths reported being an employee of IQVIA, which received consulting fees from Regeneron Pharmaceuticals, Inc for data collection, data analysis, and to support data interpretation during the conduct of the study. Mr Flore reported being an employee of IQVIA, which received consulting fees from Regeneron Pharmaceuticals, Inc for data collection, data analysis, and to support data interpretation during the conduct of the study. Dr Hussein reported being a Regeneron Pharmaceuticals, Inc. employee. Dr Podolanczuk reported receiving personal fees from Regeneron Pharmaceuticals, Inc during the conduct of the study, personal fees from Imvaria, Boehringer Ingelheim, EBSCO/Dynamed, and Roche outside the submitted work. Ms Mastey reported being a Regeneron Pharmaceuticals, Inc employee. Dr Gwaltney reported receiving personal fees from IQVIA during the conduct of study and outside the submitted work.No other disclosures were reported.

Funding/Support: This research was funded by Regeneron Pharmaceuticals, Inc.

Role of the Funder/Sponsor: Regeneron Pharmaceuticals, Inc designed, conducted, and supported the trial that collected the electronic symptom diary data and other data analyzed for this study. The finalized database was electronically transferred to IQVIA. Regeneron Pharmaceuticals, Inc was involved in the design of this study and in partnership with IQVIA the determination of the statistical psychometric analysis plan by consensus; supported the analysis of the data, performed by IQVIA; and was involved in the interpretation of results. Regeneron Pharmaceuticals, Inc and all authors were involved in the review and approval of the manuscript, and the decision to submit the manuscript for publication.

Additional Contributions: We thank the study participants, their families, and the clinicians involved in this trial. Medical writing assistance was provided by Hervé Besson, PhD, and Montse Casamayor, MD, PhD, from IQVIA; these writers did not receive any compensation beyond their normal salary. Editorial support was provided by Prime, Knutsford, UK, funded by Regeneron Pharmaceuticals, Inc.

References
1.
Tenforde  MW , Kim  SS , Lindsell  CJ ,  et al; IVY Network Investigators; CDC COVID-19 Response Team.  Symptom duration and risk factors for delayed return to usual health among outpatients with COVID-19 in a multistate health care systems network—United States, March-June 2020.   MMWR Morb Mortal Wkly Rep. 2020;69(30):993-998. doi:10.15585/mmwr.mm6930e1PubMedGoogle ScholarCrossref
2.
Food and Drug Administration. Assessing COVID-19–related symptoms in outpatient adult and adolescent subjects in clinical trials of drugs and biological products for COVID-19 prevention or treatment. Updated September 2020. Accessed July 6, 2022. https://www.fda.gov/media/142143/download
3.
US Centers for Disease Control and Prevention. Symptoms of coronavirus. Updated March 22, 2022. Accessed July 6, 2022. https://www.cdc.gov/coronavirus/2019-ncov/symptoms-testing/symptoms.html
4.
Mizrahi  B , Shilo  S , Rossman  H ,  et al.  Longitudinal symptom dynamics of COVID-19 infection.   Nat Commun. 2020;11(1):6208. doi:10.1038/s41467-020-20053-yPubMedGoogle ScholarCrossref
5.
Blair  PW , Brown  DM , Jang  M ,  et al; Ambulatory COVID Study Team.  The clinical course of COVID-19 in the outpatient setting: a prospective cohort study.   Open Forum Infect Dis. 2021;8(2):ofab007. doi:10.1093/ofid/ofab007PubMedGoogle ScholarCrossref
6.
Fisher  KA , Olson  SM , Tenforde  MW ,  et al.  Symptoms and recovery among adult outpatients with and without COVID-19 at 11 healthcare facilities-July 2020, United States.   Influenza Other Respir Viruses. 2021;15(3):345-351. doi:10.1111/irv.12832PubMedGoogle ScholarCrossref
7.
Pullen  MF , Skipper  CP , Hullsiek  KH ,  et al.  Symptoms of COVID-19 outpatients in the United States.   Open Forum Infect Dis. 2020;7(7):ofaa271. doi:10.1093/ofid/ofaa271PubMedGoogle ScholarCrossref
8.
O’Keefe  JB , Tong  EJ , O’Keefe  GD , Tong  DC .  Description of symptom course in a telemedicine monitoring clinic for acute symptomatic COVID-19: a retrospective cohort study.   BMJ Open. 2021;11(3):e044154. doi:10.1136/bmjopen-2020-044154PubMedGoogle ScholarCrossref
9.
Raveendran  AV , Jayadevan  R , Sashidharan  S .  Long COVID: an overview.   Diabetes Metab Syndr. 2021;15(3):869-875. doi:10.1016/j.dsx.2021.04.007PubMedGoogle ScholarCrossref
10.
Aiyegbusi  OL , Calvert  MJ .  Patient-reported outcomes: central to the management of COVID-19.   Lancet. 2020;396(10250):531. doi:10.1016/S0140-6736(20)31724-4PubMedGoogle ScholarCrossref
11.
Rofail  D , McGale  N , Im  J ,  et al.  Development and content validation of the Symptoms Evolution of COVID-19: a patient-reported electronic daily diary in clinical and real-world studies.   J Patient Rep Outcomes. 2022;6(1):41. doi:10.1186/s41687-022-00448-9PubMedGoogle ScholarCrossref
12.
Weinreich  DM , Sivapalasingam  S , Norton  T ,  et al; Trial Investigators.  REGEN-COV antibody combination and outcomes in outpatients with Covid-19.   N Engl J Med. 2021;385(23):e81. doi:10.1056/NEJMoa2108163PubMedGoogle ScholarCrossref
13.
World Medical Association.  World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects.   JAMA. 2013;310(20):2191-2194. doi:10.1001/jama.2013.281053.Google ScholarCrossref
14.
Food and Drug Administration. FDA patient-focused drug development guidance series for enhancing the incorporation of the patient’s voice in medical product development and regulatory decision making. Updated June 29, 2020. Accessed July 6, 2022. https://www.fda.gov/drugs/development-approval-process-drugs/fda-patient-focused-drug-development-guidance-series-enhancing-incorporation-patients-voice-medical
15.
Wild  D , Grove  A , Martin  M ,  et al; ISPOR Task Force for Translation and Cultural Adaptation.  Principles of good practice for the translation and cultural adaptation process for Patient-Reported Outcomes (PRO) measures: report of the ISPOR Task Force for Translation and Cultural Adaptation.   Value Health. 2005;8(2):94-104. doi:10.1111/j.1524-4733.2005.04054.xPubMedGoogle ScholarCrossref
16.
Yalcin  I , Bump  RC .  Validation of two global impression questionnaires for incontinence.   Am J Obstet Gynecol. 2003;189(1):98-101. doi:10.1067/mob.2003.379PubMedGoogle ScholarCrossref
17.
Dworkin  RH , Turk  DC , Wyrwich  KW ,  et al.  Interpreting the clinical importance of treatment outcomes in chronic pain clinical trials: IMMPACT recommendations.   J Pain. 2008;9(2):105-121. doi:10.1016/j.jpain.2007.09.005PubMedGoogle ScholarCrossref
18.
Shrout  PE , Fleiss  JL .  Intraclass correlations: uses in assessing rater reliability.   Psychol Bull. 1979;86(2):420-428. doi:10.1037/0033-2909.86.2.420PubMedGoogle ScholarCrossref
19.
Koo  TK , Li  MY .  A guideline of selecting and reporting intraclass correlation coefficients for reliability research.   J Chiropr Med. 2016;15(2):155-163. doi:10.1016/j.jcm.2016.02.012PubMedGoogle ScholarCrossref
20.
Coon  CD , Cook  KF .  Moving from significance to real-world meaning: methods for interpreting change in clinical outcome assessment scores.   Qual Life Res. 2018;27(1):33-40. doi:10.1007/s11136-017-1616-3PubMedGoogle ScholarCrossref
21.
Pallant  JF , Tennant  A .  An introduction to the Rasch measurement model: an example using the Hospital Anxiety and Depression Scale (HADS).   Br J Clin Psychol. 2007;46(Pt 1):1-18. doi:10.1348/014466506X96931PubMedGoogle ScholarCrossref
22.
Andrich  D .  A rating formulation for ordered response categories.   Psychometrika. 1978;43:561-573. doi:10.1007/BF02293814Google ScholarCrossref
23.
Kuehn  BM .  Hospital readmission is common among COVID-19 survivors.   JAMA. 2020;324(24):2477. doi:10.1001/jama.2020.23910PubMedGoogle ScholarCrossref
Close
Want full access to the AMA Ed Hub?
After you sign up for AMA Membership, make sure you sign in or create a Physician account with the AMA in order to access all learning activities on the AMA Ed Hub
Buy this activity
Close
Want full access to the AMA Ed Hub?
After you sign up for AMA Membership, make sure you sign in or create a Physician account with the AMA in order to access all learning activities on the AMA Ed Hub
Buy this activity
Close
With a personal account, you can:
  • Access free activities and track your credits
  • Personalize content alerts
  • Customize your interests
  • Fully personalize your learning experience
Education Center Collection Sign In Modal Right
Close

Name Your Search

Save Search
With a personal account, you can:
  • Access free activities and track your credits
  • Personalize content alerts
  • Customize your interests
  • Fully personalize your learning experience
Close
Close

Lookup An Activity

or

My Saved Searches

You currently have no searches saved.

Close

My Saved Courses

You currently have no courses saved.

Close