[Skip to Content]
[Skip to Content Landing]

Effect of Text Messaging and Behavioral Interventions on COVID-19 Vaccination UptakeA Randomized Clinical Trial

Educational Objective
To identify the key insights or developments described in this article
1 Credit CME
Key Points

Question  Can text messaging with behavioral insights increase participation in COVID-19 vaccine outreach?

Findings  In this randomized clinical trial comprising 16 045 participants, text messaging did not result in a higher response rate than outbound telephone calls. Behaviorally informed messaging did not result in a significantly higher response than usual content.

Meaning  Text messaging offers a low-cost alternative to outbound telephone calls, but additional efforts are needed to increase vaccine uptake.


Importance  COVID-19 vaccine uptake among urban populations remains low.

Objective  To evaluate whether text messaging with outbound or inbound scheduling and behaviorally informed content might increase COVID-19 vaccine uptake.

Design, Setting, and Participants  This randomized clinical trial with a factorial design was conducted from April 29 to July 6, 2021, in an urban academic health system. The trial comprised 16 045 patients at least 18 years of age in Philadelphia, Pennsylvania, with at least 1 primary care visit in the past 5 years, or a future scheduled primary care visit within the next 3 months, who were unresponsive to prior outreach. The study was prespecified in the trial protocol, and data were obtained from the intent-to-treat population.

Interventions  Eligible patients were randomly assigned in a 1:20:20 ratio to (1) outbound telephone call only by call center, (2) text message and outbound telephone call by call center to those who respond, or (3) text message, with patients instructed to make an inbound telephone call to a hotline. Patients in groups 2 and 3 were concurrently randomly assigned in a 1:1:1:1 ratio to receive different content: standard messaging, clinician endorsement (eg, “Dr. XXX recommends”), scarcity (“limited supply available”), or endowment framing (“We have reserved a COVID-19 vaccine appointment for you”).

Main Outcomes and Measures  The primary outcome was the proportion of patients who completed the first dose of the COVID-19 vaccine within 1 month, according to the electronic health record. Secondary outcomes were the completion of the first dose within 2 months and completion of the vaccination series within 2 months of initial outreach. Additional outcomes included the percentage of patients with invalid cell phone numbers (wrong number or nontextable), no response to text messaging, the percentage of patients scheduled for the vaccine, text message responses, and the number of telephone calls made by the access center. Analysis was on an intention-to-treat basis.

Results  Among the 16 045 patients included, the mean (SD) age was 36.9 (11.1) years; 9418 (58.7%) were women; 12 869 (80.2%) had commercial insurance, and 2283 (14.2%) were insured by Medicaid; 8345 (52.0%) were White, 4706 (29.3%) were Black, and 967 (6.0%) were Hispanic or Latino. At 1 month, 14 of 390 patients (3.6% [95% CI, 1.7%-5.4%]) in the outbound telephone call–only group completed 1 vaccine dose, as did 243 of 7890 patients (3.1% [95% CI, 2.7%-3.5%]) in the text plus outbound call group (absolute difference, −0.5% [95% CI, −2.4% to 1.4%]; P = .57) and 253 of 7765 patients (3.3% [95% CI, 2.9%-3.7%]) in the text plus inbound call group (absolute difference, −0.3% [95% CI, −2.2% to 1.6%]; P = .72). Among the 15 655 patients receiving text messaging, 118 of 3889 patients (3.0% [95% CI, 2.5%-3.6%]) in the standard messaging group completed 1 vaccine dose, as did 135 of 3920 patients (3.4% [95% CI, 2.9%-4.0%]) in the clinician endorsement group (absolute difference, 0.4% [95% CI, −0.4% to 1.2%]; P = .31), 100 of 3911 patients (2.6% [95% CI, 2.1%-3.1%]) in the scarcity group (absolute difference, −0.5% [95% CI, −1.2% to 0.3%]; P = .20), and 143 of 3935 patients (3.6% [95% CI, 3.0%-4.2%]) in the endowment group (absolute difference, 0.6% [95% CI, −0.2% to 1.4%]; P = .14).

Conclusions and Relevance  There was no detectable increase in vaccination uptake among patients receiving text messaging compared with telephone calls only or behaviorally informed message content.

Trial Registration  ClinicalTrials.gov Identifier: NCT04834726

Sign in to take quiz and track your certificates

Buy This Activity

JN Learning™ is the home for CME and MOC from the JAMA Network. Search by specialty or US state and earn AMA PRA Category 1 Credit(s)™ from articles, audio, Clinical Challenges and more. Learn more about CME/MOC

CME Disclosure Statement: Unless noted, all individuals in control of content reported no relevant financial relationships. If applicable, all relevant financial relationships have been mitigated.

Article Information

Accepted for Publication: April 25, 2022.

Published: June 13, 2022. doi:10.1001/jamanetworkopen.2022.16649

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2022 Mehta SJ et al. JAMA Network Open.

Corresponding Author: Shivan J. Mehta, MD, MBA, MSHP, 3600 Civic Center Blvd, 8W-206, Philadelphia, PA 19104 (shivan.mehta@pennmedicine.upenn.edu).

Author Contributions: Dr Mehta had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Mehta, Mallozzi, Shaw, Vandertuyn, Balachandran, Kopinsky, Sevinc, Johnson, Ward, Rosin, Asch.

Acquisition, analysis, or interpretation of data: Mehta, Shaw, Reitz, McDonald, Ward, Park, Snider.

Drafting of the manuscript: Mehta, Reitz, McDonald, Sevinc.

Critical revision of the manuscript for important intellectual content: Mallozzi, Shaw, Vandertuyn, Balachandran, Kopinsky, Johnson, Ward, Park, Snider, Rosin, Asch.

Statistical analysis: Shaw, Reitz, McDonald, Park.

Administrative, technical, or material support: Mehta, Mallozzi, Reitz, McDonald, Vandertuyn, Balachandran, Kopinsky, Sevinc, Johnson, Ward, Park, Rosin.

Supervision: Mehta.

Conflict of Interest Disclosures: Dr Mehta reported receiving grants from the National Cancer Institute of the National Institutes of Health during the conduct of the study and personal fees from the American Gastroenterological Association and Guardant Health outside the submitted work. Dr Shaw reported receiving nonfinancial support from Inovio outside the submitted work; and having a patent owned by the University of Pennsylvania that was licensed by Novartis. Dr Asch reported being a partner and part owner of VAL Health; and receiving personal fees from the American Association for Physician Leadership, North American Center for Continuing Medical Education LLC, and Deloitte outside the submitted work. No other disclosures were reported.

Funding/Support: This trial was funded by the University of Pennsylvania Health System. Dr Mehta was supported by grant K08CA234326 from the National Cancer Institute of the National Institutes of Health.

Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Data Sharing Statement: See Supplement 3.

Barry  V , Dasgupta  S , Weller  DL ,  et al.  Patterns in COVID-19 vaccination coverage, by social vulnerability and urbanicity—United States, December 14, 2020-May 1, 2021.   MMWR Morb Mortal Wkly Rep. 2021;70(22):818-824. doi:10.15585/mmwr.mm7022e1 PubMedGoogle ScholarCrossref
Pew Research Center. Mobile fact sheet. April 7, 2021. Accessed May 9, 2022. https://www.pewinternet.org/fact-sheets/mobile-technology-fact-sheet/
Cole-Lewis  H , Kershaw  T .  Text messaging as a tool for behavior change in disease prevention and management.   Epidemiol Rev. 2010;32:56-69. doi:10.1093/epirev/mxq004 PubMedGoogle ScholarCrossref
Kahneman  D , Tversky  A .  Prospect theory: an analysis of decision under risk.   Econometrica. 1979;47(2):263-291. doi:10.2307/1914185 Google ScholarCrossref
Tversky  A , Kahneman  D .  Judgment under uncertainty: heuristics and biases.   Science. 1974;185(4157):1124-1131. doi:10.1126/science.185.4157.1124 PubMedGoogle ScholarCrossref
Loewenstein  G , Brennan  T , Volpp  KG .  Asymmetric paternalism to improve health behaviors.   JAMA. 2007;298(20):2415-2417. doi:10.1001/jama.298.20.2415 PubMedGoogle ScholarCrossref
Chapman  GB , Li  M , Colby  H , Yoon  H .  Opting in vs opting out of influenza vaccination.   JAMA. 2010;304(1):43-44. doi:10.1001/jama.2010.892 PubMedGoogle ScholarCrossref
Mehta  SJ , Day  SC , Norris  AH ,  et al.  Behavioral interventions to improve population health outreach for hepatitis C screening: randomized clinical trial.   BMJ. 2021;373(1022):n1022. doi:10.1136/bmj.n1022 PubMedGoogle ScholarCrossref
Mehta  SJ , Khan  T , Guerra  C ,  et al.  A randomized controlled trial of opt-in versus opt-out colorectal cancer screening outreach.   Am J Gastroenterol. 2018;113(12):1848-1854. doi:10.1038/s41395-018-0151-3 PubMedGoogle ScholarCrossref
Milkman  KL , Patel  MS , Gandhi  L ,  et al.  A megastudy of text-based nudges encouraging patients to get vaccinated at an upcoming doctor’s appointment.   Proc Natl Acad Sci U S A. 2021;118(20):e2101165118. doi:10.1073/pnas.2101165118 PubMedGoogle ScholarCrossref
Halpern  SD , Ubel  PA , Asch  DA .  Harnessing the power of default options to improve health care.   N Engl J Med. 2007;357(13):1340-1344. doi:10.1056/NEJMsb071595 PubMedGoogle ScholarCrossref
Kahneman  D , Knetsch  JL , Thaler  RH .  Anomalies: the endowment effect, loss aversion, and status quo bias.   J Econ Perspect. 1991;5(1):193-206. doi:10.1257/jep.5.1.193 Google ScholarCrossref
Cialdini  RB .  Influence: Science and Practice. 4th ed. Allyn & Bacon; 2000.
Aggarwal  P , Jun  SY , Huh  JH .  Scarcity messages.   J Advertising. 2011;40(3):19-30. doi:10.2753/JOA0091-3367400302 Google ScholarCrossref
Cialdini  RB , Trost  MR . Social influence: social norms, conformity and compliance. In: Gilbert  DT , Fiske  ST , Lindzey  G , eds.  The Handbook of Social Psychology. 4th ed. McGraw-Hill; 1998:151-192.
Serper  M , Reddy  KR , Bewtra  M , Ahmad  N , Mehta  SJ .  COVID-19 vaccine perceptions among patients with chronic disease in a large gastroenterology and hepatology practice.   Am J Gastroenterol. 2021;116(6):1345-1349. doi:10.14309/ajg.0000000000001270 PubMedGoogle ScholarCrossref
Gargano  LM , Herbert  NL , Painter  JE ,  et al.  Impact of a physician recommendation and parental immunization attitudes on receipt or intention to receive adolescent vaccines.   Hum Vaccin Immunother. 2013;9(12):2627-2633. doi:10.4161/hv.25823 PubMedGoogle ScholarCrossref
Asch  DA , Ziolek  TA , Mehta  SJ .  Misdirections in informed consent—impediments to health care innovation.   N Engl J Med. 2017;377(15):1412-1414. doi:10.1056/NEJMp1707991 PubMedGoogle ScholarCrossref
Asch  DA , Volpp  KG .  On the way to health.   LDI Issue Brief. 2012;17(9):1-4.PubMedGoogle Scholar
StataCorp.  Stata Statistical Software: Release 16. StataCorp LLC; 2019.
Van Rossum  G , Drake  FL .  Python 3 Reference Manual. CreateSpace; 2009.
Seabold  S , Perktold  J . Statsmodels: econometric and statistical modeling with Python. In: Proceedings of the 9th Python in Science Conference; June 28 to July 3, 2010; Austin, Texas.
Dai  H , Saccardo  S , Han  MA ,  et al.  Behavioral nudges increase COVID-19 vaccinations.   Nature. 2021;597(7876):404-409. doi:10.1038/s41586-021-03843-2 PubMedGoogle ScholarCrossref
Santos  HC , Goren  A , Chabris  CF , Meyer  MN .  Effect of targeted behavioral science messages on COVID-19 vaccination registration among employees of a large health system: a randomized trial.   JAMA Netw Open. 2021;4(7):e2118702. doi:10.1001/jamanetworkopen.2021.18702 PubMedGoogle ScholarCrossref
Rabb  N , Swindal  M , Glick  D ,  et al. Text messages do not increase COVID-19 vaccination four weeks after universal eligibility. Accessed April 19, 2022. https://files.osf.io/v1/resources/cg3nt/providers/osfstorage/610d86b5e3801302f5962bda?action=download&direct&version=1
AMA CME Accreditation Information

Credit Designation Statement: The American Medical Association designates this Journal-based CME activity activity for a maximum of 1.00  AMA PRA Category 1 Credit(s)™. Physicians should claim only the credit commensurate with the extent of their participation in the activity.

Successful completion of this CME activity, which includes participation in the evaluation component, enables the participant to earn up to:

  • 1.00 Medical Knowledge MOC points in the American Board of Internal Medicine's (ABIM) Maintenance of Certification (MOC) program;;
  • 1.00 Self-Assessment points in the American Board of Otolaryngology – Head and Neck Surgery’s (ABOHNS) Continuing Certification program;
  • 1.00 MOC points in the American Board of Pediatrics’ (ABP) Maintenance of Certification (MOC) program;
  • 1.00 Lifelong Learning points in the American Board of Pathology’s (ABPath) Continuing Certification program; and
  • 1.00 credit toward the CME [and Self-Assessment requirements] of the American Board of Surgery’s Continuous Certification program

It is the CME activity provider's responsibility to submit participant completion information to ACCME for the purpose of granting MOC credit.

Want full access to the AMA Ed Hub?
After you sign up for AMA Membership, make sure you sign in or create a Physician account with the AMA in order to access all learning activities on the AMA Ed Hub
Buy this activity
Want full access to the AMA Ed Hub?
After you sign up for AMA Membership, make sure you sign in or create a Physician account with the AMA in order to access all learning activities on the AMA Ed Hub
Buy this activity
With a personal account, you can:
  • Access free activities and track your credits
  • Personalize content alerts
  • Customize your interests
  • Fully personalize your learning experience
Education Center Collection Sign In Modal Right

Name Your Search

Save Search
With a personal account, you can:
  • Access free activities and track your credits
  • Personalize content alerts
  • Customize your interests
  • Fully personalize your learning experience

Lookup An Activity


My Saved Searches

You currently have no searches saved.


My Saved Courses

You currently have no courses saved.