[Skip to Content]
[Skip to Content Landing]

Comparison of Quality Performance Measures for Patients Receiving In-Person vs Telemedicine Primary Care in a Large Integrated Health System

Educational Objective
To identify the key insights or developments described in this article
1 Credit CME
Key Points

Question  Is there a difference in standardized quality performance measures for primary care patients exposed to telemedicine compared with patients with office-only (in-person) care?

Findings  In this cohort study of 526 874 patients, telemedicine exposure was associated with significantly better performance or no difference in 13 of 16 comparisons, mostly in testing-based and counseling-based quality measures. Patients with office-only visits had modestly better performance in 3 of 5 medication-based quality measures.

Meaning  Findings suggest that telemedicine exposure in primary care poses a low risk for negatively affecting quality performance, highlighting its potential to suitably augment care capacity.

Abstract

Importance  Despite its rapid adoption during the COVID-19 pandemic, it is unknown how telemedicine augmentation of in-person office visits has affected quality of patient care.

Objective  To examine whether quality of care among patients exposed to telemedicine differs from patients with only in-person office-based care.

Design, Setting, and Participants  In this retrospective cohort study, standardized quality measures were compared between patients with office-only (in-person) visits vs telemedicine visits from March 1, 2020, to November 30, 2021, across more than 200 outpatient care sites in Pennsylvania and Maryland.

Exposures  Patients completing telemedicine (video) visits.

Main Outcomes and Measures  χ2 tests determined statistically significant differences in Health Care Effectiveness Data and Information Set (HEDIS) quality performance measures between office-only and telemedicine-exposed groups. Multivariable logistic regression controlled for sociodemographic factors and comorbidities.

Results  The study included 526 874 patients (409 732 office-only; 117 142 telemedicine exposed) with a comparable distribution of sex (196 285 [49.7%] and 74 878 [63.9%] women), predominance of non-Hispanic (348 127 [85.0%] and 105 408 [90.0%]) and White individuals (334 215 [81.6%] and 100 586 [85.9%]), aged 18 to 65 years (239 938 [58.6%] and 91 100 [77.8%]), with low overall health risk scores (373 176 [91.1%] and 100 076 [85.4%]) and commercial (227 259 [55.5%] and 81 552 [69.6%]) or Medicare or Medicaid (176 671 [43.1%] and 52 513 [44.8%]) insurance. For medication-based measures, patients with office-only visits had better performance, but only 3 of 5 measures had significant differences: patients with cardiovascular disease (CVD) receiving antiplatelets (absolute percentage difference [APD], 6.71%; 95% CI, 5.45%-7.98%; P < .001), patients with CVD receiving statins (APD, 1.79%; 95% CI, 0.88%-2.71%; P = .001), and avoiding antibiotics for patients with upper respiratory infections (APD, 2.05%; 95% CI, 1.17%-2.96%; P < .001); there were insignificant differences for patients with heart failure receiving β-blockers and those with diabetes receiving statins. For all 4 testing-based measures, patients with telemedicine exposure had significantly better performance differences: patients with CVD with lipid panels (APD, 7.04%; 95% CI, 5.95%-8.10%; P < .001), patients with diabetes with hemoglobin A1c testing (APD, 5.14%; 95% CI, 4.25%-6.01%; P < .001), patients with diabetes with nephropathy testing (APD, 9.28%; 95% CI, 8.22%-10.32%; P < .001), and blood pressure control (APD, 3.55%; 95% CI, 3.25%-3.85%; P < .001); this was also true for all 7 counseling-based measures: cervical cancer screening (APD, 12.33%; 95% CI, 11.80%-12.85%; P < .001), breast cancer screening (APD, 16.90%; 95% CI, 16.07%-17.71%; P < .001), colon cancer screening (APD, 8.20%; 95% CI, 7.65%-8.75%; P < .001), tobacco counseling and intervention (APD, 12.67%; 95% CI, 11.84%-13.50%; P < .001), influenza vaccination (APD, 9.76%; 95% CI, 9.47%-10.05%; P < .001), pneumococcal vaccination (APD, 5.41%; 95% CI, 4.85%-6.00%; P < .001), and depression screening (APD, 4.85%; 95% CI, 4.66%-5.04%; P < .001).

Conclusions and Relevance  In this cohort study of patients with telemedicine exposure, there was a largely favorable association with quality of primary care. This supports telemedicine’s value potential for augmenting care capacity, especially in chronic disease management and preventive care. This study also identifies a need for understanding relationships between the optimal blend of telemedicine and in-office care.

Sign in to take quiz and track your certificates

Buy This Activity

JN Learning™ is the home for CME and MOC from the JAMA Network. Search by specialty or US state and earn AMA PRA Category 1 Credit(s)™ from articles, audio, Clinical Challenges and more. Learn more about CME/MOC

CME Disclosure Statement: Unless noted, all individuals in control of content reported no relevant financial relationships. If applicable, all relevant financial relationships have been mitigated.

Article Information

Accepted for Publication: July 31, 2022.

Published: September 26, 2022. doi:10.1001/jamanetworkopen.2022.33267

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2022 Baughman DJ et al. JAMA Network Open.

Corresponding Author: Derek J. Baughman, MD, Family Medicine Residency Program, WellSpan Good Samaritan Hospital, 308 Park Ave, Lebanon, PA 17042 (baughman.derek@gmail.com).

Author Contributions: Drs D. Baughman and Waheed had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: D. Baughman, Jabbarpour, Westfall, Zain, K. Baughman, Waheed.

Acquisition, analysis, or interpretation of data: D. Baughman, Westfall, Jetty, Pollak, Waheed.

Drafting of the manuscript: D. Baughman, Jetty, Zain, K. Baughman, Waheed.

Critical revision of the manuscript for important intellectual content: D. Baughman, Jabbarpour, Westfall, Zain, K. Baughman, Pollak, Waheed.

Statistical analysis: D. Baughman, Jetty, Waheed.

Administrative, technical, or material support: Westfall, K. Baughman, Pollak, Waheed.

Supervision: Jabbarpour, Westfall, Waheed.

Conflict of Interest Disclosures: None reported.

Additional Contributions: Steve Strom, MS, was an expert consulted for SlicerDicer data mining. Theodore Bell, BS, advised on the statistical analysis approach. There were not compensated for their time. The Robert Graham Center staff provided substantial guidance in research methods and editorial feedback.

References
1.
Bodenheimer  T , Sinsky  C .  From triple to quadruple aim: care of the patient requires care of the provider.   Ann Fam Med. 2014;12(6):573-576. doi:10.1370/afm.1713PubMedGoogle ScholarCrossref
2.
Waller  M , Stotler  C .  Telemedicine: a primer.   Curr Allergy Asthma Rep. 2018;18(10):54. doi:10.1007/s11882-018-0808-4PubMedGoogle ScholarCrossref
3.
Eze  ND , Mateus  C , Cravo Oliveira Hashiguchi  T .  Telemedicine in the OECD: an umbrella review of clinical and cost-effectiveness, patient experience and implementation.   PLoS One. 2020;15(8):e0237585. doi:10.1371/journal.pone.0237585PubMedGoogle ScholarCrossref
4.
Powell  RE , Henstenburg  JM , Cooper  G , Hollander  JE , Rising  KL .  Patient perceptions of telehealth primary care video visits.   Ann Fam Med. 2017;15(3):225-229. doi:10.1370/afm.2095PubMedGoogle ScholarCrossref
5.
Player  M , O’Bryan  E , Sederstrom  E , Pinckney  J , Diaz  V .  Electronic visits for common acute conditions: evaluation of a recently established program.   Health Aff (Millwood). 2018;37(12):2024-2030. doi:10.1377/hlthaff.2018.05122PubMedGoogle ScholarCrossref
6.
Donaghy  E , Atherton  H , Hammersley  V ,  et al.  Acceptability, benefits, and challenges of video consulting: a qualitative study in primary care.   Br J Gen Pract. 2019;69(686):e586-e594. doi:10.3399/bjgp19X704141PubMedGoogle ScholarCrossref
7.
de Jong  MJ , Boonen  A , van der Meulen-de Jong  AE ,  et al.  Cost-effectiveness of telemedicine-directed specialized vs standard care for patients with inflammatory bowel diseases in a randomized trial.   Clin Gastroenterol Hepatol. 2020;18(8):1744-1752. doi:10.1016/j.cgh.2020.04.038PubMedGoogle ScholarCrossref
8.
Atmojo  JT , Sudaryanto  WT , Widiyanto  A , Ernawati  E , Arradini  D .  Telemedicine, cost effectiveness, and patients satisfaction: a systematic review.   J Health Policy Manage. 2020;5(2):103-107. doi:10.26911/thejhpm.2020.05.02.02Google ScholarCrossref
9.
Mahar JH, Rosencrance JG, Rasmussen PA. The future of telemedicine (and what’s in the way). Consult QD. Published March 1, 2019. Accessed March 20, 2021. https://consultqd.clevelandclinic.org/the-future-of-telemedicine-and-whats-in-the-way/
10.
Snoswell  CL , Comans  TA .  Does the choice between a telehealth and an in-person appointment change patient attendance?   Telemed J E Health. 2021;27(7):733-738. doi:10.1089/tmj.2020.0176PubMedGoogle ScholarCrossref
11.
Zhang  W , Cheng  B , Zhu  W , Huang  X , Shen  C .  Effect of telemedicine on quality of care in patients with coexisting hypertension and diabetes: a systematic review and meta-analysis.   Telemed J E Health. 2021;27(6):603-614. doi:10.1089/tmj.2020.0122PubMedGoogle ScholarCrossref
12.
Timpel  P , Oswald  S , Schwarz  PEH , Harst  L .  Mapping the evidence on the effectiveness of telemedicine interventions in diabetes, dyslipidemia, and hypertension: an umbrella review of systematic reviews and meta-analyses.   J Med internet Res. 2020;22(3):e16791. doi:10.2196/16791PubMedGoogle ScholarCrossref
13.
Chongmelaxme  B , Lee  S , Dhippayom  T , Saokaew  S , Chaiyakunapruk  N , Dilokthornsakul  P .  The effects of telemedicine on asthma control and patients’ quality of life in adults: a systematic review and meta-analysis.   J Allergy Clin Immunol Pract. 2019;7(1):199-216.e11. doi:10.1016/j.jaip.2018.07.015PubMedGoogle ScholarCrossref
14.
Wakefield  BJ , Holman  JE , Ray  A ,  et al.  Effectiveness of home telehealth in comorbid diabetes and hypertension: a randomized, controlled trial.   Telemed J E Health. 2011;17(4):254-261. doi:10.1089/tmj.2010.0176PubMedGoogle ScholarCrossref
15.
Wakefield  BJ , Holman  JE , Ray  A ,  et al.  Outcomes of a home telehealth intervention for patients with diabetes and hypertension.   Telemed J E Health. 2012;18(8):575-579. doi:10.1089/tmj.2011.0237PubMedGoogle ScholarCrossref
16.
Baughman  D , Ptasinski  A , Baughman  K , Buckwalter  N , Jabbarpour  Y , Waheed  A .  Comparable quality performance of acute low-back pain care in telemedicine and office-based cohorts.   Telemed J E Health. Published online March 28, 2022. doi:10.1089/tmj.2021.0535PubMedGoogle ScholarCrossref
17.
Hasselfeld BW. Benefits of telemedicine. Accessed January 5, 2022. https://www.hopkinsmedicine.org/health/treatment-tests-and-therapies/benefits-of-telemedicine
18.
Gajarawala  SN , Pelkowski  JN .  Telehealth benefits and barriers.   J Nurse Pract. 2021;17(2):218-221. doi:10.1016/j.nurpra.2020.09.013PubMedGoogle ScholarCrossref
19.
Li  KY , Zhu  Z , Ng  S , Ellimoottil  C .  Direct-to-consumer telemedicine visits for acute respiratory infections linked to more downstream visits.   Health Aff (Millwood). 2021;40(4):596-602. doi:10.1377/hlthaff.2020.01741PubMedGoogle ScholarCrossref
20.
Schoenfeld  AJ , Davies  JM , Marafino  BJ ,  et al.  Variation in quality of urgent health care provided during commercial virtual visits.   JAMA Intern Med. 2016;176(5):635-642. doi:10.1001/jamainternmed.2015.8248PubMedGoogle ScholarCrossref
21.
Reed  M , Huang  J , Graetz  I , Muelly  E , Millman  A , Lee  C .  Treatment and follow-up care associated with patient-scheduled primary care telemedicine and in-person visits in a large integrated health system.   JAMA Netw Open. 2021;4(11):e2132793. doi:10.1001/jamanetworkopen.2021.32793PubMedGoogle ScholarCrossref
22.
Mehrotra  A , Bhatia  RS , Snoswell  CL .  Paying for telemedicine after the pandemic.   JAMA. 2021;325(5):431-432. doi:10.1001/jama.2020.25706PubMedGoogle ScholarCrossref
23.
Herzer  KR , Pronovost  PJ .  Ensuring quality in the era of virtual care.   JAMA. 2021;325(5):429-430. doi:10.1001/jama.2020.24955PubMedGoogle ScholarCrossref
24.
Medicare telemedicine health care provider fact sheet. Centers for Medicare & Medicaid Services. Accessed April 19, 2021. https://www.cms.gov/newsroom/fact-sheets/medicare-telemedicine-health-care-provider-fact-sheet
25.
Shuemaker  JC , Phillips  RL , Newton  WP .  Clinical quality measures in a post-pandemic world: measuring what matters in family medicine (ABFM).   Ann Fam Med. 2020;18(4):380-382. doi:10.1370/afm.2564Google ScholarCrossref
26.
National Quality Forum. Measures, reports & tools. Accessed September 1, 2021. https://www.qualityforum.org/measures_reports_tools.aspx
27.
Core Quality Measures Collaborative. AHIP, CMS, and NQF partner to promote measure alignment and burden reduction. Accessed August 14, 2021. https://www.qualityforum.org/cqmc/
28.
Core Quality Measures Collaborative. Consensus core set: ACO and PCMH/Primary Care. Published October 9, 2020. Accessed August 24, 2022. https://www.qualityforum.org/WorkArea/linkit.aspx?LinkIdentifier=id&ItemID=88907
29.
Centers for Medicare & Medicaid Services. Comprehensive primary care plus. Accessed September 1, 2021. https://innovation.cms.gov/innovation-models/comprehensive-primary-care-plus
30.
United States Preventive Services Taskforce. Recommendation topics. Accessed January 2, 2022. https://www.uspreventiveservicestaskforce.org/uspstf/recommendation-topics
31.
Saini  V , Jaber  T , Como  JD , Lejeune  K , Bhanot  N .  623. Exploring ‘Slicer Dicer’, an extraction tool in EPIC, for clinical and epidemiological analysis.   Open Forum Infect Dis. 2021;8(suppl 1):S414-S415. doi:10.1093/ofid/ofab466.821Google ScholarCrossref
32.
Cucinotta  D , Vanelli  M .  WHO declares COVID-19 a pandemic.   Acta Biomed. 2020;91(1):157-160. doi:10.23750/abm.v91i1.9397PubMedGoogle Scholar
33.
National Quality Forum. Measuring performance. Accessed March 20, 2021. https://www.qualityforum.org/Measuring_Performance/Measuring_Performance.aspx
34.
Patel  SY , Mehrotra  A , Huskamp  HA , Uscher-Pines  L , Ganguli  I , Barnett  ML .  Variation in telemedicine use and outpatient care during the COVID-19 pandemic in the United States.   Health Aff (Millwood). 2021;40(2):349-358. doi:10.1377/hlthaff.2020.01786PubMedGoogle ScholarCrossref
35.
Baughman  D , Zain  A , Waheed  A .  Patient adherence to hemoglobin A1c testing recommendations in telemedicine and in-office cohorts during COVID-19.   JAMA Netw Open. 2021;4(9):e2127779. doi:10.1001/jamanetworkopen.2021.27779PubMedGoogle ScholarCrossref
36.
D’Hoore  W , Bouckaert  A , Tilquin  C .  Practical considerations on the use of the Charlson comorbidity index with administrative data bases.   J Clin Epidemiol. 1996;49(12):1429-1433. doi:10.1016/S0895-4356(96)00271-5PubMedGoogle ScholarCrossref
37.
Centers for Medicare & Medicaid Services. Risk adjustment. Accessed March 20, 2021. https://www.cms.gov/Medicare/Health-Plans/MedicareAdvtgSpecRateStats/Risk-Adjustors
38.
Centers for Medicare & Medicaid Services. CMS’ value-based programs. Accessed January 3, 2022. https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/Value-Based-Programs/Value-Based-Programs
39.
Uscher-Pines  L , Mulcahy  A , Cowling  D , Hunter  G , Burns  R , Mehrotra  A .  Access and quality of care in direct-to-consumer telemedicine.   Telemed J E Health. 2016;22(4):282-287. doi:10.1089/tmj.2015.0079PubMedGoogle ScholarCrossref
40.
National Center for Health Statistics. NAMCS and NHAMCS web tables. Published January 29, 2020. Accessed May 8, 2021. https://www.cdc.gov/nchs/ahcd/web_tables.htm
41.
Basu  S , Phillips  RS , Phillips  R , Peterson  LE , Landon  BE .  Primary care practice finances in the United States amid the COVID-19 pandemic.   Health Aff (Millwood). 2020;39(9):1605-1614. doi:10.1377/hlthaff.2020.00794PubMedGoogle ScholarCrossref
42.
Nair  U , Armfield  NR , Chatfield  MD , Edirippulige  S .  The effectiveness of telemedicine interventions to address maternal depression: a systematic review and meta-analysis.   J Telemed Telecare. 2018;24(10):639-650. doi:10.1177/1357633X18794332PubMedGoogle ScholarCrossref
43.
Fortney  JC , Pyne  JM , Mouden  SB ,  et al.  Practice-based versus telemedicine-based collaborative care for depression in rural federally qualified health centers: a pragmatic randomized comparative effectiveness trial.   Am J Psychiatry. 2013;170(4):414-425. doi:10.1176/appi.ajp.2012.12050696PubMedGoogle ScholarCrossref
44.
Balestrieri  M , Sisti  D , Rocchi  M ,  et al.  Effectiveness of clinical decision support systems and telemedicine on outcomes of depression: a cluster randomized trial in general practice.   Fam Pract. 2020;37(6):731-737. doi:10.1093/fampra/cmaa077PubMedGoogle ScholarCrossref
45.
Demeke  HB , Merali  S , Marks  S ,  et al.  Trends in use of telehealth among health centers during the COVID-19 pandemic—United States, June 26–November 6, 2020.   MMWR Morb Mortal Wkly Rep. 2021;70(7):240-244. doi:10.15585/mmwr.mm7007a3Google Scholar
46.
Emanuel  EJ , Mostashari  F , Navathe  AS .  Designing a successful primary care physician capitation model.   JAMA. 2021;325(20):2043-2044. doi:10.1001/jama.2021.5133PubMedGoogle ScholarCrossref
47.
Thomson K. Congress: act now to ensure telehealth access for Medicare beneficiaries. Health Affairs Blog. Accessed June 21, 2021. https://www.healthaffairs.org/do/10.1377/hblog20210505.751442/full/
48.
sturla  N , Kassab  R , Khansa  R ,  et al.  569. Characterization of COVID-19 vaccine breakthrough infections in metropolitan Detroit.   Open Forum Infect Dis. 2021;8(Suppl 1):S386. doi:10.1093/ofid/ofab466.767Google ScholarCrossref
49.
Miyashita  H , Kuno  T .  Prognosis of coronavirus disease 2019 (COVID-19) in patients with HIV infection in New York City.   HIV Med. Published online August 23, 2020. doi:10.1111/hiv.12920PubMedGoogle ScholarCrossref
50.
Walker  DM , Hefner  JL , Fareed  N , Huerta  TR , McAlearney  AS .  Exploring the digital divide: age and race disparities in use of an inpatient portal.   Telemed J E Health. 2020;26(5):603-613. doi:10.1089/tmj.2019.0065PubMedGoogle ScholarCrossref
AMA CME Accreditation Information

Credit Designation Statement: The American Medical Association designates this Journal-based CME activity activity for a maximum of 1.00  AMA PRA Category 1 Credit(s)™. Physicians should claim only the credit commensurate with the extent of their participation in the activity.

Successful completion of this CME activity, which includes participation in the evaluation component, enables the participant to earn up to:

  • 1.00 Medical Knowledge MOC points in the American Board of Internal Medicine's (ABIM) Maintenance of Certification (MOC) program;;
  • 1.00 Self-Assessment points in the American Board of Otolaryngology – Head and Neck Surgery’s (ABOHNS) Continuing Certification program;
  • 1.00 MOC points in the American Board of Pediatrics’ (ABP) Maintenance of Certification (MOC) program;
  • 1.00 Lifelong Learning points in the American Board of Pathology’s (ABPath) Continuing Certification program; and
  • 1.00 CME points in the American Board of Surgery’s (ABS) Continuing Certification program

It is the CME activity provider's responsibility to submit participant completion information to ACCME for the purpose of granting MOC credit.

Close
Want full access to the AMA Ed Hub?
After you sign up for AMA Membership, make sure you sign in or create a Physician account with the AMA in order to access all learning activities on the AMA Ed Hub
Buy this activity
Close
Want full access to the AMA Ed Hub?
After you sign up for AMA Membership, make sure you sign in or create a Physician account with the AMA in order to access all learning activities on the AMA Ed Hub
Buy this activity
Close
With a personal account, you can:
  • Access free activities and track your credits
  • Personalize content alerts
  • Customize your interests
  • Fully personalize your learning experience
Education Center Collection Sign In Modal Right
Close

Name Your Search

Save Search
With a personal account, you can:
  • Access free activities and track your credits
  • Personalize content alerts
  • Customize your interests
  • Fully personalize your learning experience
Close
Close

Lookup An Activity

or

My Saved Searches

You currently have no searches saved.

Close

My Saved Courses

You currently have no courses saved.

Close