[Skip to Content]
[Skip to Content Landing]

Assessment of Clinician Diagnostic Concordance With Video Telemedicine in the Integrated Multispecialty Practice at Mayo Clinic During the Beginning of COVID-19 Pandemic From March to June 2020

Educational Objective
To identify the key insights or developments described in this article
1 Credit CME
Key Points

Question  How concordant to an in-person diagnosis are provisional diagnoses established at a video telemedicine visit for patients presenting with a new clinical problem?

Findings  In this diagnostic study of 2393 patients who underwent a video telemedicine consultation followed by an in-person outpatient visit for the same clinical problem in the same specialty within a 90-day window, the provisional diagnosis established over video telemedicine visit matched the in-person reference standard diagnosis in 86.9% of cases.

Meaning  These findings suggest that video telemedicine visits yield a high degree of diagnostic concordance to in-person visits for most new clinical concerns.

Abstract

Importance  There was a shift in patient volume from in-person to video telemedicine visits during the COVID-19 pandemic.

Objective  To determine the concordance of provisional diagnoses established at a video telemedicine visit with diagnoses established at an in-person visit for patients presenting with a new clinical problem.

Design, Setting, and Participants  This is a diagnostic study of patients who underwent a video telemedicine consultation followed by an in-person outpatient visit for the same clinical problem in the same specialty within a 90-day window. The provisional diagnosis made during the video telemedicine visit was compared with the reference standard diagnosis by 2 blinded, independent medical reviewers. A multivariate logistic regression model was used to determine factors significantly related to diagnostic concordance. The study was conducted at a large academic integrated multispecialty health care institution (Mayo Clinic locations in Rochester, Minnesota; Scottsdale and Phoenix, Arizona; and Jacksonville, Florida; and Mayo Clinic Health System locations in Iowa, Wisconsin, and Minnesota) between March 24 and June 24, 2020. Participants included Mayo Clinic patients residing in the US without age restriction. Data analysis was performed from December 2020 to June 2021.

Exposures  New clinical problem assessed via video telemedicine visit to home using Zoom Care Anyplace integrated into Epic.

Main Outcomes and Measures  Concordance of provisional diagnoses established over video telemedicine visits compared against a reference standard diagnosis.

Results  There were 2393 participants in the analysis. The median (IQR) age of patients was 53 (37-64) years; 1381 (57.7%) identified as female, and 1012 (42.3%) identified as male. Overall, the provisional diagnosis established over video telemedicine visit was concordant with the in-person reference standard diagnosis in 2080 of 2393 cases (86.9%; 95% CI, 85.6%-88.3%). Diagnostic concordance by International Statistical Classification of Diseases and Related Health Problems, Tenth Revision chapter ranged from 64.7% (95% CI, 42.0%-87.4%) for diseases of the ear and mastoid process to 96.8% (95% CI, 94.7%-98.8%) for neoplasms. Diagnostic concordance by medical specialty ranged from 77.3% (95% CI, 64.9%-89.7%) for otorhinolaryngology to 96.0% (92.1%-99.8%) for psychiatry. Specialty care was found to be significantly more likely than primary care to result in video telemedicine diagnoses concordant with a subsequent in-person visit (odds ratio, 1.69; 95% CI, 1.24-2.30; P < .001).

Conclusions and Relevance  This diagnostic study of video telemedicine visits yielded a high degree of diagnostic concordance compared with in-person visits for most new clinical concerns. Some specific clinical circumstances over video telemedicine were associated with a lower diagnostic concordance, and these patients may benefit from timely in-person follow-up.

Sign in to take quiz and track your certificates

Buy This Activity
Our websites may be periodically unavailable between 12:00am CT March 25, 2023 and 4:00pm CT March 26, 2023 for regularly scheduled maintenance.

JN Learning™ is the home for CME and MOC from the JAMA Network. Search by specialty or US state and earn AMA PRA Category 1 Credit(s)™ from articles, audio, Clinical Challenges and more. Learn more about CME/MOC

CME Disclosure Statement: Unless noted, all individuals in control of content reported no relevant financial relationships. If applicable, all relevant financial relationships have been mitigated.

Article Information

Accepted for Publication: July 15, 2022.

Published: September 2, 2022. doi:10.1001/jamanetworkopen.2022.29958

Correction: This article was corrected on October 17, 2022, to fix a misspelled name in Supplement 2.

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2022 Demaerschalk BM et al. JAMA Network Open.

Corresponding Author: Bart M. Demaerschalk, MD, MSc, Department of Neurology and Center for Digital Health, Mayo Clinic College of Medicine and Science, Mayo Clinic Hospital, 5777 E Mayo Blvd, Phoenix, AZ 85054 (demaerschalk.bart@mayo.edu).

Author Contributions: Dr Demaerschalk and Mr Butterfield had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Demaerschalk, Pines, Butterfield, TerKonda, Bushman, Lokken, Blegen, Hoff, Coffey, Anthony, Zhang.

Acquisition, analysis, or interpretation of data: Demaerschalk, Pines, Butterfield, Haglin, Haddad, Yiannias, Colby, TerKonda, Ommen, Bushman, Coffey, Zhang.

Drafting of the manuscript: Demaerschalk, Pines, Butterfield, Haglin, TerKonda, Lokken, Coffey, Anthony.

Critical revision of the manuscript for important intellectual content: Demaerschalk, Pines, Butterfield, Haddad, Yiannias, Colby, TerKonda, Ommen, Bushman, Blegen, Hoff, Coffey, Zhang.

Statistical analysis: Demaerschalk, Butterfield, Haglin, Zhang.

Obtained funding: Demaerschalk, Bushman.

Administrative, technical, or material support: Demaerschalk, TerKonda, Bushman, Lokken, Blegen, Coffey, Anthony.

Supervision: Demaerschalk, Haddad, Ommen.

Conflict of Interest Disclosures: None reported.

Funding/Support: This study was supported by the Mayo Clinic Department of Research (grant 90256993).

Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Group Information: The Diagnostic Accuracy of Telemedicine Utilized at Mayo Clinic Alix School of Medicine Study Group Investigators are listed in Supplement 2.

References
1.
Patel  SY , Mehrotra  A , Huskamp  HA , Uscher-Pines  L , Ganguli  I , Barnett  ML .  Trends in outpatient care delivery and telemedicine during the COVID-19 pandemic in the US.   JAMA Intern Med. 2021;181(3):388-391. doi:10.1001/jamainternmed.2020.5928PubMedGoogle ScholarCrossref
2.
Demeke  HB , Merali  S , Marks  S ,  et al.  Trends in use of telehealth among health centers during the COVID-19 pandemic—United States, June 26-November 6, 2020.   MMWR Morb Mortal Wkly Rep. 2021;70(7):240-244. doi:10.15585/mmwr.mm7007a3PubMedGoogle ScholarCrossref
3.
Alexander  GC , Tajanlangit  M , Heyward  J , Mansour  O , Qato  DM , Stafford  RS .  Use and content of primary care office-based vs telemedicine care visits during the COVID-19 pandemic in the US.   JAMA Netw Open. 2020;3(10):e2021476. doi:10.1001/jamanetworkopen.2020.21476PubMedGoogle ScholarCrossref
4.
Demaerschalk  BM , Blegen  RN , Ommen  SR .  Scalability of telemedicine services in a large integrated multispecialty health care system during COVID-19.   Telemed J E Health. 2021;27(1):96-98. doi:10.1089/tmj.2020.0290PubMedGoogle ScholarCrossref
5.
Herzer  KR , Pronovost  PJ .  Ensuring quality in the era of virtual care.   JAMA. 2021;325(5):429-430. doi:10.1001/jama.2020.24955PubMedGoogle ScholarCrossref
6.
Kahn  JM .  Virtual visits: confronting the challenges of telemedicine.   N Engl J Med. 2015;372(18):1684-1685. doi:10.1056/NEJMp1500533PubMedGoogle ScholarCrossref
7.
Zulman  DM , Verghese  A .  Virtual care, telemedicine visits, and real connection in the era of COVID-19: unforeseen opportunity in the face of adversity.   JAMA. 2021;325(5):437-438. doi:10.1001/jama.2020.27304PubMedGoogle ScholarCrossref
8.
Brown  EM .  The Ontario Telemedicine Network: a case report.   Telemed J E Health. 2013;19(5):373-376. doi:10.1089/tmj.2012.0299PubMedGoogle ScholarCrossref
9.
Uscher-Pines  L , Mehrotra  A .  Analysis of Teladoc use seems to indicate expanded access to care for patients without prior connection to a provider.   Health Aff (Millwood). 2014;33(2):258-264. doi:10.1377/hlthaff.2013.0989PubMedGoogle ScholarCrossref
10.
Lum  HD , Nearing  K , Pimentel  CB , Levy  CR , Hung  WW .  Anywhere to anywhere: use of telehealth to increase health care access for older, rural veterans.   Public Policy Aging Rep. 2020;30(1):12-18. doi:10.1093/ppar/prz030Google ScholarCrossref
11.
Nord  G , Rising  KL , Band  RA , Carr  BG , Hollander  JE .  On-demand synchronous audio video telemedicine visits are cost effective.   Am J Emerg Med. 2019;37(5):890-894. doi:10.1016/j.ajem.2018.08.017PubMedGoogle ScholarCrossref
12.
Harper  K , Roof  M , Wadhawan  N ,  et al.  Vanderbilt University Medical Center ambulatory teleneurology COVID-19 experience.   Telemed J E Health. 2021;27(6):701-705. doi:10.1089/tmj.2020.0382PubMedGoogle ScholarCrossref
13.
Schoenfeld  AJ , Davies  JM , Marafino  BJ ,  et al.  Variation in quality of urgent health care provided during commercial virtual visits.   JAMA Intern Med. 2016;176(5):635-642. doi:10.1001/jamainternmed.2015.8248PubMedGoogle ScholarCrossref
14.
Dixon  RF , Stahl  JE .  A randomized trial of virtual visits in a general medicine practice.   J Telemed Telecare. 2009;15(3):115-117. doi:10.1258/jtt.2009.003003PubMedGoogle ScholarCrossref
15.
Ohta  M , Ohira  Y , Uehara  T ,  et al.  How accurate are first visit diagnoses using synchronous video visits with physicians?   Telemed J E Health. 2017;23(2):119-129. doi:10.1089/tmj.2015.0245PubMedGoogle ScholarCrossref
16.
Graber  M , Schrandt  S . Improving telediagnosis: a call to action—final project findings. Society to Improve Diagnosis in Medicine. 2021. Accessed August 2, 2022. https://www.improvediagnosis.org/wp-content/uploads/2021/09/TeleDx-Final-Report-Update.pdf
17.
Lokken  TG , Blegen  RN , Hoff  MD , Demaerschalk  BM .  Overview for implementation of telemedicine services in a large integrated multispecialty health care system.   Telemed J E Health. 2020;26(4):382-387. doi:10.1089/tmj.2019.0079PubMedGoogle ScholarCrossref
18.
Haddad  TC , Blegen  RN , Prigge  JE ,  et al.  A scalable framework for telehealth: the Mayo Clinic Center for Connected Care response to the COVID-19 pandemic.   Telemed Rep. 2021;2(1):78-87. doi:10.1089/tmr.2020.0032Google ScholarCrossref
19.
Byrt  T , Bishop  J , Carlin  JB .  Bias, prevalence and kappa.   J Clin Epidemiol. 1993;46(5):423-429. doi:10.1016/0895-4356(93)90018-VPubMedGoogle ScholarCrossref
20.
Graham  LE , McGimpsey  S , Wright  S ,  et al.  Could a low-cost audio-visual link be useful in rheumatology?   J Telemed Telecare. 2000;6(suppl 1):S35-S37. doi:10.1258/1357633001934078PubMedGoogle ScholarCrossref
21.
Russell  T , Truter  P , Blumke  R , Richardson  B .  The diagnostic accuracy of telerehabilitation for nonarticular lower-limb musculoskeletal disorders.   Telemed J E Health. 2010;16(5):585-594. doi:10.1089/tmj.2009.0163PubMedGoogle ScholarCrossref
22.
Oakley  AM , Astwood  DR , Loane  M , Duffill  MB , Rademaker  M , Wootton  R .  Diagnostic accuracy of teledermatology: results of a preliminary study in New Zealand.   N Z Med J. 1997;110(1038):51-53.PubMedGoogle Scholar
23.
Wong  HT , Poon  WS , Jacobs  P ,  et al.  The comparative impact of video consultation on emergency neurosurgical referrals.   Neurosurgery. 2006;59(3):607-613. doi:10.1227/01.NEU.0000228926.13395.F9PubMedGoogle ScholarCrossref
24.
Malhotra  S , Chakrabarti  S , Shah  R ,  et al.  Telepsychiatry clinical decision support system used by non-psychiatrists in remote areas: validity and reliability of diagnostic module.   Indian J Med Res. 2017;146(2):196-204. doi:10.4103/ijmr.IJMR_757_15PubMedGoogle ScholarCrossref
25.
Ning  AY , Cabrera  CI , D’Anza  B .  Telemedicine in otolaryngology: a systematic review of image quality, diagnostic concordance, and patient and provider satisfaction.   Ann Otol Rhinol Laryngol. 2021;130(2):195-204. doi:10.1177/0003489420939590PubMedGoogle ScholarCrossref
26.
Hampton  JR , Harrison  MJ , Mitchell  JR , Prichard  JS , Seymour  C .  Relative contributions of history-taking, physical examination, and laboratory investigation to diagnosis and management of medical outpatients.   Br Med J. 1975;2(5969):486-489. doi:10.1136/bmj.2.5969.486PubMedGoogle ScholarCrossref
27.
Peterson  MC , Holbrook  JH , Von Hales  D , Smith  NL , Staker  LV .  Contributions of the history, physical examination, and laboratory investigation in making medical diagnoses.   West J Med. 1992;156(2):163-165. doi:10.1097/00006254-199210000-00013PubMedGoogle ScholarCrossref
AMA CME Accreditation Information

Credit Designation Statement: The American Medical Association designates this Journal-based CME activity activity for a maximum of 1.00  AMA PRA Category 1 Credit(s)™. Physicians should claim only the credit commensurate with the extent of their participation in the activity.

Successful completion of this CME activity, which includes participation in the evaluation component, enables the participant to earn up to:

  • 1.00 Medical Knowledge MOC points in the American Board of Internal Medicine's (ABIM) Maintenance of Certification (MOC) program;;
  • 1.00 Self-Assessment points in the American Board of Otolaryngology – Head and Neck Surgery’s (ABOHNS) Continuing Certification program;
  • 1.00 MOC points in the American Board of Pediatrics’ (ABP) Maintenance of Certification (MOC) program;
  • 1.00 Lifelong Learning points in the American Board of Pathology’s (ABPath) Continuing Certification program; and
  • 1.00 CME points in the American Board of Surgery’s (ABS) Continuing Certification program

It is the CME activity provider's responsibility to submit participant completion information to ACCME for the purpose of granting MOC credit.

Close
Want full access to the AMA Ed Hub?
After you sign up for AMA Membership, make sure you sign in or create a Physician account with the AMA in order to access all learning activities on the AMA Ed Hub
Buy this activity
Close
Want full access to the AMA Ed Hub?
After you sign up for AMA Membership, make sure you sign in or create a Physician account with the AMA in order to access all learning activities on the AMA Ed Hub
Buy this activity
Close
With a personal account, you can:
  • Access free activities and track your credits
  • Personalize content alerts
  • Customize your interests
  • Fully personalize your learning experience
Education Center Collection Sign In Modal Right
Close

Name Your Search

Save Search
With a personal account, you can:
  • Access free activities and track your credits
  • Personalize content alerts
  • Customize your interests
  • Fully personalize your learning experience
Close
Close

Lookup An Activity

or

My Saved Searches

You currently have no searches saved.

Close

My Saved Courses

You currently have no courses saved.

Close