Physician Agreement With Recommendations Contained in a National Guideline for the Management of Incidental Pulmonary Nodules: A Case Study | Cancer Screening, Prevention, Control | Journal of the American College of Radiology | AMA Ed Hub [Skip to Content]
[Skip to Content Landing]

Physician Agreement With Recommendations Contained in a National Guideline for the Management of Incidental Pulmonary Nodules: A Case Study

Learning Objective
Describe the level of agreement by expert physicians with the Fleischner Society Guidelines for Management of Incidental
1 Credit
Abstract

Since the 1980s, there has been a growing emphasis on the use of evidence-based practice guidelines in medicine. However, national guidelines are based on evidence of variable quality, including non-evidence-based common medical practice. Although attempts are made to ensure impartial recommendations during guideline development, if there is lack of sufficient clinical evidence, some recommendations could be the result of inadvertent bias. At times, guidelines from different organizations on the same clinical entity can also contradict each other.

Description of the Problem

Since the 1980s, there has been a growing emphasis on the use of evidence-based practice guidelines in medicine.1 However, national guidelines are based on evidence of variable quality, including non-evidence-based common medical practice. Although attempts are made to ensure impartial recommendations during guideline development, if there is lack of sufficient clinical evidence, some recommendations could be the result of inadvertent bias.2 At times, guidelines from different organizations on the same clinical entity can also contradict each other.3

Even if guidelines are based on well-conducted clinical trials or other unbiased high-quality evidence, repeated studies have also shown that physician practice is often not concordant with existing guidelines.4 There is similar lack of radiologists' adherence to evidence-based imaging guidelines.5 Several explanations have been proposed for the discrepancy between clinical practice and evidence-based guidelines, including lack of awareness of the guidelines, lengthy and unclear guidelines, decreased willingness to change existing practice patterns, absence of embedded guidelines in clinical workflow, and patient preference.6

Lack of physician agreement with portions of clinical guidelines may be an important contributor to why guidelines are often not broadly followed in clinical practice. We therefore conducted a survey of a multispecialty group of expert physicians in our institution to determine their agreement with the 2017 Fleischner Society Guidelines for Management of Incidental Pulmonary Nodules (FSG), the most frequently referenced imaging guidelines for pulmonary nodules.7

What We Did

Like most guidelines, the FSG are a collection of recommendations based on multiple different clinical scenarios. The 2017 FSG were comprised of 18 unique recommendations, which for the purpose of this study were each codified into a clinical evidence logic statement (CELS), defined as a statement regarding the appropriateness of diagnostic imaging for a certain indication. CELS were formatted as “If … then …” statements, a previously described guideline representation format.8 Representation of recommendations in such a format may simplify the creation of clinical decision support tools to enable and accelerate broad adoption.

For each recommendation, the FSG include ratings for the strength of evidence supporting the recommendation based on the American Society of Chest Physicians grading system. However, before we assessed radiologist agreement with specific recommendations, two medical librarians from the Harvard Library of Evidence analyzed each CELS independently and graded the recommendation based on the supporting clinical studies using the Oxford Centre for Evidence-Based Medicine levels of evidence and the US Preventative Services Task Force I-Scores, an objective and broadly accepted methodology.8 The purpose of internally grading the strength of evidence behind each recommendation, as opposed to relying solely on the FSG's own rating, was to assess whether physician agreement with each guideline recommendation varied according to the internally or FSG-ascertained level of evidence supporting a specific recommendation.

Nine physicians were then surveyed via SurveyMonkey (SurveyMonkey Inc, San Mateo, California) to assess agreement with each of the 18 recommendations, or CELS, contained in the FSG. Physicians were invited to participate in the study based on clinical expertise in pulmonary nodule management or experience with FSG recommendations, to represent a broad range of specialties that would manage patients with incidental pulmonary nodules. Among the nine survey participants were three radiologists with subspecialties in thoracic or cancer imaging, three thoracic surgeons, and three internal medicine and pulmonology physicians. The physicians were given 18 different clinical vignettes, the associated FSG recommendation, the Harvard Library of Evidence grading of evidence, and the FSGs' own grading of evidence (Fig. 1). Survey participants were asked to agree with the FSG recommendation without modification, agree with modification, or disagree. A free text comment box was also provided.

The primary outcome was the percent of physicians who agreed with each FSG recommendation and the corresponding CELS. Physicians who chose to modify the FSG recommendation were considered to have disagreed with the recommendation for our analysis. We compared the proportion of physicians who agreed and disagreed with each of the 18 CELS using a χ2 analysis. We also performed subgroup analysis comparing percent agreement for the different nodule types and risk categories outlined in the FSG.

Outcomes and Limitations

All participants completed the survey. The percent agreement between physicians differed considerably across each of the recommendations, or CELS, ranging from 0% agreement to 100% agreement, with an average agreement rate of 48.2% (Table 1). There was significant variation in the proportion of physicians who agreed with each CELS in the FSG (P < .01). Rates of physician agreement were not associated with FSG grading of the strength of underlying evidence. For instance, 88.9% of physicians agreed with FSG that a single low-risk nodule >8 mm should be managed by a follow-up chest CT at 3 months, PET/CT, or tissue sampling—a recommendation that had an FSG grade of 1A (high-quality evidence). In contrast, the rate of physician agreement for follow-up management of multiple subsolid pulmonary nodules with the most suspicious being equal to or larger than 6 mm was also 88.9%, although that recommendation had an FSG grade of 1C (low- or very low-quality evidence; P = .21). Similar analysis was not performed for the Oxford Centre for Evidence-Based Medicine grading given that all but one recommendation had the same strength of evidence (5-I, based on expert opinion with insufficient evidence).

Table 1. Physician agreement with individual recommendations or CELS from 2017 Fleischer Society Guidelines for the Management of Incidental Pulmonary Nodules

CELSFleischner GuidelineFleischner Society RecommendationOxford GradeFleischner Society GradePhysicians Agreeing With Statement (%)P Value
1Single nodule < 6 mm, low riskNo routine CT follow-up.5-I1C22.2<.01
2Single nodule 6–8 mm, low riskChest CT at 6–12 months, then consider CT at 18–24 months5-I1C66.7
3Single nodule > 8 mm, low riskConsider chest CT at 3 months or PET/CT or tissue sampling5-I1A88.9
4Single nodule < 6 mm, high riskOptional chest CT at 12 months5-I2A0.00
5Single nodule 6–8 mm, high riskChest CT at 6–12 months, then CT at 18–24 months5-I1B33.3
6Single nodule > 8 mm, high riskConsider chest CT at 3 months or PET/CT or tissue sampling5-I1A100.0
7Multiple nodules, most suspicious nodule < 6 mm, low riskNo routine CT follow-up5-I2B11.1
8Multiple nodules, most suspicious nodule 6–8 mm, low riskChest CT at 3–6 months, then consider CT at 18–24 months5-I1B66.7
9Multiple nodules, most suspicious nodule > 8 mm, low riskChest CT at 3–6 months, then consider CT at 18–24 months5-I2A66.7
10Multiple nodules, most suspicious nodule < 6 mm, high riskOptional chest CT at 12 months5-INot stated0.0
11Multiple nodules, most suspicious nodule 6–8 mm, high riskChest CT at 3–6 months, then CT at 18–24 months5-INot stated66.7
12Multiple nodules, most suspicious nodule > 8 mm, high riskChest CT at 3–6 months, then CT at 18–24 months5-INot stated22.2
13Single ground glass nodule < 6 mmNo routine CT follow-up5-NS-contradicts1B22.2
14Single ground glass nodule ≥ 6 mmChest CT at 6–12 months to confirm persistence, then CT every 2 y until 5 y5-I1B88.9
15Single part solid nodule < 6 mmNo routine CT follow-up5-I1C0.0
16Single part solid nodule ≥ 6 mmChest CT at 3–6 months to confirm persistence; if unchanged and solid component remains < 6 mm, annual CT should be performed for 5 y5-I1B77.8
17Multiple subsolid nodules, most suspicious < 6 mmChest CT at 3–6 months; if stable, consider CT at 2 and 4 y5-I1C44.4
18Multiple subsolid nodules, most suspicious >= 6 mmCT at 3–6 months5-I1C88.9

Subgroup analysis was performed to determine if physicians showed higher rates of agreement with FSG recommendations for specific subgroups of nodules (Table 2). CELS recommendations were grouped according to their corresponding nodule description provided in the FSG. CELS regarding the management of large nodules had the highest physician-FSG agreement at 76.2%, compared with only 30.3% agreement for small and medium nodules (P < .01). There was no significant difference in rates of physician agreement with FSG recommendations between high- versus low-risk solid nodules, single versus multiple nodules, or solid versus subsolid nodules. Although a few specific FSG recommendations had 100% agreement, when FSG recommendations were grouped according to their broader nodule description, there was no unanimous acceptance of the FSG guidelines by our local expert panel.

Table 2. Physician agreement by pulmonary nodule subgroup

SubgroupCELS CategoriesAgree (%)P Value
Patient risk
High-risk solid nodules1–359.3.14
Low-risk solid nodules4–644.4
Number of nodules
Single nodule1–6, 13–1650.0.66
Multiple nodules7–12, 17–1845.8
Solid appearance
Solid nodules1–1245.4.40
Subsolid nodules13–1853.7
Nodule size
Small and medium nodules1, 2, 4, 5, 7, 8, 10, 11, 13, 15, 1730.3<.01
Large nodules3, 6, 9, 12, 14, 16, 1876.2

Our work has several limitations. First, the study surveyed only nine physicians in a single large academic institution. However, the study was designed to be exploratory and to highlight and test the hypothesis that guideline nonadherence may be partly affected by lack of physician agreement with guideline component recommendations. Despite this important limitation, the survey participants represented a wide variety of specialties that would manage patients with pulmonary nodules, and many of them had extensive experience with the content of the 2017 FSG.

Guidelines are composed of multiple unique recommendations, and the strength of the underlying evidence supporting each recommendation can vary considerably. The factors that underlie rates of adherence to guidelines are multiple and include the strength of the underlying evidence, the risks and benefits associated with the disease, and as we discussed in this article, physician agreement with the underlying clinical recommendations embedded within those guidelines. The issue of local experts disagreeing with national guidelines, especially those based primarily on professional society consensus rather than high-quality scientific evidence, may be a widespread issue that would benefit from further research.

Sign in to take quiz and track your certificates

To help improve the quality of its educational content and meet applicable education accreditation requirements, the content provider will receive record of your participation and responses to this activity.

As the official journal of the American College of Radiology, JACR will inform, educate, and engage radiologists, giving them the information and tools they need to provide evidence-based, patient-centric imaging care while moving the science of radiology forward. Learn more.

Article Information

This content and the individual contributions contained in it are protected under copyright by the American College of Radiology.

To read more about imaging and patient-centered care, visit the Journal of the American College of Radiology.

Disclosure Statement: The authors state that they have no conflict of interest related to the material discussed in this article. Dr Kapoor, Dr Lacson, Dr Hammer, Dr Cochon, Dr DiPiro, Dr Boland, and Dr Khorasani are nonpartner, non–partnership track employees.

References:
1.
Guiberteau  MJ, Wilcox  PA, Bjork  S.  Practice guidelines: the radiology perspective .  J Am Coll Radiol. 2004;1:92–97. Google ScholarCrossref
2.
Neuman  J, Korenstein  D, Ross  JS, Keyhani  S.  Prevalence of financial conflicts of interest among panel members producing clinical practice guidelines in Canada and United States: cross sectional study .  BMJ. 2011;343. d5621. Google ScholarCrossref
3.
Kachalia  A, Mello  MM.  Breast cancer screening: conflicting guidelines and medicolegal risk .  JAMA. 2013;309:2555–2556. Google ScholarCrossref
4.
Cabana  MD.  Why don't physicians follow clinical practice guidelines? A framework for improvement .  JAMA. 1999;282:1458–1465. Google ScholarCrossref
5.
Bobbin  MD, Ip  IK, Sahni  VA, Shinagare  AB, Khorasani  R.  Focal cystic pancreatic lesion follow-up recommendations after publication of ACR white paper on managing incidental findings .  J Am Coll Radiol. 2017;14:757–764. Google ScholarCrossref
6.
Timmermans  S, Mauck  A.  The promises and pitfalls of evidence-based medicine .  Health Aff (Millwood). 2005;24:18–28. Google ScholarCrossref
7.
Bueno  J, Landeras  L, Chung  JH.  Updated Fleischner Society Guidelines for Managing Incidental Pulmonary Nodules: common questions and challenging scenarios .  Radiographics. 2018;38:1337–1350. Google ScholarCrossref
8.
Lacson  R, Raja  A, Osterbur  D,  et al Assessing strength of evidence of appropriate use criteria for diagnostic imaging examinations .  J Am Med Inform Assoc. 2016;23:649–653. Google ScholarCrossref
Close
Close
Close

Name Your Search

Save Search
Close
Close

Lookup An Activity

or

Close

My Saved Searches

You currently have no searches saved.

Close

My Saved Courses

You currently have no courses saved.

Close
Close