Since the 1980s, there has been a growing emphasis on the use of evidence-based practice guidelines in medicine. However, national guidelines are based on evidence of variable quality, including non-evidence-based common medical practice. Although attempts are made to ensure impartial recommendations during guideline development, if there is lack of sufficient clinical evidence, some recommendations could be the result of inadvertent bias. At times, guidelines from different organizations on the same clinical entity can also contradict each other.
Description of the Problem
Since the 1980s, there has been a growing emphasis on the use of evidence-based practice guidelines in medicine.1 However, national guidelines are based on evidence of variable quality, including non-evidence-based common medical practice. Although attempts are made to ensure impartial recommendations during guideline development, if there is lack of sufficient clinical evidence, some recommendations could be the result of inadvertent bias.2 At times, guidelines from different organizations on the same clinical entity can also contradict each other.3
Even if guidelines are based on well-conducted clinical trials or other unbiased high-quality evidence, repeated studies have also shown that physician practice is often not concordant with existing guidelines.4 There is similar lack of radiologists' adherence to evidence-based imaging guidelines.5 Several explanations have been proposed for the discrepancy between clinical practice and evidence-based guidelines, including lack of awareness of the guidelines, lengthy and unclear guidelines, decreased willingness to change existing practice patterns, absence of embedded guidelines in clinical workflow, and patient preference.6
Lack of physician agreement with portions of clinical guidelines may be an important contributor to why guidelines are often not broadly followed in clinical practice. We therefore conducted a survey of a multispecialty group of expert physicians in our institution to determine their agreement with the 2017 Fleischner Society Guidelines for Management of Incidental Pulmonary Nodules (FSG), the most frequently referenced imaging guidelines for pulmonary nodules.7
Like most guidelines, the FSG are a collection of recommendations based on multiple different clinical scenarios. The 2017 FSG were comprised of 18 unique recommendations, which for the purpose of this study were each codified into a clinical evidence logic statement (CELS), defined as a statement regarding the appropriateness of diagnostic imaging for a certain indication. CELS were formatted as “If … then …” statements, a previously described guideline representation format.8 Representation of recommendations in such a format may simplify the creation of clinical decision support tools to enable and accelerate broad adoption.
For each recommendation, the FSG include ratings for the strength of evidence supporting the recommendation based on the American Society of Chest Physicians grading system. However, before we assessed radiologist agreement with specific recommendations, two medical librarians from the Harvard Library of Evidence analyzed each CELS independently and graded the recommendation based on the supporting clinical studies using the Oxford Centre for Evidence-Based Medicine levels of evidence and the US Preventative Services Task Force I-Scores, an objective and broadly accepted methodology.8 The purpose of internally grading the strength of evidence behind each recommendation, as opposed to relying solely on the FSG's own rating, was to assess whether physician agreement with each guideline recommendation varied according to the internally or FSG-ascertained level of evidence supporting a specific recommendation.
Nine physicians were then surveyed via SurveyMonkey (SurveyMonkey Inc, San Mateo, California) to assess agreement with each of the 18 recommendations, or CELS, contained in the FSG. Physicians were invited to participate in the study based on clinical expertise in pulmonary nodule management or experience with FSG recommendations, to represent a broad range of specialties that would manage patients with incidental pulmonary nodules. Among the nine survey participants were three radiologists with subspecialties in thoracic or cancer imaging, three thoracic surgeons, and three internal medicine and pulmonology physicians. The physicians were given 18 different clinical vignettes, the associated FSG recommendation, the Harvard Library of Evidence grading of evidence, and the FSGs' own grading of evidence (Fig. 1). Survey participants were asked to agree with the FSG recommendation without modification, agree with modification, or disagree. A free text comment box was also provided.
The primary outcome was the percent of physicians who agreed with each FSG recommendation and the corresponding CELS. Physicians who chose to modify the FSG recommendation were considered to have disagreed with the recommendation for our analysis. We compared the proportion of physicians who agreed and disagreed with each of the 18 CELS using a χ2 analysis. We also performed subgroup analysis comparing percent agreement for the different nodule types and risk categories outlined in the FSG.
All participants completed the survey. The percent agreement between physicians differed considerably across each of the recommendations, or CELS, ranging from 0% agreement to 100% agreement, with an average agreement rate of 48.2% (Table 1). There was significant variation in the proportion of physicians who agreed with each CELS in the FSG (P < .01). Rates of physician agreement were not associated with FSG grading of the strength of underlying evidence. For instance, 88.9% of physicians agreed with FSG that a single low-risk nodule >8 mm should be managed by a follow-up chest CT at 3 months, PET/CT, or tissue sampling—a recommendation that had an FSG grade of 1A (high-quality evidence). In contrast, the rate of physician agreement for follow-up management of multiple subsolid pulmonary nodules with the most suspicious being equal to or larger than 6 mm was also 88.9%, although that recommendation had an FSG grade of 1C (low- or very low-quality evidence; P = .21). Similar analysis was not performed for the Oxford Centre for Evidence-Based Medicine grading given that all but one recommendation had the same strength of evidence (5-I, based on expert opinion with insufficient evidence).
Table 1. Physician agreement with individual recommendations or CELS from 2017 Fleischer Society Guidelines for the Management of Incidental Pulmonary Nodules
| CELS | Fleischner Guideline | Fleischner Society Recommendation | Oxford Grade | Fleischner Society Grade | Physicians Agreeing With Statement (%) | P Value |
|---|
| 1 | Single nodule < 6 mm, low risk | No routine CT follow-up. | 5-I | 1C | 22.2 | <.01 |
| 2 | Single nodule 6–8 mm, low risk | Chest CT at 6–12 months, then consider CT at 18–24 months | 5-I | 1C | 66.7 | |
| 3 | Single nodule > 8 mm, low risk | Consider chest CT at 3 months or PET/CT or tissue sampling | 5-I | 1A | 88.9 | |
| 4 | Single nodule < 6 mm, high risk | Optional chest CT at 12 months | 5-I | 2A | 0.00 | |
| 5 | Single nodule 6–8 mm, high risk | Chest CT at 6–12 months, then CT at 18–24 months | 5-I | 1B | 33.3 | |
| 6 | Single nodule > 8 mm, high risk | Consider chest CT at 3 months or PET/CT or tissue sampling | 5-I | 1A | 100.0 | |
| 7 | Multiple nodules, most suspicious nodule < 6 mm, low risk | No routine CT follow-up | 5-I | 2B | 11.1 | |
| 8 | Multiple nodules, most suspicious nodule 6–8 mm, low risk | Chest CT at 3–6 months, then consider CT at 18–24 months | 5-I | 1B | 66.7 | |
| 9 | Multiple nodules, most suspicious nodule > 8 mm, low risk | Chest CT at 3–6 months, then consider CT at 18–24 months | 5-I | 2A | 66.7 | |
| 10 | Multiple nodules, most suspicious nodule < 6 mm, high risk | Optional chest CT at 12 months | 5-I | Not stated | 0.0 | |
| 11 | Multiple nodules, most suspicious nodule 6–8 mm, high risk | Chest CT at 3–6 months, then CT at 18–24 months | 5-I | Not stated | 66.7 | |
| 12 | Multiple nodules, most suspicious nodule > 8 mm, high risk | Chest CT at 3–6 months, then CT at 18–24 months | 5-I | Not stated | 22.2 | |
| 13 | Single ground glass nodule < 6 mm | No routine CT follow-up | 5-NS-contradicts | 1B | 22.2 | |
| 14 | Single ground glass nodule ≥ 6 mm | Chest CT at 6–12 months to confirm persistence, then CT every 2 y until 5 y | 5-I | 1B | 88.9 | |
| 15 | Single part solid nodule < 6 mm | No routine CT follow-up | 5-I | 1C | 0.0 | |
| 16 | Single part solid nodule ≥ 6 mm | Chest CT at 3–6 months to confirm persistence; if unchanged and solid component remains < 6 mm, annual CT should be performed for 5 y | 5-I | 1B | 77.8 | |
| 17 | Multiple subsolid nodules, most suspicious < 6 mm | Chest CT at 3–6 months; if stable, consider CT at 2 and 4 y | 5-I | 1C | 44.4 | |
| 18 | Multiple subsolid nodules, most suspicious >= 6 mm | CT at 3–6 months | 5-I | 1C | 88.9 | |
Subgroup analysis was performed to determine if physicians showed higher rates of agreement with FSG recommendations for specific subgroups of nodules (Table 2). CELS recommendations were grouped according to their corresponding nodule description provided in the FSG. CELS regarding the management of large nodules had the highest physician-FSG agreement at 76.2%, compared with only 30.3% agreement for small and medium nodules (P < .01). There was no significant difference in rates of physician agreement with FSG recommendations between high- versus low-risk solid nodules, single versus multiple nodules, or solid versus subsolid nodules. Although a few specific FSG recommendations had 100% agreement, when FSG recommendations were grouped according to their broader nodule description, there was no unanimous acceptance of the FSG guidelines by our local expert panel.
Table 2. Physician agreement by pulmonary nodule subgroup
| Subgroup | CELS Categories | Agree (%) | P Value |
|---|
| Patient risk | | | |
| High-risk solid nodules | 1–3 | 59.3 | .14 |
| Low-risk solid nodules | 4–6 | 44.4 | |
| Number of nodules | | | |
| Single nodule | 1–6, 13–16 | 50.0 | .66 |
| Multiple nodules | 7–12, 17–18 | 45.8 | |
| Solid appearance | | | |
| Solid nodules | 1–12 | 45.4 | .40 |
| Subsolid nodules | 13–18 | 53.7 | |
| Nodule size | | | |
| Small and medium nodules | 1, 2, 4, 5, 7, 8, 10, 11, 13, 15, 17 | 30.3 | <.01 |
| Large nodules | 3, 6, 9, 12, 14, 16, 18 | 76.2 | |
Our work has several limitations. First, the study surveyed only nine physicians in a single large academic institution. However, the study was designed to be exploratory and to highlight and test the hypothesis that guideline nonadherence may be partly affected by lack of physician agreement with guideline component recommendations. Despite this important limitation, the survey participants represented a wide variety of specialties that would manage patients with pulmonary nodules, and many of them had extensive experience with the content of the 2017 FSG.
Guidelines are composed of multiple unique recommendations, and the strength of the underlying evidence supporting each recommendation can vary considerably. The factors that underlie rates of adherence to guidelines are multiple and include the strength of the underlying evidence, the risks and benefits associated with the disease, and as we discussed in this article, physician agreement with the underlying clinical recommendations embedded within those guidelines. The issue of local experts disagreeing with national guidelines, especially those based primarily on professional society consensus rather than high-quality scientific evidence, may be a widespread issue that would benefit from further research.