[Skip to Content]
[Skip to Content Landing]

Development and Psychometric Validation of the Olfactory Dysfunction Outcomes Rating

Educational Objective
To identify the key insights or developments described in this article
1 Credit CME
Key Points

Question  Can a quality-of-life instrument be developed and validated for assessing the physical problems, functional limitations, and emotional consequences associated with olfactory dysfunction (OD)?

Findings  In this survey creation and validation study among 283 patients, the newly created 28-item Olfactory Dysfunction Outcomes Rating (ODOR) was internally consistent, reliable, and valid. The minimal clinically important difference was 15 out of 112 total points.

Meaning  The ODOR is a novel, concise, and valid patient-reported outcome measure of OD-associated quality of life that captures change after a given intervention; thus, it can be used as an outcome measure in future OD trials.

Abstract

Importance  Olfactory dysfunction (OD) is an increasingly common and morbid condition, especially given the ongoing COVID-19 pandemic. Thus, the ability to reproducibly measure smell loss–associated quality of life (QOL) and its response to treatment is paramount.

Objective  To develop and validate a concise and visually appealing smell loss–associated QOL patient-reported outcome measure for OD.

Design, Setting, and Participants  A secondary analysis of comments to an online survey by 1000 patients with olfactory dysfunction published in 2013 was used as the primary source to generate items of the Olfactory Dysfunction Outcomes Rating (ODOR). In addition, 30 patients with OD enrolled in 2 clinical studies at a tertiary care medical center (Washington University) were asked to identify their main concerns associated with smell loss. And finally, 4 otolaryngologists reviewed the items generated from the online survey and the patients’ interviews to identify any additional items. Prospective study design was used for data collection from the 30 patients and 4 otolaryngologists. Prospective study design was used for survey validation. Validation of the ODOR was performed with 283 patients enrolled in several prospective studies at a single institution that completed the ODOR as an outcome measure.

Main Outcomes and Measures  Item generation and selection were the outcomes of ODOR development. The psychometric and clinimetric measures evaluated for validation were internal consistency, test-retest reliability, face and content validity, concurrent validity, and discriminant validity. Minimal clinically important difference was also determined.

Results  The ODOR is a 28-item instrument with each item scored as either no difficulty or very rarely bothered (0) to complete difficulty or very frequently bothered (4) with a total instrument score range of 0 to 112 points. Higher scores indicate higher degree of dysfunction and limitation. Validation in the cohort of 283 patients (mean [SD] age, 47.0 [14.4] years; 198 female participants [73%]; 179 White participants [80%]) revealed that the instrument has high internal consistency (Cronbach α = 0.968), test-retest reliability (r = 0.90 [95% CI, 0.81-0.95]), face validity, content validity, concurrent validity (r = 0.87 [95% CI, 0.80-0.91] compared with the Questionnaire of Olfactory Disorders–Negative Statements; ρ = −0.76 [95% CI, −0.81 to −0.71] compared with a patient-reported symptom severity scale), and divergent validity (mean score difference, −33.9 [95% CI, −38.3 to −29.6] between normosmic patients and hyposmic/anosmic patients). The minimal clinically important difference was 15 points. The estimated time for survey completion was approximately 5 minutes.

Conclusions and Relevance  In this survey creation and validation study, the ODOR was shown to be a novel, concise, reliable, and valid patient-reported outcome measure of OD-associated QOL. It can be used to measure physical problems, functional limitations, and emotional consequences associated with OD and how they change after a given intervention, which is clinically applicable and particularly pertinent given the growing burden of OD associated with COVID-19.

Sign in to take quiz and track your certificates

Buy This Activity

JN Learning™ is the home for CME and MOC from the JAMA Network. Search by specialty or US state and earn AMA PRA Category 1 Credit(s)™ from articles, audio, Clinical Challenges and more. Learn more about CME/MOC

CME Disclosure Statement: Unless noted, all individuals in control of content reported no relevant financial relationships. If applicable, all relevant financial relationships have been mitigated.

Article Information

Accepted for Publication: August 26, 2022.

Published Online: October 20, 2022. doi:10.1001/jamaoto.2022.3299

Corresponding Author: Jake J. Lee, MD, MSCI, 660 S Euclid Ave, Campus Box 8115, St Louis, MO 63110 (jakejlee@wustl.edu).

Author Contributions: Drs Lee and Kallogjeri had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Lee, Kallogjeri, Peterson, Piccirillo.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Lee, Mahadev, Peterson, Piccirillo.

Critical revision of the manuscript for important intellectual content: Kallogjeri, Peterson, Gupta, Khan, Jiramongkolchai, Schneider, Piccirillo.

Statistical analysis: Lee, Mahadev, Kallogjeri.

Obtained funding: Piccirillo.

Administrative, technical, or material support: Lee, Gupta, Khan, Jiramongkolchai.

Supervision: Lee, Piccirillo.

Conflict of Interest Disclosures: Dr Lee reported a copyright for the ODOR questionnaire filed through Washington University’s Office of Technology Management. Dr Kallogjeri reported personal fees for serving as Statistics Editor for JAMA Otolaryngology–Head & Neck Surgery outside the submitted work. Dr Piccirillo had a patent for SNOT and NOSE HHT with royalties paid from Washington University in St Louis . No other disclosures were reported.

Funding/Support: Research reported in this publication was supported by the National Center for Advancing Translational Sciences of the National Institutes of Health (NIH) under Award Number TL1TR002344. Funding for clinical research training was provided by a training grant from the NIH/National Institute on Deafness and Other Communication Disorders, T32DC000022: “Development of Clinician/Researchers in Academic ENT.”

Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH (Mr Khan). The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH (Dr Lee, Ms Mahadev, Dr Peterson, Ms Gupta, and Dr Jiramongkolchai). Dr Piccirillo is the Editor and Dr Kallogjeri is the Statistical Editor of JAMA Otolaryngology–Head & Neck Surgery, but they were not involved in any of the decisions regarding review of the manuscript or its acceptance.

References
1.
Brämerson  A , Johansson  L , Ek  L , Nordin  S , Bende  M .  Prevalence of olfactory dysfunction: the Skövde population-based study.   Laryngoscope. 2004;114(4):733-737. doi:10.1097/00005537-200404000-00026 PubMedGoogle ScholarCrossref
2.
Liu  G , Zong  G , Doty  RL , Sun  Q .  Prevalence and risk factors of taste and smell impairment in a nationwide representative sample of the US population: a cross-sectional study.   BMJ Open. 2016;6(11):e013246. doi:10.1136/bmjopen-2016-013246 PubMedGoogle ScholarCrossref
3.
Kaye  R , Chang  CWD , Kazahaya  K , Brereton  J , Denneny  JC  III .  COVID-19 anosmia reporting tool: initial findings.   Otolaryngol Head Neck Surg. 2020;163(1):132-134. doi:10.1177/0194599820922992 PubMedGoogle ScholarCrossref
4.
Lechien  JR , Chiesa-Estomba  CM , De Siati  DR ,  et al.  Olfactory and gustatory dysfunctions as a clinical presentation of mild-to-moderate forms of the coronavirus disease (COVID-19): a multicenter European study.   Eur Arch Otorhinolaryngol. 2020;277(8):2251-2261. doi:10.1007/s00405-020-05965-1PubMedGoogle ScholarCrossref
5.
Menni  C , Valdes  AM , Freidin  MB ,  et al.  Real-time tracking of self-reported symptoms to predict potential COVID-19.   Nat Med. 2020;26(7):1037-1040. doi:10.1038/s41591-020-0916-2 PubMedGoogle ScholarCrossref
6.
Hummel  T , Nordin  S .  Olfactory disorders and their consequences for quality of life.   Acta Otolaryngol. 2005;125(2):116-121. doi:10.1080/00016480410022787 PubMedGoogle ScholarCrossref
7.
Deems  DA , Doty  RL , Settle  RG ,  et al.  Smell and taste disorders, a study of 750 patients from the University of Pennsylvania Smell and Taste Center.   Arch Otolaryngol Head Neck Surg. 1991;117(5):519-528. doi:10.1001/archotol.1991.01870170065015 PubMedGoogle ScholarCrossref
8.
Keller  A , Malaspina  D .  Hidden consequences of olfactory dysfunction: a patient report series.   BMC Ear Nose Throat Disord. 2013;13(1):8. doi:10.1186/1472-6815-13-8 PubMedGoogle ScholarCrossref
9.
Jiramongkolchai  P , Jones  MS , Peterson  A ,  et al.  Association of olfactory training with neural connectivity in adults with postviral olfactory dysfunction.   JAMA Otolaryngol Head Neck Surg. 2021;147(6):502-509. doi:10.1001/jamaoto.2021.0086 PubMedGoogle ScholarCrossref
10.
Pinto  JM , Wroblewski  KE , Kern  DW , Schumm  LP , McClintock  MK .  Olfactory dysfunction predicts 5-year mortality in older adults.   PLoS One. 2014;9(10):e107541. doi:10.1371/journal.pone.0107541 PubMedGoogle ScholarCrossref
11.
Weldring  T , Smith  SMS .  Patient-reported outcomes (PROs) and patient-reported outcome measures (PROMs).   Health Serv Insights. 2013;6:61-68.PubMedGoogle Scholar
12.
Black  N .  Patient reported outcome measures could help transform healthcare.   BMJ. 2013;346:f167. doi:10.1136/bmj.f167 PubMedGoogle ScholarCrossref
13.
US Department of Health and Human Services FDA Center for Drug Evaluation and Research; US Department of Health and Human Services FDA Center for Biologics Evaluation and Research; US Department of Health and Human Services FDA Center for Devices and Radiological Health.  Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims: draft guidance.   Health Qual Life Outcomes. 2006;4:79-79. doi:10.1186/1477-7525-4-79 PubMedGoogle ScholarCrossref
14.
Juniper  EF , Guyatt  GH , Jaeschke  R . How to develop and validate a new health-related quality of life instrument. In: Spilker  B , ed.  Quality of Life and Pharmacoeconomics in Clinical Trials. Lippincott-Raven Publishers; 1996:49-56.
15.
Piccirillo  JF , Edwards  D , Haiduk  A , Yonan  C , Thawley  SE .  Psychometric and clinimetric validity of the 31-item Rhinosinusitis Outcome Measure (RSOM-31).   Am J Rhinol. 1995;9:297-308. doi:10.2500/105065895781808711 Google ScholarCrossref
16.
Han  P , Su  T , Qin  M , Chen  H , Hummel  T .  A systematic review of olfactory related questionnaires and scales.   Rhinology. 2021;59(2):133-143.PubMedGoogle Scholar
17.
Frasnelli  J , Hummel  T .  Olfactory dysfunction and daily life.   Eur Arch Otorhinolaryngol. 2005;262(3):231-235. doi:10.1007/s00405-004-0796-y PubMedGoogle ScholarCrossref
18.
Simopoulos  E , Katotomichelakis  M , Gouveris  H , Tripsianis  G , Livaditis  M , Danielides  V .  Olfaction-associated quality of life in chronic rhinosinusitis: adaptation and validation of an olfaction-specific questionnaire.   Laryngoscope. 2012;122(7):1450-1454. doi:10.1002/lary.23349 PubMedGoogle ScholarCrossref
19.
Mattos  JL , Schlosser  RJ , Mace  JC , Smith  TL , Soler  ZM .  Establishing the minimal clinically important difference for the Questionnaire of Olfactory Disorders.   Int Forum Allergy Rhinol. 2018;8(9):1041-1046. doi:10.1002/alr.22135 PubMedGoogle ScholarCrossref
20.
Mattos  JL , Bodner  TE , Mace  JC ,  et al.  Psychometric properties of the brief version of the questionnaire of olfactory disorders in patients with chronic rhinosinusitis.   Int Forum Allergy Rhinol. 2021;11(10):1436-1442. doi:10.1002/alr.22800 PubMedGoogle ScholarCrossref
21.
Mattos  JL , Edwards  C , Schlosser  RJ ,  et al.  A brief version of the questionnaire of olfactory disorders in patients with chronic rhinosinusitis.   Int Forum Allergy Rhinol. 2019;9(10):1144-1150. doi:10.1002/alr.22392 PubMedGoogle ScholarCrossref
22.
Kallogjeri  D , Spitznagel  EL  Jr , Piccirillo  JF .  Importance of defining and interpreting a clinically meaningful difference in clinical research.   JAMA Otolaryngol Head Neck Surg. 2020;146(2):101-102. doi:10.1001/jamaoto.2019.3744 PubMedGoogle ScholarCrossref
23.
Boateng  GO , Neilands  TB , Frongillo  EA , Melgar-Quiñonez  HR , Young  SL .  Best practices for developing and validating scales for health, social, and behavioral research: a primer.   Front Public Health. 2018;6:149. doi:10.3389/fpubh.2018.00149PubMedGoogle ScholarCrossref
24.
Feinstein  AR .  Clinimetrics. Yale Univ Press; 1987. doi:10.2307/j.ctt1xp3vbc
25.
Feinstein  AR .  Clinimetric perspectives.   J Chronic Dis. 1987;40(6):635-640. doi:10.1016/0021-9681(87)90027-0 PubMedGoogle ScholarCrossref
26.
Gupta  S , Kallogjeri  D , Farrell  NF ,  et al.  Development and validation of a novel at-home smell assessment.   JAMA Otolaryngol Head Neck Surg. 2022;148(3):252-258. doi:10.1001/jamaoto.2021.3994 PubMedGoogle ScholarCrossref
27.
Gupta  S , Lee  JJ , Perrin  A ,  et al.  Efficacy and safety of saline nasal irrigation plus theophylline for treatment of COVID-19-related olfactory dysfunction: the SCENT2 phase 2 randomized clinical trial.   JAMA Otolaryngol Head Neck Surg. 2022;148(9):830-837. doi:10.1001/jamaoto.2022.1573 PubMedGoogle ScholarCrossref
28.
Jiramongkolchai  P , Peterson  A , Kallogjeri  D ,  et al.  Randomized clinical trial to evaluate mometasone lavage vs spray for patients with chronic rhinosinusitis without nasal polyps who have not undergone sinus surgery.   Int Forum Allergy Rhinol. 2020;10(8):936-943. doi:10.1002/alr.22586 PubMedGoogle ScholarCrossref
29.
Lee  JJ , Gupta  S , Kallogjeri  D , Piccirillo  JF .  Safety of high-dose nasal theophylline irrigation in the treatment of postviral olfactory dysfunction: a dose-escalation study.   JAMA Otolaryngol Head Neck Surg. 2022;148(9):885-886. doi:10.1001/jamaoto.2022.1574 PubMedGoogle ScholarCrossref
30.
Lee  JJ , Peterson  AM , Kallogjeri  D ,  et al.  Smell Changes and Efficacy of Nasal Theophylline (SCENT) irrigation: a randomized controlled trial for treatment of post-viral olfactory dysfunction.   Am J Otolaryngol. 2022;43(2):103299. doi:10.1016/j.amjoto.2021.103299 PubMedGoogle ScholarCrossref
31.
Busner  J , Targum  SD .  The clinical global impressions scale: applying a research tool in clinical practice.   Psychiatry (Edgmont). 2007;4(7):28-37.PubMedGoogle Scholar
32.
Doty  RL , Shaman  P , Dann  M .  Development of the University of Pennsylvania Smell Identification Test: a standardized microencapsulated test of olfactory function.   Physiol Behav. 1984;32(3):489-502. doi:10.1016/0031-9384(84)90269-5 PubMedGoogle ScholarCrossref
33.
Doty  RL , Shaman  P , Kimmelman  CP , Dann  MS .  University of Pennsylvania Smell Identification Test: a rapid quantitative olfactory function test for the clinic.   Laryngoscope. 1984;94(2, pt 1):176-178. doi:10.1288/00005537-198402000-00004 PubMedGoogle ScholarCrossref
34.
Doty  RL .  Office procedures for quantitative assessment of olfactory function.   Am J Rhinol. 2007;21(4):460-473. doi:10.2500/ajr.2007.21.3043 PubMedGoogle ScholarCrossref
35.
Duncan  HJ , Seiden  AM .  Long-term follow-up of olfactory loss secondary to head trauma and upper respiratory tract infection.   Arch Otolaryngol Head Neck Surg. 1995;121(10):1183-1187. doi:10.1001/archotol.1995.01890100087015 PubMedGoogle ScholarCrossref
36.
Rotenberg  BW , Saunders  S , Duggal  N .  Olfactory outcomes after endoscopic transsphenoidal pituitary surgery.   Laryngoscope. 2011;121(8):1611-1613. doi:10.1002/lary.21890 PubMedGoogle ScholarCrossref
37.
Lee  JJ , Thompson  ZS , Piccirillo  JF ,  et al.  Risk factors for patient-reported olfactory dysfunction after endoscopic transsphenoidal hypophysectomy.   JAMA Otolaryngol Head Neck Surg. 2020;146(7):621-629. doi:10.1001/jamaoto.2020.0673 PubMedGoogle ScholarCrossref
38.
Cronbach  LJ .  Coefficient alpha and the internal structure of tests.   Psychometrika. 1951;16:297-334. doi:10.1007/BF02310555 Google ScholarCrossref
39.
Aaronson  N , Alonso  J , Burnam  A ,  et al.  Assessing health status and quality-of-life instruments: attributes and review criteria.   Qual Life Res. 2002;11(3):193-205. doi:10.1023/A:1015291021312 PubMedGoogle ScholarCrossref
40.
Millar Vernetti  P , Perez Lloret  S , Rossi  M , Cerquetti  D , Merello  M .  Validation of a new scale to assess olfactory dysfunction in patients with Parkinson’s disease.   Parkinsonism Relat Disord. 2012;18(4):358-361. doi:10.1016/j.parkreldis.2011.12.001 PubMedGoogle ScholarCrossref
41.
Nordin  S , Brämerson  A , Murphy  C , Bende  M .  A Scandinavian adaptation of the Multi-Clinic Smell and Taste Questionnaire: evaluation of questions about olfaction.   Acta Otolaryngol. 2003;123(4):536-542. doi:10.1080/00016480310001411 PubMedGoogle ScholarCrossref
42.
Pusswald  G , Auff  E , Lehrner  J .  Development of a brief self-report inventory to measure olfactory dysfunction and quality of life in patients with problems with the sense of smell.   Chemosens Percept. 2012;5:292-299. doi:10.1007/s12078-012-9127-7 Google ScholarCrossref
43.
Takebayashi  H , Tsuzuki  K , Oka  H , Fukazawa  K , Daimon  T , Sakagami  M .  Clinical availability of a self-administered odor questionnaire for patients with olfactory disorders.   Auris Nasus Larynx. 2011;38(1):65-72. doi:10.1016/j.anl.2010.05.013 PubMedGoogle ScholarCrossref
44.
Zou  LQ , Linden  L , Cuevas  M ,  et al.  Self-reported mini olfactory questionnaire (Self-MOQ): a simple and useful measurement for the screening of olfactory dysfunction.   Laryngoscope. 2020;130(12):E786-E790. doi:10.1002/lary.28419 PubMedGoogle ScholarCrossref
Close
Want full access to the AMA Ed Hub?
After you sign up for AMA Membership, make sure you sign in or create a Physician account with the AMA in order to access all learning activities on the AMA Ed Hub
Buy this activity
Close
Want full access to the AMA Ed Hub?
After you sign up for AMA Membership, make sure you sign in or create a Physician account with the AMA in order to access all learning activities on the AMA Ed Hub
Buy this activity
Close
With a personal account, you can:
  • Access free activities and track your credits
  • Personalize content alerts
  • Customize your interests
  • Fully personalize your learning experience
Education Center Collection Sign In Modal Right
Close

Name Your Search

Save Search
With a personal account, you can:
  • Access free activities and track your credits
  • Personalize content alerts
  • Customize your interests
  • Fully personalize your learning experience
Close
Close

Lookup An Activity

or

My Saved Searches

You currently have no searches saved.

Close

My Saved Courses

You currently have no courses saved.

Close