[Skip to Content]
Access to paid content on this site is currently suspended due to excessive activity being detected from your IP address 18.207.136.184. Please contact the publisher to request reinstatement.
[Skip to Content Landing]

Development and Validation of a Clinically Based Risk Calculator for the Transdiagnostic Prediction of Psychosis

Educational Objective
To measure the proportion of individuals with a first episode of psychosis detected by At Risk Mental State (ARMS) services in secondary mental health services and to develop and externally validate a practical web-based individualized risk calculator tool for the transdiagnostic prediction of psychosis in secondary mental health care.
1 Credit CME
Key Points

Question  Can we improve the detection of individuals at risk of developing psychosis among patients accessing secondary mental health care?

Finding  This clinical register-based cohort study of 91 199 patients receiving a first index diagnosis of nonorganic and nonpsychotic mental disorder developed and externally validated an online individualized risk calculator tool, based on simple predictor variables that are easily and routinely collected in clinical settings, for the transdiagnostic prediction of psychosis in secondary mental health care.

Meaning  This individualized risk calculator tool can be used by clinicians and researchers, facilitating the prediction of psychosis and the subsequent implementation of preventive focused interventions.

Abstract

Importance  The overall effect of At Risk Mental State (ARMS) services for the detection of individuals who will develop psychosis in secondary mental health care is undetermined.

Objective  To measure the proportion of individuals with a first episode of psychosis detected by ARMS services in secondary mental health services, and to develop and externally validate a practical web-based individualized risk calculator tool for the transdiagnostic prediction of psychosis in secondary mental health care.

Design, Setting, and Participants  Clinical register-based cohort study. Patients were drawn from electronic, real-world, real-time clinical records relating to 2008 to 2015 routine secondary mental health care in the South London and the Maudsley National Health Service Foundation Trust. The study included all patients receiving a first index diagnosis of nonorganic and nonpsychotic mental disorder within the South London and the Maudsley National Health Service Foundation Trust in the period between January 1, 2008, and December 31, 2015. Data analysis began on September 1, 2016.

Main Outcomes and Measures  Risk of development of nonorganic International Statistical Classification of Diseases and Related Health Problems, Tenth Revision psychotic disorders.

Results  A total of 91 199 patients receiving a first index diagnosis of nonorganic and nonpsychotic mental disorder within South London and the Maudsley National Health Service Foundation Trust were included in the derivation (n = 33 820) or external validation (n = 54 716) data sets. The mean age was 32.97 years, 50.88% were men, and 61.05% were white race/ethnicity. The mean follow-up was 1588 days. The overall 6-year risk of psychosis in secondary mental health care was 3.02 (95% CI, 2.88-3.15), which is higher than the 6-year risk in the local general population (0.62). Compared with the ARMS designation, all of the International Statistical Classification of Diseases and Related Health Problems, Tenth Revision diagnoses showed a lower risk of psychosis, with the exception of bipolar mood disorders (similar risk) and brief psychotic episodes (higher risk). The ARMS designation accounted only for a small proportion of transitions to psychosis (n = 52 of 1001; 5.19% in the derivation data set), indicating the need for transdiagnostic prediction of psychosis in secondary mental health care. A prognostic risk stratification model based on preselected variables, including index diagnosis, age, sex, age by sex, and race/ethnicity, was developed and externally validated, showing good performance and potential clinical usefulness.

Conclusions and Relevance  This online individualized risk calculator can be of clinical usefulness for the transdiagnostic prediction of psychosis in secondary mental health care. The risk calculator can help to identify those patients at risk of developing psychosis who require an ARMS assessment and specialized care. The use of this calculator may eventually facilitate the implementation of an individualized provision of preventive focused interventions and improve outcomes of first episode psychosis.

Sign in to take quiz and track your certificates

Buy This Activity

JN Learning™ is the home for CME and MOC from the JAMA Network. Search by specialty or US state and earn AMA PRA Category 1 CME Credit™ from articles, audio, Clinical Challenges and more. Learn more about CME/MOC

Article Information

Corresponding Author: Paolo Fusar-Poli, MD, PhD, Department of Psychosis Studies, Institute of Psychiatry, Psychology and Neuroscience, 5th Floor, PO63, 16 De Crespigny Park, London SE5 8AF, England (paolo.fusar-poli@kcl.ac.uk).

Accepted for Publication: February 8, 2017.

Published Online: March 29, 2017. doi:10.1001/jamapsychiatry.2017.0284

Correction: This article was corrected on May 9, 2018, to fix errors in Table 2.

Author Contributions: Drs Fusar-Poli and Rutigliano had full access to all of the data in the study and can take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Fusar-Poli, Rutigliano, McGuire.

Acquisition, analysis, or interpretation of data: Fusar-Poli, Rutigliano, Stahl, Davies, Bonoldi, Reilly.

Drafting of the manuscript: Fusar-Poli.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Fusar-Poli, Stahl, Davies.

Obtained funding: Fusar-Poli, McGuire.

Administrative, technical, or material support: Fusar-Poli, Rutigliano, Davies, Reilly, McGuire.

Supervision: Fusar-Poli, Stahl, Bonoldi, McGuire.

Conflict of Interest Disclosures: None reported.

Funding/Support: This study was supported in part by a 2014 NARSAD Young Investigator Award to Dr Fusar-Poli and in part by the National Institute for Health Research Biomedical Research Centre at South London and Maudsley National Health Services Foundation Trust and King’s College London.

Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: The views expressed are those of the authors and not necessarily those of the National Health Service, the National Institute for Health Research, or the Department of Health.

References
1.
Millan  MJ, Andrieux  A, Bartzokis  G,  et al.  Altering the course of schizophrenia: progress and perspectives.  Nat Rev Drug Discov. 2016;15(7):485-515.PubMedGoogle ScholarCrossref
2.
Jääskeläinen  E, Juola  P, Hirvonen  N,  et al.  A systematic review and meta-analysis of recovery in schizophrenia.  Schizophr Bull. 2013;39(6):1296-1306.PubMedGoogle ScholarCrossref
3.
Fusar Poli  P, McGorry  P, Kane  J.  Improving outcomes of first episode psychosis.  World Psychiatry. in press.Google Scholar
4.
Yung  AR, McGorry  PD, McFarlane  CA, Jackson  HJ, Patton  GC, Rakkar  A.  Monitoring and care of young people at incipient risk of psychosis.  Schizophr Bull. 1996;22(2):283-303.PubMedGoogle ScholarCrossref
5.
Fusar-Poli  P, Cappucciati  M, Rutigliano  G,  et al.  At risk or not at risk? a meta-analysis of the prognostic accuracy of psychometric interviews for psychosis prediction.  World Psychiatry. 2015;14(3):322-332.PubMedGoogle ScholarCrossref
6.
Braham  A, Bannour  AS, Ben Romdhane  A,  et al.  Validation of the Arabic version of the Comprehensive Assessment of At Risk Mental States (CAARMS) in Tunisian adolescents and young adults.  Early Interv Psychiatry. 2014;8(2):147-154.PubMedGoogle ScholarCrossref
7.
Krebs  MO, Magaud  E, Willard  D,  et al.  Évaluation des états mentaux à risque de transition psychotique: validation de la version française de la CAARMS [Assessment of mental states at risk of psychotic transition: validation of the French version of the CAARMS].  Encephale. 2014;40(6):447-456.PubMedGoogle ScholarCrossref
8.
Fusar-Poli  P, Schultze-Lutter  F.  Predicting the onset of psychosis in patients at clinical high risk: practical guide to probabilistic prognostic reasoning.  Evid Based Ment Health. 2016;19(1):10-15.PubMedGoogle ScholarCrossref
9.
Fusar-Poli  P, Bechdolf  A, Taylor  MJ,  et al.  At risk for schizophrenic or affective psychoses? a meta-analysis of DSM/ICD diagnostic outcomes in individuals at high clinical risk.  Schizophr Bull. 2013;39(4):923-932.PubMedGoogle ScholarCrossref
10.
Webb  JR, Addington  J, Perkins  DO,  et al.  Specificity of incident diagnostic outcomes in patients at clinical high risk for psychosis.  Schizophr Bull. 2015;41(5):1066-1075.PubMedGoogle ScholarCrossref
11.
Fusar-Poli  P, Rutigliano  G, Stahl  D,  et al.  Long-term validity of the At Risk Mental State (ARMS) for predicting psychotic and non-psychotic mental disorders.  Eur Psychiatry. 2016;42:49-54.PubMedGoogle ScholarCrossref
12.
Kempton  MJ, Bonoldi  I, Valmaggia  L, McGuire  P, Fusar-Poli  P.  Speed of psychosis progression in people at ultra-high clinical risk: a complementary meta-analysis.  JAMA Psychiatry. 2015;72(6):622-623.PubMedGoogle ScholarCrossref
13.
Stafford  MR, Jackson  H, Mayo-Wilson  E, Morrison  AP, Kendall  T.  Early interventions to prevent psychosis: systematic review and meta-analysis.  BMJ. 2013;346:f185.PubMedGoogle ScholarCrossref
14.
Deas  G, Kelly  C, Hadjinicolaou  AV, Holt  C, Agius  M, Zaman  R.  An update on: meta-analysis of medical and non-medical treatments of the prodromal phase of psychotic illness in at risk mental states.  Psychiatr Danub. 2016;28(suppl 1):31-38.PubMedGoogle Scholar
15.
NICE. Psychosis and schizophrenia in children and young people: recognition and management. http://www.nice.org.uk/Guidance/CG155. Published 2013. Accessed January 17, 2016.
16.
Fusar-Poli  P, Nelson  B, Valmaggia  L, Yung  AR, McGuire  PK.  Comorbid depressive and anxiety disorders in 509 individuals with an at-risk mental state: impact on psychopathology and transition to psychosis.  Schizophr Bull. 2014;40(1):120-131.PubMedGoogle ScholarCrossref
17.
Conrad  AM, Lewin  TJ, Sly  KA,  et al.  Utility of risk-status for predicting psychosis and related outcomes: evaluation of a 10-year cohort of presenters to a specialised early psychosis community mental health service.  Psychiatry Res. 2017;247:336-344.PubMedGoogle ScholarCrossref
18.
van Os  J, Murray  RM.  Can we identify and treat “schizophrenia light” to prevent true psychotic illness?  BMJ. 2013;346:f304.PubMedGoogle ScholarCrossref
19.
Stewart  R, Soremekun  M, Perera  G,  et al.  The South London and Maudsley NHS Foundation Trust Biomedical Research Centre (SLAM BRC) case register: development and descriptive data.  BMC Psychiatry. 2009;9:51.PubMedGoogle ScholarCrossref
20.
Studerus  E, Ramyead  A, Riecher-Rössler  A.  Prediction of transition to psychosis in patients with a clinical high risk for psychosis: a systematic review of methodology and reporting.  Psychol Med. 2017;1-16.PubMedGoogle Scholar
21.
Benchimol  EI, Smeeth  L, Guttmann  A,  et al; RECORD Working Committee.  The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statement.  PLoS Med. 2015;12(10):e1001885.PubMedGoogle ScholarCrossref
22.
Kaplan  E, Meier  P.  Nonparametric estimation from incomplete observations.  J Am Stat Assoc. 1958;53(282):457-481.Google ScholarCrossref
23.
Greenwood  M.  The Natural Duration of Cancer. London, England: Her Majesty's Stationery Office; 1926.
24.
Kirkbride  JB, Jackson  D, Perez  J,  et al.  A population-level prediction tool for the incidence of first-episode psychosis: translational epidemiology based on cross-sectional data.  BMJ Open. 2013;3(2):e001998.PubMedGoogle ScholarCrossref
25.
Royston  P, Altman  DG.  External validation of a Cox prognostic model: principles and methods.  BMC Med Res Methodol. 2013;13:33.PubMedGoogle ScholarCrossref
26.
Steyerberg  EW, Vickers  AJ, Cook  NR,  et al.  Assessing the performance of prediction models: a framework for traditional and novel measures.  Epidemiology. 2010;21(1):128-138.PubMedGoogle ScholarCrossref
27.
Collins  GS, Reitsma  JB, Altman  DG, Moons  KG.  Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement.  Ann Intern Med. 2015;162(1):55-63.PubMedGoogle ScholarCrossref
28.
Grambsch  P, Therneau  T.  Proportional hazards tests and diagnostics based on weighted residuals.  Biometrika. 1994;81:(3)515-526. doi:10.1093/biomet/81.3.515Google ScholarCrossref
29.
Perera  G, Broadbent  M, Callard  F,  et al.  Cohort profile of the South London and Maudsley NHS Foundation Trust Biomedical Research Centre (SLaM BRC) Case Register: current status and recent enhancement of an Electronic Mental Health Record-derived data resource.  BMJ Open. 2016;6(3):e008721.PubMedGoogle ScholarCrossref
30.
Machin  D, Cheung  Y, Parmar  M.  Survival Analysis: A Practical Approach. 2nd ed. Chichester, England: John Wiley and Sons Ltd; 2006.Crossref
31.
Royston  P.  Explained variation for survival models.  Stata J. 2006;6(1):83-96.Google Scholar
32.
Crowson  CS, Atkinson  EJ, Therneau  TM.  Assessing calibration of prognostic risk scores.  Stat Methods Med Res. 2016;25(4):1692-1706.PubMedGoogle ScholarCrossref
33.
Harrell  F. Regression modeling strategies. http://biostat.mc.vanderbilt.edu/rms. Published 2017. Accessed January 13, 2017.
34.
Collins  GS, Ogundimu  EO, Altman  DG.  Sample size considerations for the external validation of a multivariable prognostic model: a resampling study.  Stat Med. 2016;35(2):214-226.PubMedGoogle ScholarCrossref
35.
Vickers  AJ, Van Calster  B, Steyerberg  EW.  Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests.  BMJ. 2016;352:i6.PubMedGoogle ScholarCrossref
36.
Vickers  AJ, Cronin  AM, Elkin  EB, Gonen  M.  Extensions to decision curve analysis, a novel method for evaluating diagnostic tests, prediction models and molecular markers.  BMC Med Inform Decis Mak. 2008;8:53.PubMedGoogle ScholarCrossref
37.
Hardoon  S, Hayes  JF, Blackburn  R,  et al.  Recording of severe mental illness in United Kingdom primary care, 2000-2010.  PLoS One. 2013;8(12):e82365.PubMedGoogle ScholarCrossref
38.
Francesconi  M, Minichino  A, Carrión  RE,  et al.  Psychosis prediction in secondary mental health services: a broad, comprehensive approach to the “at risk mental state” syndrome.  Eur Psychiatry. 2016;40:96-104.PubMedGoogle ScholarCrossref
39.
Fusar-Poli  P, Cappucciati  M, De Micheli  A,  et al.  Diagnostic and Prognostic Significance of Brief Limited Intermittent Psychotic Symptoms (BLIPS) in individuals at ultra high risk.  Schizophr Bull. 2017;43(1):48-56.PubMedGoogle ScholarCrossref
40.
Fusar-Poli  P, Cappucciati  M, Bonoldi  I,  et al.  Prognosis of brief psychotic episodes: a meta-analysis.  JAMA Psychiatry. 2016;73(3):211-220.PubMedGoogle ScholarCrossref
41.
Fusar-Poli  P, Cappucciati  M, Borgwardt  S,  et al.  Heterogeneity of psychosis risk within individuals at clinical high risk: a meta-analytical stratification.  JAMA Psychiatry. 2016;73(2):113-120.PubMedGoogle ScholarCrossref
42.
Fusar-Poli  P, Byrne  M, Badger  S, Valmaggia  LR, McGuire  PK.  Outreach and support in south London (OASIS), 2001-2011: ten years of early diagnosis and treatment for young individuals at high clinical risk for psychosis.  Eur Psychiatry. 2013;28(5):315-326.PubMedGoogle ScholarCrossref
43.
Schultze-Lutter  F, Michel  C, Schmidt  SJ,  et al.  EPA guidance on the early detection of clinical high risk states of psychoses.  Eur Psychiatry. 2015;30(3):405-416.PubMedGoogle ScholarCrossref
44.
Fusar-Poli  P, Rutigliano  G, Stahl  D,  et al.  Deconstructing pretest risk enrichment to optimize prediction of psychosis in individuals at clinical high risk.  JAMA Psychiatry. 2016;73(12):1260-1267.PubMedGoogle ScholarCrossref
45.
Valmaggia  LR, Byrne  M, Day  F,  et al.  Duration of untreated psychosis and need for admission in patients who engage with mental health services in the prodromal phase.  Br J Psychiatry. 2015;207(2):130-134.PubMedGoogle ScholarCrossref
46.
Woodberry  KA, Seidman  LJ, Bryant  C,  et al.  Treatment precedes positive symptoms in north american adolescent and young adult clinical high risk cohort.  J Clin Child Adolesc Psychol. 2016;1-10.PubMedGoogle Scholar
47.
Carrión  RE, Correll  CU, Auther  AM, Cornblatt  BA.  A severity-based clinical staging model for the psychosis prodrome: longitudinal findings from the new york recognition and prevention program.  Schizophr Bull. 2017;43(1):64-74.PubMedGoogle ScholarCrossref
48.
Addington  J, Liu  L, Perkins  DO, Carrion  RE, Keefe  RS, Woods  SW.  The role of cognition and social functioning as predictors in the transition to psychosis for youth with attenuated psychotic symptoms.  Schizophr Bull. 2017;43(1):57-63.PubMedGoogle ScholarCrossref
49.
Schmidt  A, Cappucciati  M, Radua  J,  et al.  Improving prognostic accuracy in subjects at clinical high risk for psychosis: systematic review of predictive models and meta-analytical sequential testing simulation.  Schizophr Bull. 2016;sbw098.PubMedGoogle Scholar
50.
Clark  SR, Baune  BT, Schubert  KO,  et al.  Prediction of transition from ultra-high risk to first-episode psychosis using a probabilistic model combining history, clinical assessment and fatty-acid biomarkers.  Transl Psychiatry. 2016;6(9):e897.PubMedGoogle ScholarCrossref
51.
Scott  J, Marwaha  S, Ratheesh  A,  et al.  Bipolar at-risk criteria: an examination of which clinical features have optimal utility for identifying youth at risk of early transition from depression to bipolar disorders.  Schizophr Bull. 2016;sbw154.PubMedGoogle Scholar
If you are not a JN Learning subscriber, you can either:
Subscribe to JN Learning for one year
Buy this activity
jn-learning_Modal_LoginSubscribe_Purchase
If you are not a JN Learning subscriber, you can either:
Subscribe to JN Learning for one year
Buy this activity
jn-learning_Modal_LoginSubscribe_Purchase
With a personal account, you can:
  • Access free activities and track your credits
  • Personalize content alerts
  • Customize your interests
  • Fully personalize your learning experience
Education Center Collection Sign In Modal Right

Name Your Search

Save Search
With a personal account, you can:
  • Track your credits
  • Personalize content alerts
  • Customize your interests
  • Fully personalize your learning experience
jn-learning_Modal_SaveSearch_NoAccess_Purchase

Lookup An Activity

or

My Saved Searches

You currently have no searches saved.

With a personal account, you can:
  • Access free activities and track your credits
  • Personalize content alerts
  • Customize your interests
  • Fully personalize your learning experience
Education Center Collection Sign In Modal Right
Topics
State Requirements