Psychiatry Research 208 (2013) 1–8 Contents lists available at SciVerse ScienceDirect Psychiatry Research journal homepage: www.elsevier.com/locate/psychres Symptom severity scale of the DSM5 for schizophrenia, and other psychotic disorders: diagnostic validity and clinical feasibility Michael S. Ritsner n, Maria Mar, Marina Arbitman, Alexander Grinshpoon Department of Psychiatry, Rappaport Faculty of Medicine, Technion—Israel Institute of Technology, Haifa and Sha’ar Menashe Mental Health Center, Israel a r t i c l e i n f o abstract Article history: Received 11 September 2012 Received in revised form 24 January 2013 Accepted 22 February 2013 Innovations in DSM5 include dimensional diagnosis of schizophrenia (SZ) and other psychotic (OP) disorders using the symptom severity scale (SS-DSM5). We evaluated the psychometric properties and diagnostic validity of the SS-DSM5 scale using a cross-sectional design and an unselected convenience unselected sample of 314 inpatients and outpatients with SZ/OP and mood disorders who received standard care in routine clinical practice. The SS-DSM5 scale, the Clinical Global Impression-Severity scale (CGI-S), the Positive and Negative Syndrome Scale (PANSS), and the Bech-Rafaelsen Mania Scale (BRMS) were administered. Factor structure, reliability, internal consistency, convergent and diagnostic ability of the DSM5-SS were evaluated. Factor analysis indicated two latent factors underlying the SSDSM5 (Psychotic and Deficit sub-scales). Cronbach’s alpha was 4 0.70. Convergent validity of the SSDSM5 was highly significant. Patients with SZ/PO disorders were correctly diagnosed (77.9%) using the SS-DSM5 scale (72% using PANSS). The agreement of the diagnostic decisions between the SS-DSM5 and PANSS was substantial for SZ/PO disorders (Kappa ¼ 0.75). Classifying participants with SZ/PO versus mood disorders using SS-DSM5 provided a sensitivity of 95%, and specificity of 34%. Thus, this study suggests that the SS-DSM5 has acceptable psychometric properties and that its use in clinical practice and research is feasible in clinical settings. The dimensional option for the diagnosis of schizophrenia and related disorders using SS-DSM5 is discussed. & 2013 Elsevier Ireland Ltd. All rights reserved. Keywords: DSM5 Schizophrenia Other psychoses Dimensional diagnosis 1. Introduction Numerous studies in the last century have increasingly emphasized that the boundaries between nosological entities may not be categorical, and putative comorbidity of various disorders may reflect impairments in common clinical dimensions, genetic variation, human behavior and neurobiological functions (e.g., Peralta and Cuesta, 2007; Owen et al., 2007; Ritsner and Gottesman, 2011). The categorical approach defines subgroups within the disorder, whereas the dimensional approach emphasizes the severity of different symptom clusters. As such, it is important to explicitly include dimensional assessments of the core symptoms of psychotic disorders in order to identify pertinent variability. Therefore, the most useful current approach for the classification of schizophrenia (SZ), other psychotic (OP), and mood disorders may be the complementary use of categorical and dimensional representations of functional psychoses (Salokangas, 2003; Brown and Barlow, 2005; Dikeos n Correspondence to: Acute Department Sha’ar Menashe Mental Health Center, Mobile Post Hefer 38814, Hadera, Israel. Tel.: þ972 4 6278750; fax: þ 972 4 6278045. E-mail addresses: [email protected], [email protected] (M.S. Ritsner). 0165-1781/$ - see front matter & 2013 Elsevier Ireland Ltd. All rights reserved. http://dx.doi.org/10.1016/j.psychres.2013.02.029 et al., 2006; Dutta et al., 2007; Helzer et al., 2008; van Os, 2009; Kamphuis and Noordhof, 2009). Recently, the American Psychiatric Association (APA) posted a draft of the Diagnostic and Statistical Manual of Mental Disorders, 5th ed. (DSM5) on a special web site, www.dsm5.org. One of innovations in the DSM5 is the extensive use of so-called dimensional assessments to account for severity of symptoms of schizophrenia and other psychotic disorders. In particular, the DSM5 Workgroup suggests that each of the diagnostic symptoms for these disorders (‘Criterion A’) may be rated using a new symptom severity scale that we called the Symptom Severity Scale DSM5 (SS-DSM5) in the current study (http://www.dsm5. org/Pages/Default.aspx)1. The psychometric properties of the SS-DSM5 in terms of reliability, and validity have not been presented. The purpose of our study therefore was to establish the psychometric properties (factor structure, reliability, internal consistency, convergent and diagnostic ability) of the SS-DSM5 scale in an unselected sample of patients with psychotic (SZ/OP) and mood disorders that were 1 Because the draft diagnostic criteria posted on www.dsm5.org are undergoing revisions, the specific criteria text has been removed from the website to avoid confusion or use of outdated categories and definitions. 2 M.S. Ritsner et al. / Psychiatry Research 208 (2013) 1–8 receiving standard care in inpatient and out-patient settings of a large university hospital. 2. Methods In this study, we used a cross-sectional design enrolling an unselected convenience sample of inpatients and outpatients with SZ/OP and mood disorders who received standard care from February 2010 to March 2011 in the routine clinical practice settings at Shaar Menashe Mental Health Center affiliated to the Rappaport Faculty of Medicine, Technion, Israel. The study included men and women ages 18–80 years. The design adhered to the Declaration of Helsinki and ICH/Good Clinical Practice guidelines. The Internal Review Board of Sha’ar Menashe Mental Health Center approved the study. All participants provided written informed consent for participation in the study, after receiving an explanation of study procedures. 2.1. Assessment Diagnosis was based on the Diagnostic and Statistical Manual of Mental Disorders, 4th ed. (DSM-IV; codes: 295, 296, 297, and 298), on a face-to-face interview, medical records, and consensus between two senior psychiatrists. Participants were assessed with the SS-DSM5 scale together with well-recognized and established psychometric instruments: the Clinical Global Impression-Severity scale (CGI-S; Guy, 1976), the Positive and Negative Syndrome Scale (PANSS; Kay et al., 1987), and the Bech-Rafaelsen Mania Scale (BRMAS; Bech et al., 1978). The SS-DSM5 scale includes nine items or domains (hallucinations, delusions, disorganization, abnormal psychomotor behavior, restricted emotional expression, avolition, impaired cognition, depression, and mania) that are rated for their current severity (most severe in the past month) on a five-point scale ranging from: 0¼ not present; 1¼ equivocal (severity or duration not sufficient to be considered psychosis); 2 ¼ present, but mild (little pressure to act upon voices, not very bothered by voices); 3 ¼present and moderate (some pressure to respond to voices, or is somewhat bothered by voices); to 4¼ present and severe (severe pressure to respond to voices, or is very bothered by voices). A score of 2 or higher is considered sufficient severity to fulfill a diagnostic indicator for disorder. Psychiatrists were requested to evaluate and provide their overall opinion about the suitability and feasibility of the SS-DSM5 scale. The following PANSS items were used for comparative analysis as diagnostic indicators for schizophrenia and other psychoses (DSM-V Criterion A): delusions (P1), hallucinations (P3), disorganized speech (P2), grossly abnormal psychomotor behavior, such as catatonia (P4), negative symptoms (N1, N4, G13, G16). A PANSS raw score of 3 and more for each PANSS item was used as a cut-off for a clinically relevant symptom (Ritsner, 2011). The BRMAS includes 11 items (elevated mood, pressure of speech, increased social contact, increased motor activity, sleep disturbances, social activities and distractability, hostility and irritability, increased sexual activity, increased selfesteem, flight of thoughts, and noise level of speech and other vocal activity). Each item is scored on a scale of 0 (not present or no difficulties) to 4 (Bech et al., 1978). Studies of the internal validity of the BRMAS have demonstrated that the simple sum of the 11 items of the scale is a sufficient statistic for the assessment of the severity of manic states. The inter-observer reliability was found to be high in a number of studies conducted in various countries. The BRMAS has shown acceptable external validity, in terms of both sensitivity and responsiveness (Bech, 2002). Prior to initiation of the study all raters were trained to produce acceptable levels of reliability on rating scales. The Intraclass Correlation Coefficient (ICC) for CGI-S was 0.90, for PANSS (0.89), and BRMAS total scores were 0.89 and 0.88, respectively. All rating scales administered by the same raters (RMS, MM and MA). 2.2. Subjects A total of 329 patients were screened and 314 individuals (226 inpatients and 88 outpatients) were enrolled in the study; nine of 15 subjects were excluded due to serious medical illness; and six patients refused to participate. There were 192 male subjects (61.13%), with a mean age of 41.1 7 12.8 (range¼ 19–76); education was 10.27 2.1 years, age of onset was 22.97 6.9 years, the mean duration of illness was 15.9 711.7 years. All individuals met DSM-IV criteria; in particular, 148 patients met criteria for paranoid type of schizophrenia (295.3), 56 patients met to criteria of schizoaffective disorder (295.7), 68 patients of other psychotic disorders (disorganized type [295.1; n¼ 21], catatonic type [295.2; n ¼1], residual type [295.6; n¼ 15], undifferentiated type [295.9; n¼13] types of schizophrenia; 4 delusional disorder and 10 brief psychotic disorder [codes 297/298]; schizophreniform disorder [295.4; n¼ 4]), and 42 patients of mood disorder (296; Major Depressive Disorder [MDD: single episode, 296.2, n¼ 7; recurrent episode, 296.3, n¼ 16]; [Bipolar I Disorder, BPD: single manic episode, 296.0, n¼ 13; recent episode manic, 296.4, n¼ 5; most recent episode mixed, 296.6, n ¼1]). An exacerbation of symptoms was observed among 99 patients, while stabilization of the current mental condition was found among 215 participants. Illness course was unspecified (14/314¼ 4.5%, first episode), continuous (117/314 ¼37.3%), or episodic (183 ¼58.3%). Patients in the present sample (n¼ 314) had moderate levels of illness and symptom severity (mean 7 S.D.), CGI-S was 4.8 71.0 score (range¼2–7); PANSS total scores were 89.27 24.4 (range: 38–179), PANSS negative symptom scores were 23.47 8.7 (range: 7–47), positive symptoms 22.07 8.7 (range: 7–43), and general psychopathology 43.8 710.6 (range: 22–90), BRMAS scores were 13.07 9.3 (range: 0–44). Patients were treated with various antipsychotic medications as clinically indicated. Ninety-seven patients were treated with first generation antipsychotic agents (FGAs; chlorpromazine, haloperidol, haloperidol decanoate, perphenazine, zuclopenthixol, zuclopenthixol decanoate, fluphenazine decanoate), 103 – with second generation antipsychotics (SGAs; clozapine, risperidone, olanzapine, quetiapine, ziprasidone, amisulpride), and 90 – with a combination of FGAs and SGAs (COMB). Chlorpromazine equivalent (CPZ) doses were calculated based on published data (Foster 1989; Woods 2003). The mean CPZ (7S.D.) in the FGA group was 670753 mg/day; in the SGA group, 390794 mg/day, and in the COMB therapy group 9707107 mg/day. In addition to antipsychotic medications, the patients took mood stabilizers (valproate, carbamazepine, lamotrigine; n¼ 55), benzodiazepines (n¼ 38), anti-Parkinson agents (n¼ 74), and antidepressants (n¼38). 2.3. Statistical analysis Factor structure, reliability, internal consistency, convergent validity, diagnostic ability and accuracy of the DSM5-SS were evaluated. The principal axis method of factor analysis with varimax rotated factor matrix was applied to identify the factors underlying the SS-DSM5 dimensions. This method can be used when the assumption of normality has been violated (Fabrigar and Petty, 1999). The eigenvalue criterion 41.5 was used to determine the number of factors to retain. Variables with an absolute loading greater than the amount set in the minimum loading option (40.4) were selected. Reliability testing included the identification of redundant items (item–item correlation 40.8), and the testing of internal consistency with Cronbach’s alpha (a) coefficient for the entire SS-DSM5 scale and for each subscale. The kappa (k) reliability test was used to study the agreement between SS-DSM5 scale and PANSS diagnostic decisions. Rules-of-thumb for kappa were: values less than 0.20 indicate low agreement, values between 0.21 and 0.40 indicate fair agreement, between 0.41 and 0.60 means moderate agreement, between 0.61 and 0.80 indicate substantial agreement, and 40.81 – almost perfect agreement between the two scales (Viera and Garrett, 2005). In addition, the concordance in decisions between the two diagnostic scales (SS-DSM5 and PANSS), and the P value were calculated with the McNemar test of symmetry with continuity correction (it tests for symmetry around the diagonal of the table). That is, reject the hypothesis of symmetry of the diagonal if the reported probability level is less than 0.05. To investigate convergent validity of the SS-DSM5 scale, we calculated Pearson’s correlation coefficients of the SS-DSM5 scale scores with scores on the PANSS, BRMAS, and CGI-S. The intraclass correlation coefficient (ICC) was used to measure inter-rater reliability for the three raters; it was rated as fair (0.30 to 0.49), moderate (0.50–0.69), or high (0.70–1.00) for the purposes of comparison (Landis and Koch, 1977). In order to test the diagnostic validity and accuracy of the SS-DSM5 compared to the PANSS scale, we used DSM5 ‘Criterion A’ for schizophrenia. The ‘Criterion A’ requires: (i) two or more of the characteristic symptoms: delusions, hallucinations, disorganized speech, abnormal psychomotor behavior, negative symptoms, i.e., restricted affect or avolition/asociality; and (ii) at least one of the following: delusions, hallucinations, disorganized speech. Diagnostic test evaluation was also performed (http://www.vassarstats.net/clin1.html). Quantities typically used to evaluate the diagnostic accuracy of binary variables are sensitivity, specificity, positive predictive values (PPV), and negative predictive values (NPV), positive likelihood ratio (þLR) and negative likelihood ratio ( LR), whereas an assessment of the overall prognostic accuracy of ordinal or metric variables is typically assessed by a geometric approach [the area under the corresponding receiver operating characteristic (ROC) curve] (Zhou et al., 2002; Sheskin, 2004). In the absence of general criteria of either minimum acceptable values of diagnostic accuracy measures or the choice of quantity in symptom selection, emphasis was put on both sensitivity and PPV, and the minimum acceptable value of both quantities was defined according to studies of diagnostic/prognostic accuracy of symptoms in ¨ schizophrenia (Klosterkotter et al., 2001). Thus, a slightly lower value of sensitivity than required for symptoms (Z0.25) and a value of PPV (Z0.70) were chosen as selection criteria. Furthermore, statistic procedure of the receiver operator characteristic (ROC) analysis generates both binormal and empirical (non-parametric) ROC curves. The ROC curve shows the characteristics of a diagnostic test by graphing the false-positive rate (specificity) on the horizontal axis and the truepositive rate (sensitivity) on the vertical axis for various cut-off values. The area under an ROC curve (AUC) is considered as an effective measure of inherent validity of a diagnostic test. Other things being equal, the larger the AUC, the better the test predicts the existence of the disorder. The possible values of AUC range from 0.5 (no diagnostic ability) to 1.0 (perfect diagnostic ability). Finally, the feasibility and acceptability of SS-DSM5 scale was examined by eight clinical psychiatrists and compared to PANSS and BRMAS scores. M.S. Ritsner et al. / Psychiatry Research 208 (2013) 1–8 Continuous variables were compared using the two-tailed t-test, or the Wilcoxon signed-rank test (z) for assessing the difference in medians, as well as using the general model of an analysis of variance (ANOVA). Post hoc analysis was carried out in cases of significant outcomes, using the Tukey–Kramer method. For all analyses, the level of statistical significance was defined as an alpha less than 0.05. All analyses were performed using the Number Cruncher Statistical System (NCSS Statistical Software, Kaysville, UT) (Hintze, 2006). 3 item pairs consistently suggested redundancy across diagnostic groups. Specifically, positive correlations between six SS-DSM5 symptoms (hallucinations, delusions, disorganization, psychomotor behavior, emotional expression and avolition) ranged from 0.25 (Po0.05) to 0.59 (Po0.001), excepting correlation between psychomotor behavior, and emotional expression (P40.05). Similarly, disorganization, emotional expression and avolition positively correlated with impaired cognition scores (r ranged from 0.27–0.50), while disorganization and psychomotor behavior scores correlated with mania scores (r ¼0.38 and 0.46, respectively, p o0.001). Negative correlations were observed for depression symptom scores with delusions (r ¼ 0.30, Po0.01), disorganization (r ¼ 0.32, P o0.01), and mania scores (r ¼ 0.27, po0.05). Finally, correlation coefficients did not reach significant levels between: (a) impaired cognition scores and three symptoms; (b) mania scores and five symptoms, and (c) depression scores and six symptoms (Table 2). 3. Results 3.1. Factor structure Results from an exploratory factor analysis of SS-DSM5 dimensions are summarized in Table 1. We found a model with two latent factors (scales), which were labeled as ‘Psychotic’ and ‘Deficit’ scales. The first factor included negative loadings of delusions, disorganization, abnormal psychomotor behavior and mania scores. The second factor was constructed using restricted emotional expression, avolition and impaired cognition scores with negative loadings. Correspondingly, they accounted for 50.8%, and 48.4% of the total variance of the nine SS-DSM5 items. Two SS-DSM5 items (‘hallucinations’ and ‘depression’) did not reach the minimum loading option (40.4). 3.3. The internal consistency Cronbach’s alpha coefficients for the SS-DSM5 scale total score ranged from 0.38 to 0.79. In particular, Cronbach’s a was 0.67 for the whole sample. Among individuals with paranoid type of schizophrenia, and with other psychotic disorders, alpha coefficients were 0.70, and 0.76, respectively. Among participants with schizoaffective and mood disorders, alpha coefficients for the total score were similarly low (0.58 and 0.38, respectively). Cronbach’s a coefficient was 0.75 and 0.71 for ‘psychotic’, and ‘deficit’ scales of the SS-DSM5, 3.2. Reliability Item–item Pearson’s correlation coefficients of 40.8 were used to identify redundant items. No highly correlated SS-DSM5 Table 1 Factor loadings and communalities after varimax rotation of variable values SS-DSM5 items among 314 patients with schizophrenia spectrum and mood disorders. SS-DSM5 items Intraclass correlation coefficient (ICC) Factor 1 (eigenvalue ¼ 2.19) ‘psychotic syndrome’ Hallucinations Delusions Disorganization Abnormal Psychomotor Behavior Restricted Emotional Expression Avolition Impaired Cognition Depression Mania Factors’ contribution (%) 0.92 0.91 0.87 0.81 0.81 0.79 0.80 0.95 0.97 0.87 Factor 2 (eigenvalue ¼ 2.09) ‘deficit syndrome’ Factor loading Communalities Factor loading Communalities 0.3282 0.5560 0.7518 0.7168 0.0864 0.3067 0.0109 0.3425 0.6905 50.8 0.1077 0.3092 0.5652 0.5138 0.0074 0.0941 0.0001 0.1173 0.4767 – 0.3734 0.3468 0.4788 0.1608 0.8744 0.6422 0.5268 0.0614 0.3383 48.4 0.1394 0.1202 0.2293 0.0258 0.7646 0.4124 0.2776 0.0037 0.1145 – Table 2 Pearson correlations coefficientsc between SS-DSM5, CGI-S, BRMS, and PANSS scores (n¼ 314). CGI-S Hallucinations Delusions Disorganization Psychomotor Behaviora Emotional Expressionb Avolition Impaired Cognition Depression Mania PANSS total score Negative scale Positive scale General psychopathology Bech-Rafaelsen Mania Scale a b c 1 2 3 4 5 6 7 8 9 10 11 12 13 14 0.39 0.25 0.42 0.38 0.29 0.17 0.28 0.07 0.15 0.48 0.39 0.43 0.43 0.31 0.44 0.40 0.25 0.28 0.33 0.09 0.08 0.01 0.58 0.42 0.61 0.49 0.13 0.49 0.38 0.31 0.38 0.02 0.30 0.23 0.66 0.40 0.80 0.53 0.40 0.59 0.43 0.44 0.33 0.32 0.38 0.71 0.55 0.76 0.55 0.47 0.08 0.31 0.08 0.03 0.46 0.54 0.27 0.60 0.52 0.61 0.56 0.50 0.16 0.23 0.59 0.79 0.35 0.43 0.13 0.27 0.14 0.05 0.67 0.69 0.46 0.60 0.02 0.10 0.16 0.38 0.51 0.10 0.36 0.13 0.27 0.13 0.10 0.39 0.10 0.23 0.16 0.13 0.43 0.13 0.87 0.83 0.84 0.91 0.32 0.51 0.66 0.00 0.68 0.57 0.27 Abnormal Psychomotor Behavior. Restricted Emotional Expression. Significant levels of correlation coefficients: r¼ 0.11–0.29 (Po 0.05); r ¼ 0.30–0.37 (P o 0.01); r 40.37 (P o 0.001). 4 M.S. Ritsner et al. / Psychiatry Research 208 (2013) 1–8 participants with paranoid schizophrenia, schizoaffective and mood disorders. These symptoms were used to examine diagnostic ability of the SS-DSM5 and PANSS symptoms. respectively, with a small correlation between them (r¼0.29, Po0.05). 3.4. Convergent validity 3.6. Dimensional diagnosis: validity and accuracy Table 2 displays correlations of SS-DSM5 dimensions with other rating scales, which demonstrated low (r ¼0.31) to high positive correlations (r¼0.79). The SS-DSM5 scale total score highly correlated with PANSS dimensions (ranged between r ¼0.73 and 0.90, Po0.001), and moderately with BRMAS scores (r¼ 0.43, Po0.001). The SS-DSM5 psychotic scale highly correlated with the PANSS positive scale (r¼ 0.86, Po0.001), general psychopathology (r ¼0.58, Po0.001), and BRMAS scores (r ¼0.87, Po0.001), and moderately with the PANSS negative scale (r¼ 0.37, Po0.001). The SS-DSM5 deficit scale highly positively correlated with the PANSS negative scale (r¼ 0.85, Po0.001) and general psychopathology (r ¼0.60, P o0.001), and moderately with the positive scale (r ¼0.40, Po0.001). Correlation coefficient between the SS-DSM5 deficit scale and BRMAS scores did not reach significant levels (r ¼ 0.10, P¼0.070). The mean SS-DSM5 total score for our sample was 15.1 76.2 (range: 1–31). As anticipated, the patients with a more severe disorder had the most severe symptoms as measured with SS-DSM5 scale: a total score of 8.573.5 approximately corresponded to being considered ‘‘mildly ill’’ according to the CGI-S score, 12.674.1 as ‘‘moderately ill’’, 16.075.5 as ‘‘markedly ill’’, 18.476.4 as ‘‘severely ill’’, and 22.9 77.8 scores as the ‘‘most extremely ill’’. The concurrent validity of the SS-DSM5 symptoms was constructed by comparison with the PANSS scores and the DSM-IV diagnosis made by the clinicians. As can be seen in Table 4, using the SS-DSM5 symptoms, 77.9% of 272 patients with psychotic disorders were correctly diagnosed, in particular, 79.8% of the patients with paranoid schizophrenia, 71.4% with schizoaffective disorder, 79.4% with other psychotic disorders (using PANSS symptoms: 82.0%, 81.7%, 85.7%, 79.4%, respectively). Fig. 2 depicted the falsenegative diagnostic decisions: the SS-DSM5 scale revealed falsepositives for 26.2% of the patients with mood disorders (16.7% for PANSS), and 22.1% and 18.0% for psychotic disorders, respectively. The greatest discrepancy was observed among patients with schizoaffective disorder (28.6% and 14.3%, respectively). The concordance of the decisions between SS-DSM5 and PANSS scales, measured with the reliability coefficient Kappa (k 7SE), was 0.7570.06 for all psychotic disorders (t¼16.7, Po0.001), 0.7870.08 for paranoid type of schizophrenia (t¼7.4, P¼0.007), 0.5770.12 for schizoaffective disorder (t¼5.4, P¼ 0.020), and 0.8470.12 for other psychotic disorders (t¼ 4.9, P¼0.038). The sensitivity, specificity, negative and positive predictive values and negative and positive likelihood ratios are presented in the Table 5. As can be seen, classifying participants with psychotic and mood disorders using SS-DSM5 provided a sensitivity of 95%, specificity of 34%, PPV 77.9%, NPV 73.8%, clinically important þLR 3.53, and small difference in –LR 0.35. The receiver operator characteristic (ROC) curves used to find the best cut-off points for classification. Fig. 3 depicts two receiver ROC curves for SS-DSM5 and PANSS scales. The area under the ROC curve was 0.91 70.016 (SE) with z-value (AUC 40.5)¼24.7 (po0.001) for PANSS, and 3.5. Raw scores and frequency of symptoms An inspection of Table 3 reveals consistent differences in raw rating scale scores, especially, between psychotic and mood disorders, excepting BRMAS and CGI-S scores. Fig. 1 posted frequency of SS-DSM5 symptoms (a score of 2 or higher) among Table 3 Clinical characteristics of patient’s sample (n¼ 314). Rating scales CGI-S score c PANSS total score d Negative subscale Positive subscale General Psychopathology Mania Scale e SS-DSM5 total scoref Hallucinations Delusions Disorganization Behaviorg Expressionh Avolition Cognition Depression Mania a Paranoid schizophrenia (n¼ 148) Schizoaffective disorder (n ¼56) Other psychotic Mood disorders (n¼ 68) disorders (n¼ 42) ANOVAa (d.f. ¼3,314) Mean S.D. Mean S.D. Mean S.D. Mean S.D. F Po Mean S.D. t/z P 4.7 93.7 24.9 23.8 45.0 11.4 15.9 1.5 3.0 2.0 2.0 2.5 2.6 1.0 0.5 0.5 0.9 21.9 7.7 7.5 10.0 7.4 5.5 1.5 1.0 1.1 1.2 1.0 1.0 1.0 0.8 0.9 4.7 92.0 22.5 24.0 45.4 19.2 16.1 1.0 2.9 2.1 2.5 1.6 2.3 0.7 0.9 1.8 0.8 18.6 7.3 6.8 9.3 11.8 5.3 1.3 0.9 1.3 1.1 1.0 1.2 1.0 1.2 1.5 5.0 94.1 26.2 23.1 44.6 11.9 16.5 1.4 2.5 2.5 2.2 2.4 2.5 1.3 0.5 0.7 1.1 28.1 9.8 9.2 12.9 9.1 6.8 1.4 1.3 1.4 1.4 1.4 1.3 1.1 0.9 1.0 4.8 60.4 13.9 10.5 35.8 11.7 8.6 0.1 0.7 0.6 2.0 0.4 0.9 0.4 2.5 0.7 0.8 10.3 4.1 5.2 6.0 8.6 4.0 0.3 1.1 0.9 1.1 0.7 1.0 0.6 1.6 1.3 2.0 26.9 27.5 35.4 9.4 11.3 20.8 21.0 46.4 20.9 2.2 40.5 21.8 7.3 37.1 16.3 0.12 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.084 0.001 0.001 0.001 0.001 0.001 4.8 93.6 24.8 23.7 45.0 13.2 16.1 1.4 2.9 2.2 2.2 2.3 2.5 1.0 0.6 0.8 1.0 22.8 8.3 7.8 10.6 9.4 5.8 1.5 1.1 1.2 1.5 1.2 1.1 1.1 0.9 1.2 0.3 9.2 8.3 10.6 5.5 1.0 8.0 5.6 11.7 7.5 1.0 9.8 8.2 3.1 10.8 0.4 0.77 o 0.001 o 0.001 o 0.001 o 0.001 0.34 o 0.001 o 0.001 o 0.001 o 0.001 0.32 o 0.001 o 0.001 0.002 o 0.001 0.65 ANOVA (d.f. ¼3.272) comparisons between four subgroups of patients. Significance (t-test) between psychotic and mood disorders. CGI-S: Clinical Global Impression-Severity scale. d PANSS: Positive and Negative Syndrome Scale. e BRMAS: Bech-Rafaelsen Mania Scale. f SS-DSM5: Symptom severity scale of DSM5. g Abnormal Psychomotor Behavior. h Restricted Emotional Expression. b c Significanceb (between psychotic and mood disorders) Psychotic disorders (n¼ 272) M.S. Ritsner et al. / Psychiatry Research 208 (2013) 1–8 5 70 60 Percent 50 40 30 20 ia n an M D C ep og re ni ss tio io n on iti ol Av ga is D H Paranoid schizophrenia (n = 148) Schizoaffective disorder (n = 56) Mood disorders (n = 42) E Ex mo pr tio es n si al on Ps y Be ch h a om v i o to or r n tio za ni us al or lu D ci el na tio io ns 0 ns 10 SS-DSM5 symptoms Fig. 1. Frequency of SS-DSM5 symptoms across diagnostic groups of patients. Table 4 Diagnostic psychotic and mood disorders using DSM-V ‘Criterion A’ and DSM5-SS and PANSS scales. Number of characteristic symptoms Paranoid schizophrenia ( n¼ 148) SS-DSM5 scale 0a 1a 2 3 4 5 PANSSb 0a 1a 2 3 4 5 Schizoaffective disorder (n ¼56) Other psychotic disorders (n¼ 68) Total psychotic disorders Mood disorders (n¼ 272) (n¼42) n % n % n % n % n % 7 23 33 31 21 33 4.7 15.5 22.3 20.9 14.2 22.3 1 15 9 16 9 6 1.8 26.8 16.1 28.6 16.1 10.7 5 9 9 7 13 25 7.4 13.2 13.2 10.3 19.1 36.8 13 47 51 54 43 64 4.8 17.3 18.8 19.8 15.8 23.5 19 12 10 1 0 0 45.2 28.6 23.8 2.4 0 0 6 21 40 40 15 26 4.1 14.2 27.0 27.0 10.1 17.6 1 7 15 15 12 6 1.8 12.5 26.8 26.8 21.4 10.7 5 9 12 8 21 13 7.4 13.2 17.6 11.8 30.9 19.1 12 37 67 63 48 45 4.4 13.6 24.6 23.2 17.7 16.5 18 17 5 2 0 0 42.9 40.5 11.9 4.8 0.0 0.0 a False negative decision refers to a test result that tells you a disease or condition is not present, when in reality, there is disease. PANSS characteristic symptoms: delusions (P1), conceptual disorganization (P2), hallucinatory behavior (P3), excitement (P4), negative symptoms, i.e., blunted affect (N1), emotional withdrawal (N2), disturbance of volition (G13), and active social avoidance (G16). b 35 SS-DSM5 PANSS 28.6 30 26.2 25 20.6 20.6 Percent 20.2 20 16.7 22.1 18.3 18 14.3 15 10 5 0 Mood disorders (n = 42) Paranoid SZ (n = 148) Schizoaffective (n = 56) Other psychotic (n = 68) Total psychotic (n = 272) Diagnostic groups (DSM-IV) Fig. 2. False-negative decisions are based on the SS-DSM5 and PANSS scales (Criterion A). 6 M.S. Ritsner et al. / Psychiatry Research 208 (2013) 1–8 Table 5 Diagnostic test evaluation using SS-DSM5 of the probability that it is psychotic (SZ/PO) disorders versus mood disorders. Parameters Estimated value 95% Confidence interval Lower limit Upper limit 0.6560 0.9110 0.2466 0.7590 0.9738 0.4483 For any particular test result, the probability that it will be psychotic or mood disorder Positive (psychotic disorders) 0.8662 Negative (mood disorders) 0.1337 0.822376 0.099115 0.9008 0.1776 For any particular positive test result, the probability that it is psychotic disorder True positive (Positive Predictive Value) c 0.7794 False Positive 0.2205 0.724479 0.173725 0.8262 0.2755 For any particular negative test result, the probability that it is mood disorder 0.7380 True negative (Negative Predictive Value)d False Negative 0.2619 0.576769 0.143863 0.8561 0.4232 Likelihood Ratios (weighted by prevalence)e Positive Likelihood Ratio (þ LR) Negative Likelihood Ratio (–LR) 2.80 0.21 4.45 0.59 Prevalence Sensitivitya Specificityb 0.7101 0.9506 0.3406 3.53 0.35 a The sensitivity, i.e. correctly identified ‘psychotic disorders’ (true positive rate) The specificity, i.e. correctly identified ‘mood disorders’ (true negative rate). Positive predictive value (PPV) is probability that the ‘psychotic disorders’ is present when the test is positive. d Negative predictive value (NPV) is probability that the ‘mood disorders’ is not present when the test is negative. e Positive likelihood ratio (þ LR)¼true positive rate/false positive rate¼sensitivity/ (1 specificity). Negative likelihood ratio (–LR) ¼false negative rate/true negative rate¼ (1 sensitivity)/specificity. b c 0.85þ0.024, z ¼14.3 (p o0.001) SS-DSM5, which means that from 91% to 85% of the patients were correctly classified. (iii) 3.7. Feasibility and acceptability The intraclass correlation coefficient (ICC) for individual items of the SS-DSM5 and PANSS ranged from 0.79 to 0.97, and those for the total score was 0.87; that is quite comparable with CGI-S (0.90), PANSS (0.89), and BRMAS (Table 1). The feasibility and acceptability of the SS-DSM5 scale was examined compared to the PANSS and the BRMAS. After completing these scales, eight clinical psychiatrists reported that the SS-DSM5 scale eased their assessment of the intensity of symptoms, it was easier to understand, less burdensome to administer, and quite acceptable to use in routine clinical practice. The amount of time needed to administer the SS-DSM5 during psychiatric examination was 1077.5 min, compared to the BRMAS (22713.8 min.), and the PANSS (3879.4 min). They considered the SS-DSM5 scale useful. However, the dimensional diagnostic procedure was somewhat burdensome for psychiatrists since this procedure included three steps: (1) assessment of the mental health state with SS-DSM5, (2) transformation of the raw scores to symptoms, and (3) checking the diagnostic ‘Criterion A’ of DSM5 with the obtained dimensional symptoms. 4. Discussion This study aimed to establish the psychometric properties and diagnostic validity of the SS-DSM5 scale for dimensional diagnosis of SZ/OP disorders. The key psychometric properties of the SS-DSM5 scale are: (i) The factorial analysis demonstrates a two-factorial structure of the SS-DSM5 dimensions, which were labeled ‘psychotic’, and ‘deficit’ scales, respectively. (ii) The reliability and internal consistency of the SS-DSM5 total score and its scales were shown to be strong. Cronbach’s alpha was 40.70 for the entire sample; 0.75 and 0.71 for the (iv) (v) (vi) (vii) ‘psychotic’, and ‘deficit’ scales, respectively, with a small correlation between them (r ¼0.29, Po0.05). The convergent validity was confirmed statistically: correlations of SS-DSM5 and its scales with CGI-S and the PANSS counterpart were all highly significant (r range from 0.73– 0.90, Po0.001), and moderate with BRMAS scores (r¼ 0.43, Po0.001). The intraclass correlation coefficient (ICC) for individual items of the SS-DSM5 PANSS ranged from 0.79– 0.97 (0.87 for the total score) comparable to other rating scales. The patients with more severe disorders had the most severe symptoms as measured with the SS-DSM5 scale: CGI-S ‘‘mildly ill’’ corresponded to SS-DSM5 total scores of 8.5 73.5, and the CGI-S ‘‘extremely ill’’ to SS-DSM5 scores of 22.9 77.8. The agreement of the diagnostic decisions between the SS-DSM5 and PANSS was substantial for all psychotic disorders (77.9% vs. 72% using PANSS; Kappa k¼0.75, Po0.001). Classifying participants with ‘psychotic’ or ‘mood’ disorders using SS-DSM5 provided a sensitivity of 95%, specificity of 34%, PPV 77.9%, and NPV 73.8%. Clinical psychiatrists found that the SS-DSM5 scale eased their assessment of the intensity of symptoms, was easier to understand, less burdensome to administer, and quite acceptable for use in routine clinical practice. Since, according to the authors’ best knowledge, this is the first article that reports the psychometric properties and feasibility of the SS-DSM5 scale, it is not possible to compare the current results with those of other studies. Results of this study suggest that the SS-DSM5 compares well to the PANSS and may be validly employed in the diagnostic assessment of schizophrenia and other psychotic disorders. A reliable and valid measure of symptom severity is needed to match the growing interest in dimensional diagnostic procedures in mental health care. Indeed, a standardized rating system for individual symptoms would contribute to knowledge of the severity of the polymorphic symptomatology and other presentations of mental health disorders. M.S. Ritsner et al. / Psychiatry Research 208 (2013) 1–8 1. ROC Curve of psychotic versus mood disorders 1.00 Sensitivity 0.75 0.50 0.25 0.00 0.00 Criteria PANSS SS-DSM5 0.25 0.50 Specificity 0.75 1.00 Sensitivity 2. ROC Curve of psychotic versus mood disorders 7 N1, N4, G13, and G16) as dimensional options for the diagnosis of schizophrenia and related disorders. Indeed, these PANSS items revealed only 18% false-positives for psychotic disorders and 16.7% for mood disorders (21.1% and 26.2% for SS-DSM5, respectively). There are several limitations in this study. First, test–retest reliability and sensitivity to change were not tested because of the cross-sectional design, covering symptoms during the last four weeks. Second, the diagnosis was made by the clinicians according to DSM-IV criteria, but not using standardized diagnostic tools. In conclusion, the SS-DSM5 scale appears to have solid psychometric features that are likely to make it useful in mental health services. Although clinical psychiatrists found the SS-DSM5 scale quite acceptable and very feasible, its utility for improving diagnostic practice remains open. Future research with more diverse clinical samples is underway and should further identify the strengths and weaknesses of the measure. 1.00 Authors’ contributions 0.75 RMS originated the study design. RMS and AG were responsible for collection of data, statistical analysis, and an interpretation of data and drafted the manuscript. MM and MA participated in acquisition of data and interpretation of data. All authors have given final approval of the version to be published. 0.50 Acknowledgments 0.25 The authors especially thank R. Kurs, B.A. for editing this manuscript. Criteria PANSS SS-DSM5 0.00 0.00 0.25 0.50 Specificity 0.75 References 1.00 Fig. 3. Receiver operator characteristic (ROC) analysis. The first plot shows the empirical ROC curve. The second plot shows an ROC curve based on the binormal assumption. However, there are two problems associated with the dimensional approach to diagnosis in an official nomenclature: (a) there is no widely tested and accepted system of dimensional diagnosis, and (b) clinicians find the added work of rating dimensions burdensome (Frances 2009). Furthermore, different categorical cut-off points may lead to different or even opposing clinical conclusions (Kraemer et al., 2004). It is important to keep in mind that although a dimensional approach has been offered for the diagnoses of various disorders (e.g., Peralta, Cuesta, 2007; Helzer et al., 2007; Allardyce et al., 2007), practical use and clinical relevance in psychiatry remain unclear. The SS-DSM5 dimensions do not appear to be diagnosis specific for categorical diagnoses in the schizophrenia spectrum. There could be some concerns about the proposed SS-DSM5 dimensions. For instance, they do not possess a standardized metric measure for: (1) what the rather crude scoring ranging from 0 to 4 represents; (2) issues that need to be resolved regarding dimension weighting in different psychotic disorders; and (3) the lack of dimensions for the assessment of severity of emotional distress, and quality of life impairment. The PANSS is a commonly used instrument to assess symptom severity in SZ/OP disorders; however, it has not yet been used in the dimensional diagnostic procedure of psychotic disorders. The present study raises the question regarding PANSS items (P1-P4, Allardyce, J., Suppes, T., Van Os, J., 2007. Dimensions and the psychosis phenotype. International Journal of Methods Psychiatric Research 16 (1), S34–S40. Bech, P., 2002. The Bech-Rafaelsen Mania Scale in clinical trials of therapies for bipolar disorder: a 20-year review of its use as an outcome measure. CNS Drugs 16, 47–63. Bech, P., Rafaelsen, O.J., Kramp, P., Bolwig, T.G., 1978. Mania Rating Scale: scale construction and inter-observer agreement. Neuropharmacology 17, 430–431. Brown, T.A., Barlow, D.H., 2005. Dimensional versus categorical classification of mental disorders in the fifth edition of the diagnostic and statistical manual of mental disorders and beyond: comment on the special section. Journal of Abnormal Psychology 114, 551–556. Diagnostic and Statistical Manual of Mental Disorders, 5th ed. Available from: (DSM-5) web sites, http://www.dsm5.org. and /http://www.dsm5.org/Pages/ Default.aspxS. Dikeos, D.G., Wickham, H., McDonald, C., Walshe, M., Sigmundsson, T., Bramon, E., Grech, A., Toulopoulou, T., Murray, R., Sham, P.C., 2006. Distribution of symptom dimensions across Kraepelinian divisions. British Journal of Psychiatry 189, 346–353. Dutta, R., Greene, T., Addington, J., McKenzie, K., Phillips, M., Murray, R.M., 2007. Biological, life course, and cross-cultural studies all point toward the value of dimensional and developmental ratings in the classification of psychosis. Schizophrenia Bulletin 33, 868–876. Fabrigar, L.R., Petty, R.E., 1999. The role of the affective and cognitive bases of attitudes insusceptibility to affectively and cognitively based persuasion. Personality and Social Psychology Bulletin 25, 91–109. Frances, A., 2009. Whither DSM-V? The British Journal of Psychiatry 195, 391–392. Foster, P., 1989. Neuroleptic equivalence. The Pharmaceutical Journal 243, 431–432. Guy, W., 1976. ECDU Assessment Manual for Psychopharmacology. US Department of Health and Human Services, Rockville, MD. Helzer, J.E., Bucholz, K.K., Gossop, M., 2007. A dimensional option for the diagnosis of substance dependence in DSM-V. International Journal of Methods in Psychiatric Research 16 (1), S24–S33. Helzer, J.E., Kraemer, H.C., Krueger, R.F., Wittchen, H.U., Sirovatka, P.J., Regier, D.A., 2008. Dimensional Approaches in Diagnostic Classification: Refining the Research Agenda for DSM-V. American Psychiatric Association, Arlington, VA. Hintze, J.L., 2006. NCSS-2007. Statistical System for Windows. User’s Guide. Number Cruncher Statistical Systems, Kaysville Utah. Kamphuis, J.H., Noordhof, A., 2009. On categorical diagnoses in DSM-V: cutting dimensions at useful points? Psychological Assessment 21, 294–301. 8 M.S. Ritsner et al. / Psychiatry Research 208 (2013) 1–8 Kay, S.R., Fiszbein, A., Opler, L.A., 1987. The Positive and Negative Syndrome Scale (PANSS) for schizophrenia. Schizophrenia Bulletin 13, 261–276. ¨ Klosterkotter, J., Hellmich, M., Steinmeyer, E.M., Schultze-Lutter, F., 2001. Diagnosing schizophrenia in the initial prodromal phase. Archives of General Psychiatry 58, 158–164. Kraemer, H.C., Noda, A., O’Hara, R., 2004. Categorical versus dimensional approaches to diagnosis: methodological challenges. Journal of Psychiatric Research 38, 17–25. Landis, J.R., Koch, G.G., 1977. The measurement of observer agreement for categorical data. Biometrics 33, 159–174. Owen, M.J., Craddock, N., Jablensky, A., 2007. The genetic deconstruction of psychosis. Schizophrenia Bulletin 33, 905–911. Peralta, V., Cuesta, M.J., 2007. A dimensional and categorical architecture for the classification of psychotic disorders. World Psychiatry 6, 36–37. Ritsner, M.S., 2011. Toward a Multidimensional Continuum Model of functional psychoses for research purposes. In: Ritsner, M.S. (Ed.), Handbook of Schizophrenia Spectrum Disorders, vol. I. Springer-Verlag, Dordrecht, pp. 85–113. Ritsner, M.S., Gottesman, I.I., 2011. The schizophrenia construct after 100 years of challenges. In: Ritsner, M.S. (Ed.), Handbook of Schizophrenia Spectrum Disorders, vol. I. Springer-Verlag, Dordrecht, pp.1–44. Salokangas, R.K.R., 2003. Symptom dimensions and outcome in schizophrenia. World Psychiatry 2, 172–178. Sheskin, D.J., 2004. Handbook of Parametric and Nonparametric Statistical Procedures, 3rd ed. Chapman & Hall, CRC, Boca Raton. Viera, A.J., Garrett, J.M., 2005. Understanding Interobserver agreement: the Kappa Statistic. Family Medicine 37, 360–363. Woods, S.W., 2003. Chlorpromazine equivalent doses for the newer atypical antipsychotics. Journal of Clinical Psychiatry 64, 663–667. Zhou, X.-H., Obuchowski, N.A., McClish, D.K., 2002. Statistical Methods in Diagnostic Medicine. Wiley & Sons Interscience, New York 437p. van Os, J., 2009. Salience syndrome’ replaces ‘schizophrenia’ in DSM-V and ICD-11: psychiatry’s evidence-based entry into the 21st century? Acta Psychiatrica Scandinavica 120, 363–372.
© Copyright 2024