A new tool to assess duration of untreated illness:
Psychopathological Onset and Latency to Treatment Questionnaire
(POLT-Q)

Un nuovo strumento per valutare la durata di malattia non trattata:
il questionario sull’esordio psicopatologico e la latenza al trattamento
(POLT-Q)

LUCIO OLDANI1, BEATRICE BENATTI1, VERA DE CARLO1, IVAN CORTINOVIS2,
ALFREDO CARLO ALTAMURA1, BERNARDO DELL’OSSO3,4,5
*E-mail: lucio.oldani@gmail.com

1Department of Psychiatry, University of Milan, Fondazione IRCCS Ca’ Granda, Ospedale Maggiore Policlinico, Milan, Italy
2Laboratory G. A. Maccacaro, Department of Clinical Sciences and Community Health, University of Milan, Italy
3Department of Biomedical and Clinical Sciences “Luigi Sacco”, Department of Psychiatry, University of Milan, Italy
4CRC “Aldo Ravelli” for Neuro-technology & Experimental Brain Therapeutics, University of Milan, Italy
5Department of Psychiatry and Behavioral Sciences, Bipolar Disorders Clinic, Stanford Medical School, Stanford University, CA, USA


SUMMARY. Introduction. Duration of untreated illness (DUI) has been increasingly investigated as a predictor of clinical outcome and course in different psychiatric disorders. To date, however, there are no tools for measuring this variable. Our group developed the Psychopathological Onset and Latency to Treatment Questionnaire (POLT-Q), focused on the onset of psychiatric disorders. Aim of this study was to assess the reproducibility and manageability of POLT-Q. Methods. Fifty consecutive in- and out-patients aged 16-65 with different DSM-5 psychiatric disorders were recruited. Two raters were present during the interview: one of them administered the POLT-Q to the patient and both independently completed the questionnaire. Collected values were compared using Cohen’s Kappa test and McNemar test. Results. 62.5% of the replies showed a 100% consistency between the two raters. In the 6.25% the agreement was <95%. For all the replies, the K coefficient was >0.8, a high degree of agreement. Discussion and conclusion. The POLT-Q assesses variables related to the psychopathological onset and first pharmacological treatment and, according to present findings, it represents a convenient, reliable and standardised measure for DUI. Further studies on larges sample are needed to confirm our preliminary results.

KEY WORDS: duration of untreated illness, age at onset, psychopatological onset, first pharmacological treatment, clinical questionnaire.


RIASSUNTO. Introduzione. La durata di malattia non trattata (duration of untreated illness - DUI) è stata progressivamente studiata come predittore dell’andamento clinico e dell’outcome di malattia in diversi disturbi psichiatrici. A oggi, non esistono strumenti formali per misurare questa variabile. Il nostro gruppo di studio ha sviluppato il Questionario sull’esordio psicopatologico e la latenza al trattamento (Psychopathological Onset And Latency To Treatment Questionnaire - POLT-Q), che si focalizza sull’esordio dei disturbi psichiatrici. Lo scopo di questo lavoro consiste nella valutazione della riproducibilità e della maneggevolezza del POLT-Q. Metodi. Per il presente studio sono stati reclutati consecutivamente 50 pazienti, in regime di ricovero o ambulatoriale, di età compresa tra i 16 e i 65 anni e con diversi disturbi psichiatrici in accordo al DSM-5. Due valutatori erano presenti all’intervista clinica: mentre uno di questi somministrava il POLT-Q, entrambi compilavano il questionario indipendentemente. Le variabili così raccolte sono state confrontate utilizzando l’indice Kappa di Cohen e il test di McNemar. Risultati. Nel 62,5% delle risposte fornite i due valutatori hanno raggiunto una concordanza pari al 100%. Nel 6,25% delle risposte fornite, la concordanza è stata inferiore al 95%. Per tutte le risposte, il coefficiente K è risultato superiore a 0,8, corrispondente a un elevato livello di concordanza. Discussione e conclusioni. Il POLT-Q valuta le variabili relative all’esordio psicopatologico e al primo trattamento farmacologico e, secondo i risultati presentati nel corrente studio, rappresenta uno strumento conveniente, affidabile e standardizzato per misurare la DUI. Ulteriori studi su un campione più ampio potranno confermare i nostri risultati preliminari.

PAROLE CHIAVE: durata di malattia non trattata, età d’esordio, esordio psicopatologico, primo trattamento farmacologico, questionario clinico.

INTRODUCTION
Duration of untreated illness (DUI)
Over the last two decades, the duration of untreated illness (DUI), defined as the interval between the onset of a specific psychiatric disorder and the administration of the first appropriate pharmacological treatment, according to guidelines in compliant subjects1, has been increasingly investigated as a predictor of clinical outcome and course across different conditions2. Most published studies on this topic, however, investigated the prognostic role of the DUI in schizophrenia and psychotic disorders, focusing on the duration of untreated psychosis (DUP)3-5. Although the association between the DUP and the clinical outcome of psychotic disorders has shown mixed results6,7, a longer DUP in schizophrenic patients has been associated with a worse long-term outcome, a higher risk of relapse, higher rates of suicide, more severe positive and negative symptoms, as well as reduced treatment response in the acute treatment of first episode8,9. In addition, a shorter DUP has been associated with a greater response to antipsychotic treatment, as measured by severity of global psychopathology, positive and negative symptoms, and functional outcomes10.
Likewise, DUI has been recognized as an important outcome parameter in depressive, anxiety and obsessive-compulsive and related disorders11,12.
Furthermore, in bipolar disorder (BD), a longer DUI has been associated with a worse outcome, an increased suicidality and a higher number of lifetime suicide attempts13. Similarly, the course of major depressive disorder (MDD) was found to be influenced by the DUI, in terms of response and remission, with related rates gradually decreasing when the DUI extends beyond specific temporal thresholds13-15. With respect to anxiety disorders, the DUI was found to be longer in generalized anxiety disorder compared to panic disorder16, and a longer DUI has been associated to a worse treatment response in OCD patients17-19. Similarly, DUI seems to be crucial for the treatment of personality disorders and eating disorders20,21.
Measuring the DUI and related variables: the state of the art
To date, there is no international agreement about the definition of DUI and no standardized, structured and reproducible tool for measuring it. As a matter of fact, if many questionnaires have been proposed to assess the DUP22, the identification of a reliable tool for quantifying the components of the psychopathological onset is still lacking.
Some clinical interviews and questionnaires have an introductive section, which can be used to collect some general data about the patient’s clinical history. For instance, the second module of the Structured Clinical Interview for DSM-IV Axis I Disorders (SCID-I)23, titled ‘Overview’, collects some elements concerning the medical history of the patient. One box, in particular (P11), deals with the time of onset of the patient’s symptoms, while other questions concern the possible presence of triggering factors at the onset (box P12) and the time passed until the initial symptoms have fully developed. Box P16 asks when patients have had their first contact with anyone due to their psychiatric symptoms and whether a pharmacologic treatment was prescribed or not. Therefore, the DUI might be somehow deduced from these elements, but not univocally determined, in the absence of specific and definitive questions.
In light of the above, our group developed the Psychopathological Onset and Latency to Treatment Questionnaire (POLT-Q), an ad-hoc questionnaire focused on the onset of psychiatric disorders and administration of the first pharmacological treatment. Authors, in order to assess patients with different clinical presentation, have already extensively and successfully used this questionnaire in several previous studies, aiming for a better characterization of symptoms at onset, while exploring the different components of psychopathological onset and nature of first pharmacological treatment11,12.
The principle aim of this study was to assess the reproducibility and manageability of the POLT-Q and to explore possible advantages of its regular use in clinical practice.
METHODS
Psychopathological Onset and Latency to Treatment Questionnaire (POLT-Q)
The POLT-Q’s aim is to provide a standardized form to gather data about the onset of psychopathology, and nature of first psychopharmacological treatment.
The design of the POLT-Q relies on the identification of specific concepts about the psychopathological onset of a psychiatric disorder. Onset in POLT-Q is described as the period between the first reported/observed change(s) in mental state/behaviour and the identification of a psychiatric diagnosis, achieved according to main diagnostic systems (International Statistical Classification of Diseases and Related Health Problems and Diagnostic and Statistical Manual of Mental Disorders)24,25. On the other hand, age at onset consists of the age when the patient began to experience his first psychopathological symptoms.
POLT-Q is composed by open and closed questions: some closed questions allow more than one answer in order to capture the complexity of onset (i.e. coexistence of many symptoms in the onset description or different reasons for having first seen a health provider).
Onset and nature of the first symptoms
Patients are asked the age (in years) when the first symptom(s) occurred, referring to the first change(s) in their mental state/behaviour26. The nature of symptoms is grouped into different clusters (mood, anxiety, psychotic, neurovegetative, eating, substance misuse and impulse-control), each of which has its own list of the most commonly reported symptoms, in order to better characterize that specific clinical dimension. The items are provided as illustrative examples and probes for interview but are not designed as an exhaustive list of all prodromal symptoms. The raters invite the patient to report the initial symptoms of their own psychiatric condition, thus ticking the corresponding areas; the raters then check the list by naming each area and asking the patient to state whether those symptoms were present as first changes in their psychic state or not. The positive areas are marked afterwards.
Age at first pharmacological treatment
Patients are then asked the age (in years) in which they took their first appropriate psychopharmacological treatment. This means the treatment had to be approved for the mental disorder they were suffering from and had to be compliantly taken at standard doses and for an adequate period of time, according to currently available International treatment guidelines27.
The difference (in years) between the age of first appropriate pharmacological treatment and the starting date of the first symptoms generates the Latency to First Pharmacological Treatment (which, for the purpose of this work, coincides with the DUI).
In order to help the patient outlining this lapse of time, crucial for the identification of the DUI, a visual summary has been proposed as an integrative part of the POLT-Q (available in the Appendix section).
In addition to these elements, some additional data about the psychopathological onset are collected by the POLT-Q. Although not essential for the identification of the latency to treatment, these factors contribute to describe the beginning and the development of patient’s mental disorder. Among these:
• presence of stressful event(s) at the first symptom(s), including a list of suitable categories (life events, traumas, different types of abuse, medical condition, and/or others);
• time transpired from the occurrence of the above mentioned stressful event and the experience of the first symptom(s);
• time transpired from the onset of the first symptom(s) and the request for help from a health provider (general practitioner, psychologist, neurologist, psychiatrist, and/or others), including a list of possible reason(s) for this delay;
• time transpired from the onset of the first symptom(s) and the request for help from a psychiatrist, in case the patient’s first contact was not with a psychiatrist.

Ultimately, the POLT-Q includes a preliminary part, a collection of patient’s demographic data, and two sections; one dealing with patient’s psychopathological onset (section 1) and the other with first psychopharmacological treatment (section 2). A version of the POLT-Q is included in the Appendix. Its full version, with extended section 1) and 2), is available upon request to the corresponding author of the present article.
Procedures
Two raters were trained on how to use the POLT-Q by the questionnaire’s developers, which consisted of the explanation of its components and rating rules. The two raters had not taken any part in the development of the questionnaire per se and did not know the clinical history of the patients enrolled in the study.
The interview took part in a quiet room. During each interview, both raters were present at the same time and one of them, chosen by the toss of a coin, administered the POLT-Q to the patient, in order to minimize potential bias. If patient’s replies seemed uncertain, the selected rater repeated the question again. Both raters independently completed the interview schedule for each subject and could not compare the collected answers.
Fifty consecutive patients aged 16-65, collected either from the out- or the in-patient units of our hospital, were enrolled in this study. Due to its non-interventional but rather purely observational nature, this study had few exclusion criteria, represented by conditions which might pose a risk of unreliability in the reconstruction of patient’s clinical history, including: the presence of major cognitive impairments, a history of head trauma, the existence of a CNS disease and the presence of acute although transient psychotic features due to alcohol or substance ingestion.
It is worthy to note that the nature of the mental disorder the patient was diagnosed with was not considered in this study, in order to avoid any chance of bias and according to the Authors’ aim to test the POLT-Q reliability in a wide range of psychiatric conditions.
Statistical analysis
The values of the replies collected by the two raters were compared by means of the Cohen’s Kappa test and the McNemar test. The first calculates the inter-rater agreement for the replies registered by the two evaluators, while the latter provides a measurment of the association between the replies, in the hypothesis of an even distribution of the inconsistent replies.
The statistical analyses were performed by means of the statistical software SAS vers. 9.2 (SAS Institute, Inc. Cary, N.C. USA 2009).
RESULTS
Table 1 shows the main socio-demographic characteristics of the sample.
The sample of patients was characterized by the following clinical features (median [min-max]): age 45 [20-74] years, age at first symptoms 25.5 [5-64] years, age at full expression of psychopathological picture 26.75 [5-64.25] years, age at help seeking 30.5 [10-64] years, months transpired between the consultation of a non-psychiatrist care-provider and a specialist in psychiatry 12 [0-360], age at first pharmacological treatment (other than benzodiazepines) 33 [18-73] years.



As shown in Table 2, in 20 out of 32 replies (62.5%), the consistency between the replies registered by the two raters was equivalent to 100%. In 2 out of 32 replies (6.25%) the agreement was below 95%, i.e. “family history of psychiatric disorder” and “nature of first symptoms: anxiety”.
In 2 cases, for the question exploring the nature of first symptoms (in the “Onset” section), the replies provided by the patients were not differentiated enough to build a table comparing the inter-rater agreement consistency. More in detail, for one option of reply (unusual experiences, e.g.: suspiciousness, hallucinations, misperceptions, feelings/conviction of environment hostility, social withdrawal, etc.), both raters registered “no” from all the subjects. For the second one (impulsivity: discontrol episodes, aggressiveness), the first rater registered 48 “no” and 2 “yes”, while the second rater registered 50 “no”. Such condition did not allow calculating neither the K coefficient nor the McNemar test.
However, for all the replies, the K coefficient was higher than 0.8, being the latter unanimously recognized as a very good degree of agreement28.



DISCUSSION AND CONCLUSIONS
More than a half of the replies given by the proposed patients were registered with no discrepancy between the two operators. A minor percent of replies (6.25%) was subjected to an inter-rater agreement below 95%.
Regarding the latter, the two questions involved consist of “family history of psychiatric disorder” and “nature of first symptoms: anxiety”. The family medical history is an important risk factor for several chronic conditions, yet challenges remain in efficiently identifying individuals at increased risk. Some Authors tried to pinpoint a questionnaire capable to detect such subjects, but further investigation is required to develop and formally validate a generic questionnaire, as an useful screening tool in primary care 29.
The capability of patients to consistently report about their relatives’ history of mental disorder has been previously discussed. Often, patients might not know whether their relatives ever experienced a psychiatric disorder in their life, as such information might be difficult to be shared within certain families, often due to the discomfort and stigma still connected to psychiatric illnesses. In addition, some patients, when interviewed, might remain reticent and contradictory towards such data, in the attempt to protect their relatives’ privacy 30,31.
Regarding the anxious nature of the first symptoms experienced by the subjects interviewed, the discordance in the registered replies might be explained by the multiform presentation of anxiety and the overlapping symptoms of depressive spectrum. As a matter of fact, some well-established questionnaires, routinely administered to assess patients with a depressive symptomatology (e.g., Hamilton-Depression Rating Scale, Montgomery-Asberg Depression Rating Scale, Beck Depression Inventory, etc.)32-34, also investigate symptoms pertaining to the anxiety spectrum (e.g., changes in sleep pattern, asthenia or fatigue, loss of energy, lack of concentration, changes in appetite, etc.)35,36. In addition, comorbidity of depressive, anxiety and somatoform disorders has often been observed in patients assessed in primary care clinics and in the general population: in such cases of comorbidities, anxiety disorders might not be detected by the patient or the clinician as a single entity, especially in the context of the psychopathological onset35,37-40.
Finally, physical symptoms of anxiety could often be misinterpreted by patients, who usually impute them to organic diseases (i.e. cardiologic, neurological, etc.), because of the predominantly somatic clinical picture: increased heart rate, increased perspiration, muscle weakness, shaking and trembling, paresthesia, tachypnea, etc.41.
It has not been possible to calculate either the K coefficient or the McNemar in two circumstances. All the answers were negative for both raters concerning the option of reply “unusual experiences”. This might be explained considering that such symptoms are normally disruptive for patients and, therefore, they tend to distinctly remember whether such features were present or not at the beginning of their psychopathological picture. In other words, a potential recall bias is less likely to occurr for such a disturbing clinical onset. Another reason for this lack of differentiation in answers might be tracked back to the small sample size: when the questionnaire is administered on a large scale of patients, more varied replies may be provided. As a matter of fact, the prevalence of unusual experiences is lower that prevalence of other clinical features of onset (e.g. depression, anxiety) for patients at their first access to the psychiatric services (as the sample collected for the purpose of this work), and mirrors the prevalence of different psychiatric conditions. With regard to the option of reply “impulsivity”, all replies were negative for one rater, while 2 out of 50 were positive for the other rater. We believe this might be due to the operator-dependent nature of the POLT-Q, as further explained below.
As it is conceived, the POLT-Q does not aim to formulate a current diagnosis or to detect a previous one. We believe this is a major point as, by not being a diagnostic tool, the POLT-Q can be used in all the patients with a psychiatric diagnosis and in those who do not have one yet. In addition, it does not investigate the severity of the clinical picture experienced by the patient. Therefore, it can be employed regardless of the stage of illness the interviewee is experimenting.
As a matter of fact, the POLT-Q is intended to be a brief and practical tool to collect some crucial variables of the clinical history of the interviewee. Some further advantages might consist of:
• rapid administration: corresponding to 13 minutes on average;
• manageable: a specialist education is not required in order to administer such questionnaire, but a 1-hour training is sufficient to contemporarily train a variety of sanitary operators (e.g.: general physicians, resident doctors, nurses, psychologists, etc.);
• inexpensive: both the formation of the personnel and the hard copies of the questionnaire are advantageous in terms of costs;
• patient-centred: apt to assess different kinds of patients attending various types of medical settings, e.g. inpatient, outpatients, subjects following rehabilitation program.

Regarding the limitations of the POLT-Q, some of them may be considered operator-related, while some others are patient-related. Although operators are trained to administer the questionnaire in a way to minimize the eventual interviewee’s ambivalence (see the relative section), patients can still produce disputable replies, subjected to different interpretations by two distinct interviewers. As any other operator-dependent examination, the POLT-Q might, therefore, lack of objectivity and can depend on interviewer’s interpretation of data collection. Operator-dependency might explain the minor degree of discordance observed for the 37.25% of the replies registered (yet, for such replies, the inter-rater level of agreement was included between 95% and 99.9%). Recall bias is another limitation of the questionnaire; as any interviewing tool that relies on patient’s memory to summon the earlier stages of clinical history, the POLT-Q might suffer from interviewee’s impaired reliability, especially when an older age plays a crucial role 42. Finally, the strength of evidence might be limited by the small sample size; in consideration of the heterogeneity of the disorders assessed, a inter-rater reliability test over a larger number of patients might be suitable.
According to our experience, as the POLT-Q identifies two key time points in the emergence of the psychopathological onset, i.e. the starting date of the first symptoms and the age at first pharmacological treatment, such tool could be used in the daily clinical practice as a convenient, reliable and standardised measure for DUI. Further studies on larges sample of patients are needed in order to confirm such preliminary results.


Conflict of interests: the authors have no conflict of interests to declare.










REFERENCES
 1. Altamura AC, Camuri G, Dell’Osso B. Una revisione critica sul ruolo della durata di malattia non trattata nei disturbi psichiatrici. Riv Psichiatr 2010; 45: 197-208.
 2. Bukh JD, Bock C, Vinberg M, Kessing LV. The effect of prolonged duration of untreated depression on antidepressant treatment outcome. J Affect Disord 2013; 145: 42-8.
 3. Farooq S, Large M, Nielssen O, Waheed W. The relationship between the duration of untreated psychosis and outcome in low-and-middle income countries: a systematic review and meta analysis. Schizophr Res 2009; 109: 15-23.
 4. Lloyd-Evans B, Crosby M, Stockton S, et al. Initiatives to shorten duration of untreated psychosis: systematic review. Br J Psychiatry 2011; 198: 256-63.
 5. Large M, Nielssen O, Slade T, Harris A. Measurement and reporting of the duration of untreated psychosis. Early Interv Psychiatry 2008; 2: 201-11.
 6. Craig TJ, Bromet EJ, Fennig S, Tanenberg-Karant M, Lavelle J, Galambos N. Is there an association between duration of untreated psychosis and 24-month clinical outcome in a first-admission series? Am J Psychiatry 2000; 157: 60-6.
 7. Ho BC, Andreasen NC, Flaum M, Nopoulos P, Miller D. Untreated initial psychosis: its relation to quality of life and symptom remission in first-episode schizophrenia. Am J Psychiatry 2000; 157: 808-15.
 8. Altamura AC, Dell’Osso B, Vismara S, Mundo E. May duration of untreated illness influence the long-term course of major depressive disorder? 2008;23(2):92-6.
 9. Palazzo MC, Arici C, Dell’Osso B, et al. Access and latency to first antipsychotic treatment in Italian patients with schizophrenia and other schizophrenic spectrum disorders across different epochs. Hum Psychopharmacol 2016; 31: 113-20.
10. Perkins DO, Gu H, Boteva K, Lieberman JA. Relationship between duration of untreated psychosis and outcome in first-episode schizophrenia: a critical review and meta-analysis. Am J Psychiatry 2005; 162: 1785-804.
11. Dell’Osso B, Cremaschi L, Palazzo C, et al. Factors characterizing access and latency to first pharmacological treatment in Italian patients with schizophrenia, mood, and anxiety spectrum disorders. Int Clin Psychopharmacol 2015; 30: 29-35.
12. Dell’Osso B, Cremaschi L, Grancini B, et al. Italian patients with more recent onset of Major Depressive Disorder have a shorter duration of untreated illness. Int J Clin Pract 2017; 71: e12926.
13. Altamura AC, Dell’Osso B, Berlin HA, Buoli M, Bassetti R, Mundo E. Duration of untreated illness and suicide in bipolar disorder: a naturalistic study. Eur Arch Psychiatry Clin Neurosci 2010; 260: 385-91.
14. Okuda A, Suzuki T, Kishi T, et al. Duration of untreated illness and antidepressant fluvoxamine response in major depressive disorder. Psychiatry Clin Neurosci 2010; 64: 268-73.
15. Altamura AC, Dell’Osso B, Mundo E, Dell’Osso L. Duration of untreated illness in major depressive disorder: a naturalistic study. Int J Clin Pract 2007; 61: 1697-700.
16. Camuri G, Oldani L, Dell’Osso B, et al. Prevalence and disability of comorbid social phobia and obsessive-compulsive disorder in patients with panic disorder and generalized anxiety disorder. Int J Psychiatry Clin Pract 2014; 18: 248-54.
17. Dell’Osso B, Camuri G, Benatti B, Buoli M, Altamura AC. Differences in latency to first pharmacological treatment (duration of untreated illness) in anxiety disorders: a study on patients with panic disorder, generalized anxiety disorder and obsessive-compulsive disorder. Early Interv Psychiatry 2013; 7: 374-80.
18. Dell’Osso B, Benatti B, Oldani L, Spagnolin G, Altamura AC. Differences in duration of untreated illness, duration, and severity of illness among clinical phenotypes of obsessive-compulsive disorder. CNS Spectr 2015; 20: 474-8.
19. Dell’Osso B, Benatti B, Buoli M, et al. The influence of age at onset and duration of illness on long-term outcome in patients with obsessive-compulsive disorder: a report from the International College of Obsessive Compulsive Spectrum Disorders (ICOCS). Eur Neuropsychopharmacol 2013; 23: 865-71.
20. Seivewright H, Tyrer P, Johnson T. Prediction of outcome in neurotic disorder: a 5-year prospective study. Psychol Med 1998; 28: 1149-57.
21. Gumz A, Uhlenbusch N, Weigel A, Wegscheider K, Romer G, Löwe B. Decreasing the duration of untreated illness for individuals with anorexia nervosa: Study protocol of the evaluation of a systemic public health intervention at community level. BMC Psychiatry 2014; 14: 1-8.
22. Compton MT, Carter T, Bergner E, et al. Defining, operationalizing and measuring the duration of untreated psychosis: advances, limitations and future directions. Early Interv Psychiatry 2007; 1: 236-50.
23. First MB, Williams JB, Karg RS, Spitzer R. SCID-5-CV: structured clinical interview for DSM-5 disorders, clinician version. Arlington, VA: American Psychiatric Publishing, 2016.
24. World Health Organization. The ICD-10 Classification of Mental and Behavioural Disorders. Int Classif 1992; 10: 1-267.
25. American Psychiatric Association. Diagnostic and statistical manual of mental disorders (5th ed.). Arlington, VA: American Psychiatric Publishing, 2013.
26. Singh SP, Cooper JE, Fisher HL, et al. Determining the chronology and components of psychosis onset: the Nottingham Onset Schedule (NOS). Schizophr Res 2005; 80: 117-30.
27. APA American Psychiatric Association, Guidelines P. No Title. 2013.
28. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159-74.
29. Reid GT, Walter FM, Brisbane JM, Emery JD. Family history questionnaires designed for clinical use: a systematic review. Public Health Genomics 2008; 12: 73-83.
30. Kendler KS. Family history information in biomedical research. J Contin Educ Health Prof 2001; 21: 215-23.
31. Gray AJ. Stigma in psychiatry. J R Soc Med 2002; 95: 72-6.
32. Hamilton M. A rating scale for depression. J Neurol Neurosurg Psychiatry 1960; 23: 56-62.
33. Montgomery SA, Asberg M. A new depression scale designed to be sensitive to change. Br J Psychiatry 1979; 134: 382-9.
34. Beck AT, Steer RA, Ball R, Ranieri WF. Comparison of Beck Depression Inventories-IA and-II in psychiatric outpatients. J Pers Assess 1996; 67: 588-97.
35. Hanel G, Henningsen P, Herzog W, et al. Depression, anxiety, and somatoform disorders: vague or distinct categories in primary care? Results from a large cross-sectional study. J Psychosom Res 2009; 67: 189-97.
36. Thombs BD, Ziegelstein RC, Pilote L, et al. Somatic symptom overlap in beck depression inventory-II scores following myocardial infarction. Br J Psychiatry 2010; 197: 61-6.
37. Gierk B, Kohlmann S, Kroenke K, et al. The Somatic Symptom Scale-8 (SSS-8). JAMA Intern Med 2014; 174: 399.
38. Haug TT, Mykletun A, Dahl AA. The association between anxiety, depression, and somatic symptoms in a large population: the HUNT-II study. Psychosom Med 2004; 66: 845-51.
39. de Waal MWM, Arnold IA, Eekhof JAH, van Hemert AM. Somatoform disorders in general practice: prevalence, functional impairment and comorbidity with anxiety and depressive disorders. Br J Psychiatry 2004; 184: 470-6.
40. Lieb R, Meinlschmidt G, Araya R. Epidemiology of the association between somatoform disorders and anxiety and depressive disorders: an update. Psychosom Med 2007; 69: 860-3.
41. Sadock BJ, Sadock VA, Ruiz P. Kaplan and Sadock’s Comprehensive Textbook of Psychiatry. Philadelphia, PA: Lippincott Williams & Wilkins, 2009.
42. Boschloo L, Nolen WA, Spijker AT, et al. The Mood Disorder Questionnaire (MDQ) for detecting (hypo)manic episodes: its validity and impact of recall bias. J Affect Disord 2013; 151: 203-8.