A rapid screening tool for anorectic/bulimic risk in online settings: a pilot-study on the DiCA33 validation in Italian young female students

Alessandro Chinello1, Raffaele Simone Scuotto2, Simone Cadeo1, Luigi Enrico Zappa1,
Paola Ricciardelli2,3

118.56 Monitor Lab, Maria Bianca Corno Foundation, Monza, Italy; 2Department of Psychology, University of Milano-Bicocca, Milan, Italy; 3NeuroMI: Milan Center for Neuroscience, Milan, Italy.

Summary. The early identification of anorexia (AN) and bulimia nervosa (BN) in the general population represents a crucial strategy to avoid their chronicization and clinical worsening. This pilot-study aims to test the validity of a new screening tool (DiCA33) dedicated AN/BN risk in online settings, based on the Italian version of EAT-26, a self-report questionnaire for measuring AN/BN symptoms. First analyses excluded the effect of demographic factors on results and suggested a limited explanation power of the mere total scores of DiCA33 for risk detection. Alternatively, a selection of risked items from the DiCA33 checklists (as evidenced on EAT-26 scores), then combined in a subscale, showed a necessary sensitivity for screening purposes. The DiCA33-subscale constitutes a reliable and useful index for the early and quick detection of AN/BN risk in young Italian female population, composed mainly by students. Considering the non-diagnostic nature of this tool, subsequent rigorous and psychiatric evaluations are necessary for positive cases to confirm the risk. Further studies may validate the tool even recruiting patients with eating disorders to improve tool specificity.

Key words. Eating disorders, internet, screening.

Uno strumento di screening rapido per il rischio anoressico/bulimico in ambito online: uno studio-pilota sulla validazione DiCA33 nelle giovani studentesse italiane.

Riassunto. L’individuazione precoce di anoressia (AN) e bulimia nervosa (BN) nella popolazione generale rappresenta una strategia cruciale per evitare la cronicizzazione e il peggioramento clinico. Il presente studio pilota ha come obiettivo primario la validazione di un nuovo strumento di screening (DiCA33) dedicato all’identificazione del rischio di AN/BN in internet, utilizzando la versione italiana dell’EAT-26, un questionario autosomministrato per l’individuazione dei sintomi di AN/BN. Dalle analisi iniziali, non risultano influenze da variabili demografiche; tuttavia, il punteggio totale DiCA33 ha un limitato potere nell’individuazione del rischio AN/BN. In alternativa, una selezione degli item a rischio dalle liste DiCA33 (come dimostrato dai punteggi di EAT-26), poi combinati in una sottoscala, ha mostrato una buona sensibilità per gli obiettivi di screening. La sottoscala del DiCA33 rappresenta un indicatore di screening affidabile e valido per la precoce individuazione del rischio AN/BN nella giovane popolazione italiana femminile, composta principalmente da studenti. Considerando la natura non diagnostica di questo strumento, risulta fondamentale una successiva e rigorosa valutazione psichiatrica dei casi positivi per confermarne il rischio. Ulteriori studi potrebbero validare lo strumento includendo pazienti con disturbi dell’alimentazione al fine di migliorare la specificità dello strumento.

Parole chiave. Disturbi dell’alimentazione, internet, screening.

Introduction

Eating disorders (ED) constitute complex psychiatric disorders requiring multiple and integrated interventions offered by a multiprofessional team. In adolescents and young adults, high mortality risk is associated with all eating disorders, in particular for anorexia nervosa (AN) in young women1-3.

Specifically, the early identification of these disorders in the general population represents a crucial strategy to avoid their chronicization with beneficial consequences on clinical resistance and treatment duration4,5, based on a rapid orientation of individuals at risk to mental health services6. However, about 50% of cases are not detected before their clinical and significant manifestation7,8.

In the last decades, few studies focused their attention on the development of new screening tools for the ED risk detection on the Italian population, administered more in real settings than online spaces (website, blog, forum). Some efforts in this direction are represented by the validation of multi-items self-reports, such as EAT-269-11 .This 26-items screening instrument can adequately identify ED cases in specialist settings9,12,, focusing especially on behaviors and attitudes of anorexia and bulimia nervosa13, but poor identification of other EDs (i.e., BED) or subclinical variants13-15. Other tools, such as rapid questionnaires16 employ an easy screening of DSM-5 eating disorders, but without enough evidence in settings about primary care and communities17. As screening tools, all these questionnaires recommend a post-screening clinical evaluation in order to make a correct diagnosis.

The present pilot-study aims to test the validity of a new screening tool (DiCA33) created for the detection of anorexia and bulimia nervosa risk in online settings, based on the EAT-26 scores, and having the following structure:

• a brief checklist, as an alternative to mostly diffuse questionnaire tools available in literature requiring long time to answer to the multi-items they are constructed by and involving linguistic loads that influence questions comprehension. In particular, a food checklist was chosen since it is composed of more concrete and understandable items about daily life and requires an easier and rapid procedure to be completed18.

• two checklist subscales aimed to disentangle weekly eating patterns and food preferences, by considering both diet histories and individual desires associated with food intake19-21.

• a selection of critical foods extracted from a linguistic corpus of the Italian ProAna websites/blogs22 characterizing the “anorectic vocabulary” on internet. Indeed, the so-called “ProAna phenomenon” promotes anorectic lifestyles with a reduction of life quality, self-esteem and body satisfaction in adolescents and ProAna blog users23,24. In particular, it is known that the participation in online communities emphasizing restrictive behaviors, weight loss, vomiting, use of drugs and diets that may constitute potential risk factors for ED25-27. Therefore, in validating the “DiCA33” tool, we also took into account several demographic variables (including the covid-19 related ones), such as age, sex, BMI, stress levels and modifications of eating patterns during the covid-19 pandemic.

Our ultimate aim was to provide a valid, rapid, easy, everyday language-dependent screening instrument based on online Proana items, dedicated to young women and possibly administered in online settings (blog, chat, forum).

Methods

Participants

The participant recruitment followed specific inclusion criteria as preliminary conditions to start the study. In particular, the participants had to be: 1) Italian native females, 2) aging between 18 and 35 years old, 3) without past or current neuropsychiatric diagnoses and 4) giving their informed consent about the study participation and privacy. The non-compliance of these criteria led to the automatic exclusion of the participant from the study. The study was approved by the ethics committee of the Psychology Department of the University of Milan-Bicocca (CRIP, N°RM 2020-302). A minimum sample size of 88 participants was decided according to a priori power analysis (|p|=.3, alfa=.05, Power=.9) using G*Power software28.

From an initial sample of 338 participants, we excluded from the analysis 49 participants due to the exclusion criteria and 24 participants due to incomplete questionnaires/partial data. The final sample was composed of 265 Italian women, mostly with age between 18 and 25 years old (n=203, 76.6%), followed by 25-30 (n=51, 19.2%), and 30-35 age ranges (n=11, 4.2%).

Procedure

From 22nd/June/2020 to 22nd/August/2020, the recruitment procedure took place through direct contacts of researchers considering online forum, pages and communities of university students on Facebook, Instagram and Whatsapp apps (sending a specific link to potential participants via email; 91.6%) and through the Sona System (Department of Psychology, University of Milan-Bicocca; 9.4%). This system is a web-based university database dedicated to student recruitment for scientific studies, associated with formative course credits acquisition (+0.1 CFU).

The study consisted of an online survey created with Qualtrics software29 (a tool to create web based surveys and generate reports), which presented a demographic questionnaire made of two parts: the EAT-26 self-report questionnaire9 and the DiCA33 checklists22:

• The following demographic and individual variables were collected for each participant: age range (18-24, 25-30, 30-35 y.o.), sex (M/F), neuropsychiatric diagnoses (presence/absence), height and weight (for BMI calculation), education, profession, city of residence, stress level associate to covid-19 pandemic (10-points Likert scale, “In your opinion, how does the current covid-19 situation influence your stress level?”) and modifications of eating patterns during the covid-19 pandemic (10-points Likert scale, “In your opinion, how does the current covid-19 situation influences your eating habits?) in consideration of the specific recruitment interval30.

• The Italian version of the EAT-26 questionnaire10, a screening self-report validated on non-clinical populations for anorexia and bulimia nervosa risk detection13. This tool is composed of twenty-six items scored on a 6-points Likert scale (from “Never” to “Always”) with a cutoff at 20 points (total score) for ED cases.

• The DiCA33 checklist was based on specific foods (N=23) previously identified on 10 ProAna websites22,24. The procedure involved the selection of foods contained in the two separated checklists following different instructions: for the Real subscale the R-question was: “Which of these foods and drinks have you taken in the last two weeks?”, whereas for the Preference subscale the P-question was: “Which of these foods and drinks would you like to take?”. Each item in the checklist and each subscale were randomly shown, controlling for order effect.

Results

The final sample was made of mostly students (74.0%) with high school diploma (47.2%) or bachelor degree (36.2%) (table 1), living mainly in the Lombardy region (71.3%).




Individual Body Mass Index (BMI) was calculated from both height and weight, showing a mean BMI of 21.7 kg/m2 (SD=3.8). Considering a pandemic situation (covid-19) during the data collection30 we introduced two 0-10 points scales to control for potential influences of this variable on the results, examining the level of self-perceived stress (mean=5.2, SD=2.4) and recent modifications of eating patterns (mean=4.3, SD=2.7).

The EAT-26 self-report showed an average score of 8.3 (SD=8.8). The traditional cut-off (>20)10 separates two subgroups with different risk levels: at risk (RK+; N=36, 13.6%) or not at risk (RK-; N=229, 86.4%). Both subgroups are homogenous about the main demographic variables (table 2).




Food selection (total score) was significantly different between the Preference (mean=8.8, SD=4.4; min 0, max=22) and Real subscales (mean=11.3, SD=3.0; min=3, max=22; t(264)=-9.22, p<.001), with a significant correlation between the two subscales (r=.41, p<.001).

Considering the total scores, correlations between EAT-26 score and DiCA33 versions did not find any significant results: Preference (p=.38), Real (p=.29) and merged versions (p=.26). Moreover, t-test contrasts do not show significant differences between RK+ and RK- subgroups in relation to the numerosity of selected items (total score) of Preference version (p=.36), Real version (p=.47), and the merged version (p=.32).

Specifically, a subsequent analysis was performed to better identify the crucial role of some DICA33 items on AN/BN risk by using T-test contrasts based on EAT-26 scores contrasting the selection (or not) of each DiCA33 item (only significant contrasts are displayed in table 5).




After identifying those items associated to a significant EAT-26 mean differences (if selected or not by the participants, see Table 5), then we created a composed subscale with “+” (if selected item > not selected item, Table 5) or “–“ (if selected item < not selected, Table 5), as following: P_frutti di bosco + P_pasta - P_yogurt magro + R_frutti di bosco + R_pasta + R_biscotti secchi - R_yogurt magro + R_succo di frutta. Interestingly, the DICA33-subscale was negatively correlated with EAT-26 score (r=-.44, p<.001). Then, the DiCA33-subscale was tested in a ROC curve, between RK+/RK- subgroups, showing higher sensitivity (.85) than specificity (.58; AUC=.75, p=.<001) with a cutoff=1.5 at max Yuden index (figure 1).




Discussion

This pilot-study focuses on the validation of an innovative screening tool to evaluate the AN/BN risk in online settings based on checklists of ProAna foods in Italy. Indeed, the main innovative feature of DiCA33 checklist is its web-based nature: items of the checklist were selected on the basis of previous studies on ProAna online communities22,24 and the entire tool administration procedure was done online via Qualtrics. Aimed at a quick identification of AN/BN risk in the female population, the study recruited adult women, mostly young (18-25 y.o.), a population constituting the main users of ProAna communities31,32 and particularly at risk to develop anorexia and bulimia nervosa33,34.

The DiCA33 validation was based on EAT-26 scores, a self-report questionnaire widely adopted to screen the eating disorders risk on the general population (AN/BU)9. In general, participants food selection frequencies showed how women in our sample consume less food (R: 9 items) than desired food (P:11 items) with highly selected food (>40%; e.g., water, pasta, fresh fruits, lettuce) for both Real and Preference checklists (tables 3, 4) as typical Mediterranean dietary food in Italy.

This pilot-study reports the first validation of DiCA33 that adopted a quantitative approach. Analyses based on the EAT-26 cutoff separates two subgroups: at risk (RK+) or not (RK-), with similar demographic variables (table 2), suggesting an homogeneous distribution of these features between subgroups, independent from eating disorder (ED) risk. Considering the covid-19 pandemic during data collection (2020), we introduced some factors with the aim to control for any effect of this situation on our results. External factors associated with stress levels, modification of eating patterns during pandemic and other demographic variables do not influence subgroups (table 2).




In general, our findings showed a significant internal correlation between checklists, but the absence of mutual correlations between EAT-26 and DiCA33 (based on total scores), suggesting the limited explanation power of the mere total scores of the two checklists. In consideration of this limit, the subsequent analyses focused the attention on the selection of those items found highly influenced by risk detection. Indeed, by taking into account the results on Table 5, we were able to identify those foods on both DiCA33 checklists determining significant differences on the AN/BN risk score (measured by EAT-26). A specific subscale (P_frutti di bosco + P_pasta - P_yogurt magro + R_frutti di bosco + R_pasta + R_biscotti secchi - R_yogurt magro + R_succo di frutta) is based on the EAT-26 means differences. Among them, the item “low-fat yogurt”, characterized by a linguistic label (“low-fat”), was present in both checklists, confirming its inclusion in the restricted diets of individuals at risk of AN/BU, as previously shown22, while the selection of “Pasta” item seems to be more associated with a reduced AN/BN risk.




In addition, the ROC validation of DiCA33-subscale showed a necessary sensitivity (.847) for a screening tool aimed to detect eating symptoms risk in young adult women on an online setting. Indeed, a high rate of false positive cases is acceptable for a screening tool, as found in other rapid and easy questionnaires (i.e. SCOFF: sensitivity= 93.5%, specificity= 53.7%35). These findings underline the useful role of DiCA33-subscale as preliminary detection that need, then, to be followed by a clinical/diagnostic interview for positive cases.

As pilot-study, these findings present some limitations. Firstly, the sample considered only young adults (>18 y.o.) with the exclusion of teenagers, with a possible delay about risk detection on this fragile subpopulation36. So, further research is needed to establish validity and reliability of DiCA33-subscale in a younger population. Secondly, the sample is composed by only women. The lack of males in the present sample was due to two main reasons: 1) young women and female adolescents are affected more by eating disorders (AN/BU) and the girls are the individuals mainly involved in ProAna communication32,37; 2) according to recent data38, male manifestation of AN/BN risk may be different, with abnormal eating patterns including a drive to gain weight and an overregulation of protein consumption. Lastly, further study is required to determine if DiCA33-subscale may detect even subclinical ED forms or other abnormal eating behaviors (i.e., BED), in addition to people at risk for anorexia/bulimia nervosa. Finally, further studies may validate the DiCA33-subscale even recruiting patients with eating disorders to improve specificity.

In conclusion, the DiCA33-subscale is a reliable and useful index for the early and quick detection of anorexia and bulimia nervosa risk in young Italian women, mainly students. As an online screening tool, subsequent rigorous and psychiatric evaluations are necessary for positive cases to confirm the risk. If compared to other risk questionnaires (EAT-26, SCOFF), the DiCA33 strengths consist in a low linguistic/comprehension load that reduces possible errors related to misunderstanding of sentences. Then, the neutrality of a food checklists might solve the problem of false negatives in screening assessment, particularly present in patients with eating disorders who often deny the symptoms severity39.

Conflict of interests: the authors have no conflict of interests to declare.

References

1. World Health Organization (WHO), 2017.

2. Wilson GT, Shafran R. Eating disorders guidelines from NICE. Lancet 2005; 365: 79-81.

3. Smink FR, Van Hoeken D, Hoek HW. Epidemiology of eating disorders: incidence, prevalence and mortality rates. Curr Psychiatry Rep 2012; 14: 406-14.

4. Steinhausen HC. The outcome of anorexia nervosa in the 20th century. Am J Psychiatry 2002; 159: 1284-93.

5. Keel PK, Brown TA. Update on course and outcome in eating disorders. Int J Eat Disord 2010; 43: 195-204.

6. Bravender T, Bryan-Waugh R, Herzog D, et al. Classification of eating disturbance in children and adolescents: Proposed changes for the DSM‐V. Eur Eat Disord Rev 2010; 18: 79-89.

7. Becker A, Grinspoon S, Klibanski A, Herzog D. Assessment and management of eating disorders. N Engl J Med 1999; 14: 1092-8.

8. Pritts, SD, Susman J. Diagnosis of eating disorders in primary care. Am Fam Physician 2003; 67: 297-304.

9. Garner DM, Olmsted MP, Bohr Y, Garfinkel P. The eating attitudes test: psychometric features and clinical correlates. Psychol Med 1982; 12: 871-8.

10. Dotti A, Lazzari R. Validation and reliability of the Italian EAT-26. Eat Weight Disord 1998; 3: 188-94.

11. Marronaro M, Rossi A, Aquilio E, Scacchioli F. Eating Attitude Test (EAT-26) in una popolazione di 1809 studenti delle scuole medie della provincia dell’Aquila. Psicologia Clinica dello Sviluppo 2009; 13: 195-202.

12. Williams P, Hand D, Tarnopolsky A. The problem of screening for uncommon disorders – a comment on the eating attitudes test. Psychol Med 1982; 12: 431-4.

13. Orbitello B, Ciano R, Corsaro M, et al. The EAT-26 as screening instrument for clinical nutrition unit attenders. Int J Obes (Lond) 2006; 30: 977-81.

14. Hay PJ, Marley J, Lemar S. Covert eating disorders: the prevalence, characteristics and help-seeking of those with bulimic eating disorders in general practice. Prim Care Community Psychiatr 1998; 4: 95-9.

15. Mintz LB. The eating attitude test: validation with DSM IV Eating Disorder Criteria. J Pers Assess 2000; 74: 489-503.

16. Pannocchia L, Di Fiorino M, Giannini M, Vanderlinden J. A psychometric exploration of an Italian translation of the SCOFF questionnaire. Eur Eat Disord Rev 2011; 19: 371-3.

17. Kutz AM, Marsh AG, Gunderson CG, Maguen S, Masheb RM. Eating disorder screening: a systematic review and meta-analysis of diagnostic test characteristics of the SCOFF. J Gen Intern Med 2020; 35: 885-93.

18. Wilson C. Credible checklists and quality questionnaires: a user-centered design method. Waltham, MA: Morgan Kaufman, 2013.

19. Drewnowski A. Obesity and eating disorders: cognitive aspects of food preference and food aversion. Bull Psychon Soc 1991; 29: 261-4.

20. French SA, Perry CL, Leon GR, Fulkerson JA. Food preferences, eating patterns, and physical activity among adolescents: correlates of eating disorders symptoms. J Adolesc Health 1994; 15: 286-94.

21. Stoner SA, Fedoroff, IC, Andersen AE, Rolls BJ. Food preferences and desire to eat in anorexia and bulimia nervosa. Int J Eat Disord 1996; 19: 13-22.

22. Chinello A, Parma F, Frigerio F, et al. Food semantics on pro-anorexia websites in Italy. Riv Psichiatr 2020; 55: 297-300.

23. Fox N, Ward K, O’Rourke A. Pro-anorexia, weight-loss drugs and the internet: an “anti-recovery” explanatory model of anorexia. Sociol Health Illn 2005; 27: 944-71.

24. Richichi V, Chinello A, Parma F, Zappa LE, Mazzoni E, Monti F. Anoressia nervosa e internet. Uno studio sui blog Pro-Ana in Italia. Psicologia Clinica dello Sviluppo 2018; 22: 499-514.

25. Tierney S. The dangers and draw of online communication: pro-anorexia websites and their implications for users, practitioners, and researchers. Eat Disord 2006; 14: 181-90.

26. Bardone-Cone AM, Cass KM. What does viewing a pro‐anorexia website do? An experimental examination of website exposure and moderating effects. Int J Eat Disord 2007; 40: 537-48.

27. Mulè A, Sideli L. Eating disorders on the web: risks and resources. Stud Health Technol Inform 2009; 144: 8-12.

28. Faul F, Erdfelder E, Buchner A, Lang AG. Statistical power analyses using G*power 3.1: tests for correlation and regression analyses. Behav Res Methods 2009; 41: 1149-60.

29. Qualtrics. Qualtrics survey software. Provo, UT: Qualtrics, 2005.

30. Fernández‐Aranda F, Casas M, Claes LB, et al. COVID‐19 and implications for eating disorders. Eur Eat Disord Rev 2020; 28: 239.

31. Arseniev-Koehler A, Lee H, McCormick T, Moreno MA. # Proana: Pro-eating disorder socialization on Twitter. J Adolesc Health 2016; 58: 659-64.

32. Bert F, Gualano MR, Camussi, E, Siliquini R. Risks and threats of social media websites: twitter and the proana movement. Cyberpsychol Behav Soc Netw 2016; 19: 233-8.

33. Abbate-Daga G, Pierò A, Rigardetto R, Gandione M, Gremaglia C, Fassino S. Clinical, psychological and personality features related to age of onset of anorexia nervosa. Psychopathology 2007; 40: 261-8.

34. Fairburn CG, Harrison PJ. Eating disordess. Lancet 2003; 361: 407-16.

35. Solmi F, Hatch SL, Hotopf M, Treasure J, Micali N. Validation of the SCOFF questionnaire for eating disorders in a multiethnic general population sample. Int J Eat Disord 2015; 48; 312-6.

36. Rohde P, Stice E, Marti CN. Development and predictive effects of eating disorder risk factors during adolescence: implications for prevention efforts. Int J Eat Disord 2015; 48: 187-98.

37. Norris ML, Boydell KM, Pinhas L, Katzman DK. Ana and the internet: a review of pro-anorexia websites. Int J Eat Disord 2006; 39: 443-7.

38. Murray SB, Nagata JM, Griffiths S, et al. The enigma of male eating disorders: a critical review and synthesis. Clin Psychol Rev 2017; 57: 1-11.

39. Vandereycken W, Pierloot R. Drop‐out during in‐patient treatment of anorexia nervosa: a clinical study of 133 patients. Br J Health Psychol 1983; 56: 145-56.