Publication date: Jul 03, 2025
Inflammatory bowel disease (IBD) is a chronic autoimmune disorder with an increasing prevalence in the general population. Internet-based communities have become vital for communication among patients with IBD, especially throughout the COVID-19 pandemic. However, these internet-based patient-to-patient communications remain largely underexplored. This study aims to analyze community posts from 3 of the largest IBD support groups on Reddit between March 1, 2020, and December 31, 2022, using a pretrained transformer model, and to validate the classification system’s results via comparison to human scoring. We collected posts (N=53,333) from subreddits r/CrohnsDisease, r/UlcerativeColitis, and r/IBD and classified them using OpenAI’s GPT-3. 5 Turbo model to determine sentiment, categorize topics, and identify demographic information and mentions of the COVID-19 pandemic. A subset of posts (n=397) was manually scored to measure interrater agreement between human raters and the GPT-3. 5 Turbo model. Fleiss _705 and Gwet AC1 coefficients indicated a high level of agreement between raters, with values ranging from 0. 53 to 0. 91. The raters demonstrated almost perfect agreement on the classification of gender, with a Fleiss _705 of 0. 91 (P
Open Access PDF
Semantics
| Type | Source | Name |
|---|---|---|
| disease | MESH | Inflammatory Bowel Disease |
| pathway | KEGG | Inflammatory bowel disease |
| disease | MESH | COVID-19 Pandemic |