Exploring Large Language Models for Summarizing and Interpreting an Online Brain Tumor Support Forum
Author ORCID Identifier
Christy Muasher-Kerwin: https://orcid.org/0009-0003-8201-2995
M. Courtney Hughes: https://orcid.org/0000-0002-8699-5701
Michelle L Foster: https://orcid.org/0009-0003-1786-2686
Ibrahim Al Azher: https://orcid.org/0009-0007-9377-6359
Hamed Alhoori: https://orcid.org/0000-0002-4733-6586
Document Type
Article
Publication Title
Digital Health
Abstract
Objective
This study explored the capabilities of large language models (LLMs) GPT-3.5, GPT-4, and Llama 3 to summarize qualitative data from an online brain tumor support forum, assessing the differences between these methods and traditional thematic analysis.Methods
Eight posts and responses were collected in September 2024 from the American Brain Tumor Association Brain Tumor Support Group, using the passive/unobtrusive method. The data were analyzed using two methods: (1) traditional thematic coding with Dedoose software and (2) summarization and interpretation using LLMs. Prompts guided the LLMs in generating summaries and identifying key challenges, with results evaluated using the metrics BLEU, ROUGE-1, ROUGE-2, ROUGE-L, METEOR, and BERTScore (f1). Flesch-Kincaid grade levels and readability ease scores were also calculated and compared.Results
GPT-4 demonstrated superior performance across ROUGE and METEOR metrics, outperforming GPT-3.5 and Llama 3. Semantic similarity scores were comparable across models. GPT-4's capacity to process entire transcripts increased efficiency, while GPT-3.5 and Llama 3 required data segmenting. Summaries produced by LLMs aligned closely with human-generated thematic analysis, with significant reductions in time and labor.Conclusion
LLMs, particularly GPT-4, show strong potential for summarizing complex, qualitative health data, offering time-efficient and consistent outputs. These tools may enhance research efficiency and support in patient-centered environments. However, challenges such as training data biases and capacity limitations in some models warrant further investigation.DOI
https://doi.org/10.1177/20552076251337345
Publication Date
2025
Recommended Citation
Muasher-Kerwin, C., Hughes, M. C., Foster, M. L., Al Azher, I., & Alhoori, H. (2025). Exploring large language models for summarizing and interpreting an online brain tumor support forum. Digital Health, 11, 20552076251337345.
Original Citation
Muasher-Kerwin, C., Hughes, M. C., Foster, M. L., Al Azher, I., & Alhoori, H. (2025). Exploring large language models for summarizing and interpreting an online brain tumor support forum. Digital Health, 11, 20552076251337345.
Department
School of Health Studies| School of Allied Health and Communicative Disorders| Department of Computer Science
