Publication Date
2026
Document Type
Dissertation/Thesis
First Advisor
Alhoori, Hamed
Degree Name
M.S. (Master of Science)
Legacy Department
Department of Computer Science
Abstract
Large language models (LLMs) have shown strong promise in healthcare, especially for tasks that involve clinical reasoning and medical question answering. However, their use in practice raises important concerns about safety, reasoning quality, and reliability across different specialties. This thesis studies these challenges through three parts: a background review of LLMs in health, a structured multi-specialty framework, and a clinically grounded evaluation in a specialized domain.First, the thesis reviews the broader role of LLMs in healthcare, including their applications and the main challenges associated with them, such as hallucination, bias, privacy, explainability, evaluation, and deployment.
Second, the thesis studies a multi-specialty Mixture-of-Experts (MoE) framework that uses a DSPy-based dispatcher to decide which specialty best fits each question. The corresponding specialty agent is then used to answer the case. This architecture is used to examine whether specialty-based routing improves performance across different medical specialties.
Third, the work evaluates LLM performance on complex, open-ended pediatric gastroenterology cases using clinically grounded evaluation criteria. Moving to open-ended questions provides more room to evaluate model behavior and diagnostic capabilities.
Overall, this research shows that while benchmark performance is an important metric, it is insufficient for assessing the readiness of LLMs for real-world cases. Reasoning quality, clinically grounded evaluation, and performance across different specialties all matter when assessing how these systems may be used in healthcare.
Recommended Citation
Khaizaran, Dalia, "Evaluating Large Language Models in Healthcare: Multi-Specialty Reasoning and Clinical Benchmarking" (2026). Graduate Research Theses & Dissertations. 8215.
https://huskiecommons.lib.niu.edu/allgraduate-thesesdissertations/8215
Extent
78 pages
Language
en
Publisher
Northern Illinois University
Rights Statement
In Copyright
Rights Statement 2
NIU theses are protected by copyright. They may be viewed from Huskie Commons for any purpose, but reproduction or distribution in any format is prohibited without the written permission of the authors.
Media Type
Text
