Publication Date

2024

Document Type

Dissertation/Thesis

First Advisor

Styck, Kara M.

Degree Name

Ph.D. (Doctor of Philosophy)

Legacy Department

Department of Psychology

Abstract

Measures of academic engagement and disruptive behavior in students are strong predictors of social, academic, and lifelong career outcomes. Educators need a measurement tool that is feasible, accurate, and cost-effective in order to identify students who may be in need of additional behavioral support. Direct Behavior Ratings (DBRs) offer a solution, but are prone to rater effects, threatening the psychometric properties of the tool. Two methods are proposed for the mitigation of such rater effects: rater training and statistical adjustment. Previous studies have demonstrated mixed results regarding the effectiveness of rater training on DBRs, and no studies have investigated the impact of statistical adjustment on the mitigation of rater effects and DBR accuracy over time. In the current study, 23 participants were recruited to complete DBRs on video recordings of middle school students at two time points. One group received training while the other received no training. Additionally, many facet Rasch measurement (MFRM) was used to investigate the impact of statistical adjustment on DBR accuracy and rater effects. Results demonstrated that across both behaviors, neither rater training nor MFRM adjustment significantly improved DBR accuracy, and rater effects were stable over time.

Extent

76 pages

Language

en

Publisher

Northern Illinois University

Rights Statement

In Copyright

Rights Statement 2

NIU theses are protected by copyright. They may be viewed from Huskie Commons for any purpose, but reproduction or distribution in any format is prohibited without the written permission of the authors.

Media Type

Text

Download

Included in

Psychology Commons

Share

COinS