Styck, Kara M.
M.A. (Master of Arts)
Department of Psychology
Educators have a responsibility to accurately measure student behavior in order to identify students in need of additional behavioral support. Current behavior screening tools can be lengthy or difficult to complete, and Direct Behavior Ratings (DBRs) offer a solution. However, DBRs are rater-mediated assessments, prone to rater effects. Rater training methods can be used to mitigate these rater effects, but previous research has not investigated the best training method for reduction of rater effects. Additionally, Many Facet Rasch Measurement (MFRM) provides an opportunity to adjust student ratings in response to individual rater tendencies of severity and leniency. Therefore, the primary purpose of the study was to investigate the optimal training method for the reduction of rater effects in DBRs. A secondary purpose was to identify the training method that produced the highest level of external criterion validity between DBRs and Systematic Direct Observations (SDOs). Results indicated that rater training reduced rater effects for DBRs on Disruptive Behavior, but did not significantly reduce rater effects for DBRs on Academic Engagement. Additionally, rater training produced the highest levels of validity across all DBRs. Implications for practice include providing training to raters and using MFRM to identify problematic DBR items and achieve high DBR validity.
Pruitt, Abigail E., "The Effects of Rater Training on Rater Effects and Validity of Direct Behavior Ratings" (2022). Graduate Research Theses & Dissertations. 7569.
Northern Illinois University
Rights Statement 2
NIU theses are protected by copyright. They may be viewed from Huskie Commons for any purpose, but reproduction or distribution in any format is prohibited without the written permission of the authors.