Sagarin, Brad J.
Ph.D. (Doctor of Philosophy)
Department of Psychology
Despite researchers’ best efforts, it still is unclear whether researchers and practitioners can feel comfortable randomizing survey items because of its inconsistent impact on psychometric properties. Using moderated non-linear factor analysis, the present study explored whether randomization, construct strength, and differences in conscientiousness and neuroticism affected respondents’ survey responding behavior to randomized and grouped versions of a communication anxiety measure. Randomization did not affect how respondents interpreted the measure’s content but did affect how they responded to matrix items that appeared next to repeated response scales. Construct strength positively predicted factor level differences, with respondents who reported feeling more certain in their answers and finding them more accessible reporting higher levels of communication anxiety and responding with greater variability. Lastly, respondents higher in conscientiousness and neuroticism were more susceptible to randomization than their counterparts, evidenced by more factor- and item-level differences. Taken together, these findings suggest researchers can randomize items without fear of affecting how respondents interpret the measure’s content. Nevertheless, randomization and grouping artificially inflated and attenuated respondent scores, respectively, calling into question the reliability of diagnoses for respondents who fall near clinically significant thresholds. Thus, I encourage researchers and practitioners to consider administering question formats besides matrices, especially those interested in using absolute scores. Ultimately, the present study emphasized the importance of considering how all survey design features can affect survey responding behavior.
Pawirosetiko, Joy Samini, "Considering construct strength and personality differences on susceptibility to context effects" (2022). Graduate Research Theses & Dissertations. 7534.
Northern Illinois University
Rights Statement 2
NIU theses are protected by copyright. They may be viewed from Huskie Commons for any purpose, but reproduction or distribution in any format is prohibited without the written permission of the authors.