Publication Date

2022

Document Type

Dissertation/Thesis

First Advisor

Sagarin, Brad J.

Degree Name

Ph.D. (Doctor of Philosophy)

Legacy Department

Department of Psychology

Abstract

Despite researchers’ best efforts, it still is unclear whether researchers and practitioners can feel comfortable randomizing survey items because of its inconsistent impact on psychometric properties. Using moderated non-linear factor analysis, the present study explored whether randomization, construct strength, and differences in conscientiousness and neuroticism affected respondents’ survey responding behavior to randomized and grouped versions of a communication anxiety measure. Randomization did not affect how respondents interpreted the measure’s content but did affect how they responded to matrix items that appeared next to repeated response scales. Construct strength positively predicted factor level differences, with respondents who reported feeling more certain in their answers and finding them more accessible reporting higher levels of communication anxiety and responding with greater variability. Lastly, respondents higher in conscientiousness and neuroticism were more susceptible to randomization than their counterparts, evidenced by more factor- and item-level differences. Taken together, these findings suggest researchers can randomize items without fear of affecting how respondents interpret the measure’s content. Nevertheless, randomization and grouping artificially inflated and attenuated respondent scores, respectively, calling into question the reliability of diagnoses for respondents who fall near clinically significant thresholds. Thus, I encourage researchers and practitioners to consider administering question formats besides matrices, especially those interested in using absolute scores. Ultimately, the present study emphasized the importance of considering how all survey design features can affect survey responding behavior.

Extent

260 pages

Language

eng

Publisher

Northern Illinois University

Rights Statement

In Copyright

Rights Statement 2

NIU theses are protected by copyright. They may be viewed from Huskie Commons for any purpose, but reproduction or distribution in any format is prohibited without the written permission of the authors.

Media Type

Text

Download

Included in

Psychology Commons

Share

COinS