Publication Date

2008

Document Type

Dissertation/Thesis

First Advisor

Silber, Kenneth H.

Degree Name

Ed.D. (Doctor of Education)

Legacy Department

Department of Educational Technology, Research and Assessment

LCSH

Educational tests and measurements--Evaluation

Abstract

This study was designed to address the change in demographic trends in the oil- and gas-refining industry as well as the recent increase in the amount of safety regulations and assessment a company needs to operate. These trends have driven the setting of this study, an international oil company, to require more performance assessment to ensure that its personnel are competent to perform their jobs safely. A critical role in this assessment process is that of technical trainer, which is used to create training and assessments for the various technical aspects of the jobs. A consistent expectation of this position is the ability to develop assessments to measure performance. The purpose of this study was to develop a valid and reliable assessment evaluation instrument and use it to measure the varying levels of feedback complexity that may affect a technical trainer’s assessment-development competency. The experiment was a pretest-posttest design, with the two types of feedback complexity being the treatment. The change in score on the instrument between the pretest and posttest was used to evaluate which feedback complexity was more effective in improving the technical trainer’s test-writing ability. The previous assessment-development training of a technical trainer was also included in the study as a variable to address the variation in the participants’ backgrounds. A two-way between-subjects analysis of variance was conducted on the experiment’s results. The level of feedback complexity during assessment development was not found to be statistically significant regarding the technical trainer’s assessment-development competency. The second variable, assessment-development training of a technical trainer, was found to be statistically significantly related to assessment-development competency. Furthermore, the interaction between the two control variables, feedback complexity and assessment-development training, was statistically significant and shown to be a critical interaction to consider for future research and practical application. The results of the experiment show that feedback from the instrument was 100% successful in improving assessment-development scores for all 40 participants in the study; this indicates that this instrument provides helpful feedback to the population and can be used as a feedback and training tool with technical trainers in the future.

Comments

Includes bibliographical references (pages 100-107)

Extent

viii, 120 pages

Language

eng

Publisher

Northern Illinois University

Rights Statement

In Copyright

Rights Statement 2

NIU theses are protected by copyright. They may be viewed from Huskie Commons for any purpose, but reproduction or distribution in any format is prohibited without the written permission of the authors.

Media Type

Text

Download

Share

COinS