Feedback at scale: designing for accurate and timely practical digital skills evaluation

Publication Title

European Journal of Information Systems

ISSN

0960085X

E-ISSN

14769344

Document Type

Article

Abstract

The global demand for digital proficiency has resulted in increasing pressure to “massify” education. As practical digital skills development becomes more important, there is a need to design accurate and timely performance feedback systems that can scale to a large number of learners. This paper contributes meta-requirements and design principles for designing a socio-technical artefact that offers a solution to the general problem of providing performance feedback at scale. The artefact evaluation provides interesting results for achieving the three objectives of a) scalability to a large number of learners, b) validity and reliability of the feedback, and c) positive impact on learners’ behaviour and engagement with the feedback system. These results are obtained through the synergistic contribution of pedagogical prioritisation (i.e., what skills to cover), assignment design (i.e., what tasks to use to evaluate mastery) and automated measurement (i.e., grading engine functionalities for error detection).

First Page

114

Last Page

133

Publication Date

3-3-2020

DOI

10.1080/0960085X.2019.1701955

Keywords

Design science research, digital skills evaluation, feedback systems, Iris Junglas, Pär Ågerfalk

Department

Department of Operations Management and Information Systems (OMIS)

Link to Full Text

Share

COinS