Feedback at scale: designing for accurate and timely practical digital skills evaluation
European Journal of Information Systems
The global demand for digital proficiency has resulted in increasing pressure to “massify” education. As practical digital skills development becomes more important, there is a need to design accurate and timely performance feedback systems that can scale to a large number of learners. This paper contributes meta-requirements and design principles for designing a socio-technical artefact that offers a solution to the general problem of providing performance feedback at scale. The artefact evaluation provides interesting results for achieving the three objectives of a) scalability to a large number of learners, b) validity and reliability of the feedback, and c) positive impact on learners’ behaviour and engagement with the feedback system. These results are obtained through the synergistic contribution of pedagogical prioritisation (i.e., what skills to cover), assignment design (i.e., what tasks to use to evaluate mastery) and automated measurement (i.e., grading engine functionalities for error detection).
Design science research, digital skills evaluation, feedback systems, Iris Junglas, Pär Ågerfalk
Piccoli, Gabriele; Rodriguez, Joaquin; Palese, Biagio; and Bartosiak, Marcin Lukasz, "Feedback at scale: designing for accurate and timely practical digital skills evaluation" (2020). NIU Bibliography. 633.
Department of Operations Management and Information Systems (OMIS)