Campus Units
English
Document Type
Article
Publication Version
Accepted Manuscript
Publication Date
2-1-2016
Journal or Book Title
Educational Psychology: An International Journal of Experimental Educational Psychology
DOI
10.1080/01443410.2015.1136407
Abstract
An increasing number of studies on the use of tools for automated writing evaluation (AWE) in writing classrooms suggest growing interest in their potential for formative assessment. As with all assessments, these applications should be validated in terms of their intended interpretations and uses. A recent argument-based validation framework outlined inferences that require backing to support integration of one AWE tool, Criterion, into a college-level English as a Second Language (ESL) writing course. The present research appraised evidence for the assumptions underlying two inferences in this argument. In the first of two studies, we assessed evidence for the evaluation inference, which includes the assumption that Criterion provides students with accurate feedback. The second study focused on the utilisation inference involving the assumption that Criterion feedback is useful for students to make decisions about revisions. Results showed accuracy varied considerably across error types, as did students’ abilities to use Criterion feedback to correct written errors. The findings can inform discussion of whether and how to integrate the use of AWE into writing classrooms while raising important questions regarding standards for validation of AWE as formative assessment, Criterion developers’ approach to accuracy, and instructors’ assumptions about the underlying purposes of AWE-based writing activities.
Copyright Owner
Taylor & Francis
Copyright Date
2016
Language
en
File Format
application/pdf
Recommended Citation
Ranalli, Jim; Link, Stephanie; and Chukharev-Hudilainen, Evgeny, "Automated writing evaluation for formative assessment of second language writing: investigating the accuracy and usefulness of feedback as part of argument-based validation" (2016). English Publications. 79.
https://lib.dr.iastate.edu/engl_pubs/79
Included in
Bilingual, Multilingual, and Multicultural Education Commons, Curriculum and Instruction Commons, Educational Assessment, Evaluation, and Research Commons, Educational Methods Commons
Comments
This is an Accepted Manuscript of an article published by Taylor & Francis in Educational Psychology: An International Journal of Experimental Educational Psychology on February 1, 2016, available online: http://www.tandf.com/10.1080/01443410.2015.1136407.