Campus Units
English
Document Type
Article
Publication Version
Accepted Manuscript
Publication Date
2015
Journal or Book Title
Writing & Pedagogy
Volume
7
Issue
2-3
DOI
10.1558/wap.v7i2-3.26376
Abstract
Supported by artificial intelligence (AI), the most advanced Automatic Writing Evaluation (AWE) systems have gained increasing attention for their ability to provide immediate scoring and formative feedback, yet teachers have been hesitant to implement them into their classes because correlations between the grades they assign and the AWE scores have generally been low. This begs the question of where improvements in evaluation may need to be made, and what approaches are available to carry out this improvement.
This mixed-method study involved 59 cause and effect essays collected from English language learners enrolled in six different sections of a college level academic writing course and utilized theory proposed by Slater and Mohan (2010) regarding the developmental path of cause. The study compared the results of raters who used this developmental path with the accuracy of AWE scores produced by Criterion, an AWE tool developed by Educational Testing Service (ETS), and the grades reported by teachers.
Findings suggested that if Criterion is to be used successfully in the classroom, writing teachers need to take a meaning-based approach to their assessment, which would allow them and their students to understand more fully how language constructs cause and effect. Using the developmental path of cause as an analytical framework for assessment may then help teachers assign grades that are more in sync with AWE scores, which in turn can help students gain more trust in the scores they receive from both their teachers and Criterion.
Copyright Owner
Equinox Publishing
Copyright Date
2015
Language
en
File Format
application/pdf
Recommended Citation
Ma, Hong and Slater, Tammy, "Using the Developmental Path of Cause to Bridge the Gap between AWE Scores and Writing Teachers’ Evaluations" (2015). English Publications. 128.
https://lib.dr.iastate.edu/engl_pubs/128
Included in
Educational Assessment, Evaluation, and Research Commons, Language and Literacy Education Commons
Comments
This is a manuscript of an article published as Ma, Hong, and Tammy Slater. "Using the Developmental Path of Cause to Bridge the Gap between AWE Scores and Writing Teachers' Evaluations." Writing & Pedagogy 7 (2015). 10.1558/wap.v7i2-3.26376 Posted with permission.