Campus Units

English

Document Type

Article

Publication Version

Accepted Manuscript

Publication Date

2015

Journal or Book Title

Writing & Pedagogy

Volume

7

Issue

2-3

DOI

10.1558/wap.v7i2-3.26376

Abstract

Supported by artificial intelligence (AI), the most advanced Automatic Writing Evaluation (AWE) systems have gained increasing attention for their ability to provide immediate scoring and formative feedback, yet teachers have been hesitant to implement them into their classes because correlations between the grades they assign and the AWE scores have generally been low. This begs the question of where improvements in evaluation may need to be made, and what approaches are available to carry out this improvement.
This mixed-method study involved 59 cause and effect essays collected from English language learners enrolled in six different sections of a college level academic writing course and utilized theory proposed by Slater and Mohan (2010) regarding the developmental path of cause. The study compared the results of raters who used this developmental path with the accuracy of AWE scores produced by Criterion, an AWE tool developed by Educational Testing Service (ETS), and the grades reported by teachers.
Findings suggested that if Criterion is to be used successfully in the classroom, writing teachers need to take a meaning-based approach to their assessment, which would allow them and their students to understand more fully how language constructs cause and effect. Using the developmental path of cause as an analytical framework for assessment may then help teachers assign grades that are more in sync with AWE scores, which in turn can help students gain more trust in the scores they receive from both their teachers and Criterion.

Comments

This is a manuscript of an article published as Ma, Hong, and Tammy Slater. "Using the Developmental Path of Cause to Bridge the Gap between AWE Scores and Writing Teachers' Evaluations." Writing & Pedagogy 7 (2015). 10.1558/wap.v7i2-3.26376 Posted with permission.

Copyright Owner

Equinox Publishing

Language

en

File Format

application/pdf

Published Version

Share

COinS