This dissertation study is intended to develop a test of productive grammatical writing ability in academic English as a supplement to the essay test of the English Placement Test at Iowa State University. In addition, it attempts to validate the score interpretation and use of the academic grammar test, using a validation framework adapted from both Kane's (2006) interpretive/validity argument model and Bachman and Palmer's (2010) Assessment Use Argument (AUA) model. Philosophically grounded in a constructivist-realist approach to the interpretation of test scores, the test particularly aims to measure non-native English speaking students' grammatical writing ability exerted in the academic setting. The entire processes of test development and evaluation are guided by an interpretive argument, which consists of seven inferences--Domain Description, Evaluation, Generalization, Extrapolation, Explanation, Utilization, and Ramification. Warrants for these seven inferences and underlying assumptions are identified through the interpretive argument, and the necessary types of backing evidence are also specified, to support these assumptions.

Using the quantitative and qualitative data collected in F12 and Sp13, the study investigates whether each of the seven inferences is sustained by sufficient backing evidence. Overall, the test appears to elicit the target construct of the test effectively from test takers, reflecting the linguistic characteristics of academic writing as a target language use (TLU) domain. Evidence pertaining to score interpretations also supports several claims the examinees' performance on the academic grammar test corresponds to their ability to use the target construct in both testing and non-testing writing contexts, and satisfies theoretical expectations to some extent. Evidence for the validity of the test use for the intended purpose is sought from the interviews with three different stakeholders (i.e., the EPT coordinator as a decision-maker, students as test takers, and instructors of ESL writing courses), by asking their perceptions about potential positive and negative impacts of the use of the academic grammar test as well as about the value of grammatical writing ability. Based on these pieces of supporting evidence, a validity argument for score interpretation and use of the academic grammar test is successfully established, despite only weak evidence for a few of the inferences.

Three issues related to the improvement of backing are discussed in regard to task design, the research in the acquisitional order of advanced grammatical features, and the revision of ESL writing curriculum and pedagogical approaches to grammar instruction. This dissertation concludes with a summary of the validity argument and a brief discussion of two logistic issues concerning test implementation, methodological limitations of the study, and suggestions for future research.