Date of Award
Doctor of Philosophy
Applied Linguistics and Technology
Carol A. Chapelle
In response to growing concerns over aviation safety stemming from the limited command of aviation English by non-native English speaking practitioners, this study aimed to demonstrate the development process of aviation English test tasks in a virtual environment and investigate the validity for a task-based aviation English performance assessment in the context of Korean Army Aviation. In the current dissertation study, the development and validation of the VITAEA test were based on four inferences – domain description, evaluation, generalization, and explanation – and underlying assumptions in an interpretive argument that developed with reference to argument-based validity, evidence-centered design, target language use situation analysis for test development in language for specific purposes, and task-based language assessment. Adopting a mixed method with a triangulation design, qualitative and quantitative evidence was collected to provide valid support for the inferences and to strengthen the validity argument.
Based on a task-based needs analysis with 81 military air traffic controllers on the required aviation English knowledge, skills, processes, target tasks, and task procedures in the TLU situations, virtual interactive aviation English tasks were developed in Second Life. A total of 20 controllers completed the prototype virtual interactive tasks for aviation English assessment, and their output was then rated by two rater groups, one engaging in task-centered rating and one accomplishing language-centered rating. Data included 20 task-based performance assessment sample audio files; 19 follow-up test taker interviews and online survey questionnaires; three language-centered raters' post-rating questionnaire responses; two task-centered raters' post-rating interview transcripts; military aviation English training manuals and references; and coded transcripts of 12 test takers' stimulated recall and their actual task performance.
The validity evidence collected in the various phases of test development and validation serves as backing for the four inferences in the interpretive argument as well as provides invaluable resources for the revision of the prototype virtual interactive tasks for aviation English assessment. Furthermore, empirical processes for prototype test development and partial validation based on the theoretical guidance presented in this dissertation study can be seen as among the first to be constructed utilizing the three theoretical frameworks – argument-based approach, evidence-centered design, and task-based language assessment. In addition, this dissertation study can shed light on the steps required in application of an argument-based approach for task-based second language assessment. Lastly, this study provides additional grounds for the potential use of an immersive interface and simulated target language use situations in virtual environments to provide test takers with more authentic opportunities to perform the target tasks.
Park, Moonyoung, "Development and validation of virtual interactive tasks for an aviation English assessment" (2015). Graduate Theses and Dissertations. 14933.
Applied Linguistics Commons, Bilingual, Multilingual, and Multicultural Education Commons, Curriculum and Instruction Commons, Educational Assessment, Evaluation, and Research Commons, English Language and Literature Commons, Instructional Media Design Commons