Campus Units

Aerospace Engineering, Industrial and Manufacturing Systems Engineering, Mechanical Engineering, Human Computer Interaction, Virtual Reality Applications Center, Gerontology

Document Type


Publication Version

Published Version

Publication Date


Journal or Book Title

Frontiers in Psychology



First Page


Research Focus Area(s)

​Operations Research, Ergonomics and Human Factors




This research assessed how the performance and team skills of three-person teams working with an Intelligent Team Tutoring System (ITTS) on a virtual military surveillance task were affected by feedback privacy, participant role, task experience, prior team experience, and teammate familiarity. Previous work in Intelligent Tutoring Systems (ITSs) has focused on outcomes for task skill training for individual learners. As research extends into intelligent tutoring for teams, both task skills and team skills are necessary for good team performance. This work includes a brief review of previous research on ITTSs, feedback, teams, and teamwork, including the recounting of two categories of a framework of teamwork performance, Communication and Cognition, which are relevant to the present study. This research examines the effects of an intelligent agent, as well as features of the team, its members, and the task being undertaken, on team communication (measured by relevant key-presses) and team situation awareness (as measured by scores on a quiz). Thirty-seven teams of three participants, each at their own computer running a multiplayer surveillance simulation, were given just-in-time private (individually delivered) or public (team-delivered) performance feedback during four 5-min trials. In the fourth trial, two of the three participants switched roles. Feedback type, teamwork experience, and teammate familiarity had no statistically significant effect on communication or team situation awareness. However, higher levels of role experience and task experience showed significant and medium-sized effects on communication performance. Results, based on performance data and structured interview responses, also revealed areas of improvement in future feedback design and a potential benchmark for feedback frequency in an action-oriented serious game-based ITTS. Among the conclusions are six design objectives for future ITTSs, establishing a foundation for future research on designing effective ITTSs that train interpersonal skills to nascent teams.


This article is published as Ouverson, Kaitlyn M., Alec G. Ostrander, Jamiahus Walton, Adam Kohl, Stephen B. Gilbert, Michael C. Dorneich, Eliot Winer, and Anne M. Sinatra. "Analysis of Communication, Team Situational Awareness, and Feedback in a Three-Person Intelligent Team Tutoring System." Frontiers in Psychology 12 (2021): 553015. DOI: 10.3389/fpsyg.2021.553015.


Works produced by employees of the U.S. Government as part of their official duties are not copyrighted within the U.S. The content of this document is not copyrighted.



File Format