Campus Units

Aerospace Engineering, Industrial and Manufacturing Systems Engineering, Human Computer Interaction, Virtual Reality Applications Center

Document Type

Conference Proceeding

Publication Version

Published Version

Publication Date


Journal or Book Title

Workshop Proceedings: Approaches and Challenges in Team Tutoring

First Page


Last Page


Research Focus Area(s)

​Operations Research, Ergonomics and Human Factors

Conference Title

Approaches and Challenges in Team Tutoring Workshop (TTW-AIED 2019)

Conference Date

June 29, 2019


Chicago, IL


In this paper we introduce a conceptual framework for the design of automated team evaluation processes (FATE), inspired by lessons learned from multiple intelligent team tutoring experiences. The framework consists of five phases. The first, Team Construct, defines the theoretical basis of the evaluation and therefore the end goal of the evaluation process. The second, Behavioral Markers, defines a method for identifying the otherwise unobservable constructs. The third, Raw Data, defines the data to be captured and recorded. The fourth, Enriched State Representation, defines a method for making the data directly relevant for team evaluation. The fifth, Team Metric, is the end goal of the evaluation defined by team constructs and derived from the enriched state representation. The framework is organized in a “V” shape to act both as a hierarchical model relating teaming theory to scenario-specific data and as a sequential process flow diagram representing the steps recommended to design an ideal team evaluation process. Each phase of the framework is described in detail, and its use is demonstrated with a hypothetical emergency response training scenario.


This proceeding is published as Ostrander, Alec, Stephen Gilbert, and Michael Dorneich. "Team Data Analysis Using FATE: Framework for Automated Team Evaluation." In Workshop Proceedings: Approaches and Challenges in Team Tutoring. Proceedings of the Approaches and Challenges in Team Tutoring Workshop held in conjunction with the 20th Artificial Intelligence in Education Conference (AIED 2019). Chicago, IL, USA, June 29, 2019. (Anne M. Sinatra and Jeanine A. DeFalco, eds.) (2019): 5-14. Posted with permission.

Creative Commons License

Creative Commons Attribution 4.0 License
This work is licensed under a Creative Commons Attribution 4.0 License.

Copyright Owner

The Authors



File Format



Article Location