A Speech-Based Model for Tracking the Progression of Activities in Extreme Action Teamwork

Swathi Jagannath, Neha Kamireddi, Katherine Ann Zellner, Randall S. Burd, Ivan Marsic, Aleksandra Sarcevic

Research output: Contribution to journalArticlepeer-review


Designing computerized approaches to support complex teamwork requires an understanding of how activity-related information is relayed among team members. In this paper, we focus on verbal communication and describe a speech-based model that we developed for tracking activity progression during time-critical teamwork. We situated our study in the emergency medical domain of trauma resuscitation and transcribed speech from 104 audio recordings of actual resuscitations. Using the transcripts, we first studied the nature of speech during 34 clinically relevant activities. From this analysis, we identified 11 communicative events across three different stages of activity performance-before, during, and after. For each activity, we created sequential ordering of the communicative events using the concept of narrative schemas. The final speech-based model emerged by extracting and aggregating generalized aspects of the 34 schemas. We evaluated the model performance by using 17 new transcripts and found that the model reliably recognized an activity stage in 98% of activity-related conversation instances. We conclude by discussing these results, their implications for designing computerized approaches that support complex teamwork, and their generalizability to other safety-critical domains.

Original languageEnglish (US)
Article number73
JournalProceedings of the ACM on Human-Computer Interaction
Issue numberCSCW1
StatePublished - Apr 7 2022

ASJC Scopus subject areas

  • Social Sciences (miscellaneous)
  • Human-Computer Interaction
  • Computer Networks and Communications


  • activity recognition
  • narrative schemas
  • speech modeling


Dive into the research topics of 'A Speech-Based Model for Tracking the Progression of Activities in Extreme Action Teamwork'. Together they form a unique fingerprint.

Cite this