TY - JOUR
T1 - A Speech-Based Model for Tracking the Progression of Activities in Extreme Action Teamwork
AU - Jagannath, Swathi
AU - Kamireddi, Neha
AU - Zellner, Katherine Ann
AU - Burd, Randall S.
AU - Marsic, Ivan
AU - Sarcevic, Aleksandra
N1 - Funding Information: This work is supported by the U.S. National Science Foundation, under grant IIS-1763509. Author’s addresses: S. Jagannath, N. Kamireddi, K. A. Zellner, and A. Sarcevic, College of Computing and Informatics, Drexel University, 3675 Market St., Philadelphia, PA 19104, USA; R. S. Burd, Trauma and Burn Surgery, Children’s National Hospital, 111 Michigan Avenue NW, Washington, DC, 20010, USA; I. Marsic, Electircal and Computer Engineering, Rutgers University, 94 Brett Road, Piscataway, NJ 08854, USA. Publisher Copyright: © 2022 ACM.
PY - 2022/4/7
Y1 - 2022/4/7
N2 - Designing computerized approaches to support complex teamwork requires an understanding of how activity-related information is relayed among team members. In this paper, we focus on verbal communication and describe a speech-based model that we developed for tracking activity progression during time-critical teamwork. We situated our study in the emergency medical domain of trauma resuscitation and transcribed speech from 104 audio recordings of actual resuscitations. Using the transcripts, we first studied the nature of speech during 34 clinically relevant activities. From this analysis, we identified 11 communicative events across three different stages of activity performance-before, during, and after. For each activity, we created sequential ordering of the communicative events using the concept of narrative schemas. The final speech-based model emerged by extracting and aggregating generalized aspects of the 34 schemas. We evaluated the model performance by using 17 new transcripts and found that the model reliably recognized an activity stage in 98% of activity-related conversation instances. We conclude by discussing these results, their implications for designing computerized approaches that support complex teamwork, and their generalizability to other safety-critical domains.
AB - Designing computerized approaches to support complex teamwork requires an understanding of how activity-related information is relayed among team members. In this paper, we focus on verbal communication and describe a speech-based model that we developed for tracking activity progression during time-critical teamwork. We situated our study in the emergency medical domain of trauma resuscitation and transcribed speech from 104 audio recordings of actual resuscitations. Using the transcripts, we first studied the nature of speech during 34 clinically relevant activities. From this analysis, we identified 11 communicative events across three different stages of activity performance-before, during, and after. For each activity, we created sequential ordering of the communicative events using the concept of narrative schemas. The final speech-based model emerged by extracting and aggregating generalized aspects of the 34 schemas. We evaluated the model performance by using 17 new transcripts and found that the model reliably recognized an activity stage in 98% of activity-related conversation instances. We conclude by discussing these results, their implications for designing computerized approaches that support complex teamwork, and their generalizability to other safety-critical domains.
KW - activity recognition
KW - narrative schemas
KW - speech modeling
UR - http://www.scopus.com/inward/record.url?scp=85128434622&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85128434622&partnerID=8YFLogxK
U2 - https://doi.org/10.1145/3512920
DO - https://doi.org/10.1145/3512920
M3 - Article
SN - 2573-0142
VL - 6
JO - Proceedings of the ACM on Human-Computer Interaction
JF - Proceedings of the ACM on Human-Computer Interaction
IS - CSCW1
M1 - 73
ER -