Conference · IEEE ISMAR 2022
Studying the Effects of Network Latency on Audio-Visual Perception During an AR Musical Task
IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Singapore, pp. 26–34, 2022.
Abstract
Augmented Reality (AR), with its ability to make people feel like they are in the same space as friends from across the world, is an ideal medium for the purpose of Networked Musical Collaboration. Most conventional systems that enable networked musical collaboration minimize network latency by focusing on the transfer of auditory information at the expense of visual feedback. Studies into human perception have shown that sensory integration of audio and visual stimuli can take place even when there is a slight delay between the two signals. We studied the way changes in network latency affect participants’ auditory and visual perception in latency detection, latency tolerance and attention focus; in this paper, we explore the trade-off between the presence of AR visuals and the minimization of latency. Twenty-four participants were asked to play a hand drum and collaborate with a prerecorded remote musician rendered as an avatar in AR. Multiple trials involving different levels of audio-visual latency were conducted. We then analyzed the subjective responses of the participants together with the recorded musical information from each session. Results indicate a minimum noticeable delay value—defined as the highest amount of delay that can be experienced before two stimulated senses are perceived as separate events—between 160 milliseconds (ms) and 320 ms. Players also reported that a delay between sound and an accompanying avatar animation became less tolerable at 320 ms of delay, but was never completely intolerable, even up to 1200 ms of delay. We conclude that players begin to notice delay at about 320 ms and most players can tolerate large delays between sound and animation.
Cite
Torin Hopkins, Suibi Che-Chuan Weng, Rishi Vanukuru, Emma Wenzel, Amy Banić, Mark D. Gross, Ellen Yi-Luen Do. 2022. Studying the Effects of Network Latency on Audio-Visual Perception During an AR Musical Task. IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 26–34. https://doi.org/10.1109/ISMAR55827.2022.00016
@inproceedings{hopkins2022studying,
title = {{Studying the Effects of Network Latency on Audio-Visual Perception During an AR Musical Task}},
author = {Hopkins, Torin and Weng, Suibi Che-Chuan and Vanukuru, Rishi and Wenzel, Emma and Banić, Amy and Gross, Mark D. and Do, Ellen Yi-Luen},
booktitle = {IEEE International Symposium on Mixed and Augmented Reality (ISMAR)},
pages = {26--34},
year = {2022},
doi = {10.1109/ISMAR55827.2022.00016}
}