Torin Hopkins, PhD

Conference · IEEE AIxVR 2024

XR Jam: Design Considerations for Music Networking with AI in XR

Torin Hopkins, Suibi Che-Chuan Weng, Rishi Vanukuru, Sasha Novack, Chad Tobin, Emma Wenzel, Amy Banić, Mark D. Gross, and Ellen Yi-Luen Do

IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR), Los Angeles, CA, USA, pp. 97–101, 2024.

Read PDF DOI: 10.1109/AIxVR59861.2024.00021
Figure from “XR Jam: Design Considerations for Music Networking with AI in XR”

Abstract

The XR Jam project presents an integrated system designed to enhance musical interaction and spatial awareness in Extended Reality (XR) environments. Leveraging AI-driven Virtual Musical Agents (AIVMs), the platform elevates traditional musical experiences. Built on an XR framework, the system pushes conventional boundaries by enabling novel forms of interaction, such as object manipulation and social drumming with AIVMs and remote players. Furthermore, avatar-based interfaces help overcome social inhibitions, offering players a unique, uninhibited interactive experience. Pilot studies revealed the possibility for effective use of haptics in spatial localization. While haptic feedback enhanced spatial orientation in specific scenarios, front and back directions remained challenging to discern. The system promotes musical interaction, social engagement, and spatial orientation, although further refinements are needed for synchronization and player experience. Overall, this research opens new avenues for immersive musical experiences through a fusion of AI, XR, and haptic technologies.

  • extended reality
  • AI-driven virtual musicians
  • haptics
  • networked music

Cite

Torin Hopkins, Suibi Che-Chuan Weng, Rishi Vanukuru, Sasha Novack, Chad Tobin, Emma Wenzel, Amy Banić, Mark D. Gross, Ellen Yi-Luen Do. 2024. XR Jam: Design Considerations for Music Networking with AI in XR. IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR), 97–101. https://doi.org/10.1109/AIxVR59861.2024.00021

@inproceedings{hopkins2024design,
  title = {{XR Jam: Design Considerations for Music Networking with AI in XR}},
  author = {Hopkins, Torin and Weng, Suibi Che-Chuan and Vanukuru, Rishi and Novack, Sasha and Tobin, Chad and Wenzel, Emma and Banić, Amy and Gross, Mark D. and Do, Ellen Yi-Luen},
  booktitle = {IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR)},
  pages = {97--101},
  year = {2024},
  doi = {10.1109/AIxVR59861.2024.00021}
}

Project: Networking AI-Driven Virtual Musicians in XRProject: XR JamAll publications