Torin Hopkins, PhD

Extended Reality · HCI & Interfaces

MR-DAW

Towards collaborative digital audio workstations in mixed reality: designing collaborative music interaction for networked XR environments.

Read the paper (PDF)DOI
(A) A violinist recording with a foot pedal; (B) the spatial track panel overlaid on a piano; (C) a second player at a remote keyboard.
(A) A violinist recording with a foot pedal; (B) the spatial track panel overlaid on a piano; (C) a second player at a remote keyboard.

MR-DAW is a shared recording studio in mixed reality. Musicians in different places control one digital audio workstation (DAW) while moving freely with their own instruments, looping together through physical foot pedals instead of a desk and a mouse.

Motivation

DAWs are central to modern music production, but they tether musicians to a desk and get in the way of natural interaction with their instrument. Remote collaboration is also still hard: existing tools are hampered by network latency and asynchronous file sharing. We asked whether mixed reality could remove these barriers and support real-time, remote musical collaboration.

The design probe

MR-DAW lets multiple, geographically dispersed musicians control a single shared DAW instance. Each player wears a Meta Quest 3 headset showing a spatial panel of eight loop tracks (red for recording, green for playback, blue for selection), session controls that mirror the foot pedal, and a debug window. A Python server controls Ableton Live, and each remote musician uses a physical foot pedal for hands-free collaborative looping.

We used qualitative and speculative design methods. Twenty musicians from Taiwan and the United States, working in 10 pairs, described how they use DAWs today in semi-structured interviews, used the system together, and imagined the collaborative studios they want next.

Key findings

  • Mixed reality affords unencumbered musical interaction: musicians stayed with their instruments rather than at a computer.
  • Familiar, hands-free foot-pedal looping merged naturally with a shared virtual session.
  • The study offers a speculative outlook on remote, collaborative DAWs in the Musical Metaverse.

Gallery

Each musician’s setup: headset, microphone, foot pedal, audio interface, and the Python/Ableton server.
Each musician’s setup: headset, microphone, foot pedal, audio interface, and the Python/Ableton server.
Real-time system architecture linking two Quest 3 players to a single Ableton Live session.
Real-time system architecture linking two Quest 3 players to a single Ableton Live session.

Publications

  1. MR-DAW: Towards Collaborative Digital Audio Workstations in Mixed Reality

    Torin Hopkins, Shih-Yu Ma, Suibi Che-Chuan Weng, Ming-Yuan Pai, Ellen Yi-Luen Do, and Luca Turchet

    IEEE International Symposium on the Internet of Sounds (IS2), L'Aquila, Italy, pp. 1–10, 2025.

    AbstractPDFDOI

IEEE IS2 2026 · Accepted

CrowdForge

Effects of audio and visual audience participation modalities on agency, collaboration, and engagement in AI-mediated mixed reality performances.

Project

IEEE AIxVR 2024

XR Jam

Design considerations for networked jamming with AI-driven musicians and remote players in extended reality.

ProjectPDFDOI

Frontiers in Virtual Reality · 2022

AR Drum Circle

Networked musical interaction and real-time collaboration between remote drummers in augmented reality.

ProjectPDFDOI