AI & Music · Extended Reality
Augmented Personification of Intelligent Music Tools
Personifying loopers and generative music tools as humanoid avatars in augmented reality, for creativity and collaboration.
Intelligent music tools are often designed to replicate human creativity, yet we interact with them through screens and foot pedals. This workshop paper proposes personifying these tools as humanoid avatars in augmented reality, so that a looper or a generative model becomes a visible collaborator in the room.
Two prototypes
- Live looping: a foot pedal records and plays back loops as usual, but each loop is also performed by a virtual avatar in front of the musician. As more loops are added, more avatars appear, almost recreating the feeling of a drum circle.
- Generative music: AI virtual musicians respond to the player’s MIDI input through Max, Python, and Magenta neural networks, and perform their parts as animated avatars.
What’s next
We planned user tests on how musicians’ use of these tools changes when the virtual musicians are present, and more instruments such as keyboards and guitars. The work fed directly into the AI-driven virtual musicians of my later research.
Publications
-
Augmented Personification of Intelligent Music Tools for Creativity and Collaboration
ACM CHI Conference on Human Factors in Computing Systems (Workshop), New Orleans, LA, USA, 2022.
Related projects
Virtual AI Jam
AI-driven, animated virtual musicians for human-in-the-loop musical improvisation.
Computer-Supported Musical Improvisation
Doctoral research framework presented at the NIME Doctoral Consortium.