AI & Music
Rhythmic Conversations
Jamming with audience-controlled, improvising AI-driven drummers.
Rhythmic Conversations is an audience-interactive performance with improvising AI drummers. Audience members play drum pads placed among the seats. AI-driven virtual percussionists "listen" and answer with new parts, and I improvise on guitar with all of them.
Motivation
Most performances are a one-way flow from performer to audience. AI-driven virtual musicians can let audiences actively take part and steer the music, even without expertise on a physical instrument. Each pad is audible only to the person playing it, so anyone can join without fear of being judged.
The system
A Python package bridges Ableton Live and the AI. Drum beats from the audience are ingested in real time and fed to neural network and Transformer models trained on Google Magenta’s Groove dataset, which generate complementary, stylistically coherent beats. Performers can then manipulate tempo and sound clips. The setup needs only a laptop, scales to any number of players and instruments, and can be extended in Max or Python.
Where it’s been performed
- Exhibitions: visitors drum with the AI or pick up another instrument and jam.
- Formal stage performance at Grusin Music Hall, CU Boulder, with the virtual drummer projected on screen.
- An algorave at St Mary’s Church, Brighton, UK, as part of AI Music Creativity 2023.
Video
Gallery
Publications
-
Rhythmic Conversations: Musical Performance with Improvising AI Drummers
AI Music Creativity Conference (AIMC), Brighton, UK, 2023.
Related projects
Stringesthesia
Dynamically shifting musical agency between audience and performer based on the performer’s neurally measured trust in an improvised performance.
Virtual AI Jam
AI-driven, animated virtual musicians for human-in-the-loop musical improvisation.
CrowdForge
Effects of audio and visual audience participation modalities on agency, collaboration, and engagement in AI-mediated mixed reality performances.