AI & Music
Automatic Arrangement
A unified framework and neural network architecture that enables automatic multitrack music arrangement.
Music arrangement adapts a piece for a new set of instruments, such as rewriting it for a band, reducing it for solo piano, or adding a drum part. We built a single framework that lets one pre-trained symbolic music model handle all of these scenarios, outperforming task-specific state-of-the-art models.
The problem
Arrangement takes many forms: reinterpretation for different instruments, simplification for solo performance, and additive generation of new tracks. They share a common structure, generating new tracks conditioned on existing ones under content and instrument constraints. But prior work treated each task separately, with specialized architectures that lack cross-task generality and can’t draw on the musical knowledge in large pre-trained symbolic models.
Our approach
At the core is a segment-level reconstruction objective over token-level disentangled content and style. This allows flexible any-to-any instrumentation transformations at inference time, without parallel training data, so a single generative model adapts to each task through lightweight fine-tuning.
To support track-wise modeling we introduced REMI-z, a structured tokenization scheme for multitrack symbolic music. It produces shorter, lower-complexity sequences, enables instrument-level control, and reduces note-level perplexity in unconditional generation as well.
Key findings
- Outperforms task-specific state-of-the-art models on band arrangement, piano reduction, and drum arrangement, in both objective metrics and perceptual evaluations.
- Achieves this with a small model and modest-scale pre-training.
- REMI-z later powered the generative arranger in the CrowdForge performance.
Publications
-
Unifying Symbolic Music Arrangement: Track-Aware Reconstruction and Structured Tokenization
Advances in Neural Information Processing Systems (NeurIPS), San Diego, CA, USA, 2025.
