Building a vocal arranger with AI

Using Codex to turn a lead sheet and musical instructions into a five-part a cappella arrangement.

I sing in an a cappella group, and I wanted to arrange a song for us. That was the inspiration for this project. I wondered whether Codex could help me turn a lead sheet and some plain-English instructions into a five-part a cappella arrangement. I tried it with Paul Simon’s “Diamonds on the Soles of Her Shoes,” and I was pleasantly surprised by what came back.

A lead sheet supplies a song’s melody, chords and lyrics. I wanted the software to develop that foundation into parts for soprano, alto, tenor, baritone and bass. I specified eleven singers sharing those five parts: one soprano, four altos, and two each on tenor, baritone and bass. I asked for intermediate difficulty, ten different arrangements, and three finalists with printable scores, editable notation, rehearsal files and simulated singing.

I worked with Codex to build the software, giving it musical goals and letting it choose the detailed parameters. After the first batch, I asked for one revision: more rhythmic variety and syncopation, occasional backing responses using words from the song, and more sharing of complete lead phrases among the voices. I wanted the backing parts to have something interesting to do, too.

My workflow had four steps:

  1. Read the music. The software turned the lead sheet into a precise list of notes, durations, words and chords that it could work with.
  2. Give the voices different jobs. Codex wrote rules for bass movement, harmony, accompaniment rhythms, lead handoffs and changes in texture. It developed the short supplied edition into arrangements a little over three minutes long, with introductions, repeated sections and endings.
  3. Check and compare. Software checked things such as whether a bar added up correctly, a part stayed within its assumed range, or the original words and melody were preserved. Separate AI reviewers assessed the ten candidates for source fidelity, harmony, rhythm, singability and musical coherence.
  4. Make the results usable. The project exported scores and rehearsal tracks, then used the Sinsy singing synthesizer to make vocal demonstrations from the written parts.

What impressed me was having several complete, editable arrangements to work with after that one revision. The three finalists were “Velvet Conversation,” “Street-Corner Replies” and “Dancing Counterlines.” I could see my requests showing up in the music. I’d still want musicians to try the parts and tell me what works; an AI review score only goes so far.

The Sinsy sung demos sounded hilarious. I noticed a bug where the bass, baritone and tenor weren’t singing the lyrics. The alto and soprano did sing them, but with what sounded to me like German accents. It was a very funny way to preview the music.

Here’s “Velvet Conversation,” one of the completed arrangements, along with its matching sung demo. You can listen and follow the score below.

Velvet Conversation — Sinsy sung demo

Open or download the sung demo (3:07).

Diamonds on the Soles of Her Shoes — Velvet Conversation
Complete five-part arrangement
Velvet Conversation score, page 1 of 14

Page 1 of 14. Open the pages below to follow the complete arrangement.

View score pages 2–14

Page 2 of 14

Velvet Conversation score, page 2 of 14

Page 3 of 14

Velvet Conversation score, page 3 of 14

Page 4 of 14

Velvet Conversation score, page 4 of 14

Page 5 of 14

Velvet Conversation score, page 5 of 14

Page 6 of 14

Velvet Conversation score, page 6 of 14

Page 7 of 14

Velvet Conversation score, page 7 of 14

Page 8 of 14

Velvet Conversation score, page 8 of 14

Page 9 of 14

Velvet Conversation score, page 9 of 14

Page 10 of 14

Velvet Conversation score, page 10 of 14

Page 11 of 14

Velvet Conversation score, page 11 of 14

Page 12 of 14

Velvet Conversation score, page 12 of 14

Page 13 of 14

Velvet Conversation score, page 13 of 14

Page 14 of 14

Velvet Conversation score, page 14 of 14

Open or download the complete score.

The harder problem appeared when I tried starting with a recording instead of a known-good lead sheet. A waveform or MP3 contains overlapping sounds; producing readable music requires identifying the melody and chords, finding the beat and bar lines, and deciding how performed timing should be written. A singer’s slight delay, a swung pair of notes and a deliberate offbeat entrance need different interpretations. A transcription can capture plausible pitches yet lose the rhythm and feel that make the song recognizable to a musician. That helped me understand the limitations described by the tools themselves: Spotify’s Basic Pitch says it works best on one instrument at a time, while Sheet Sage’s documentation describes tempo and downbeat errors that can distort the notation. In my experiments, getting a trustworthy, intuitive lead sheet remained the weak link.

I came away impressed by how far I could get with Codex, a good lead sheet and a little feedback. I also came away with a much greater appreciation for the work involved in writing that lead sheet in the first place. I’ve released the code on GitHub, including an original demo that others can try. The code repository is separate from the “Diamonds” score and sung example included here.