Training a 125M-Parameter Model to Autocomplete Piano

Simon Edwardsson spent close to a year on one question: connect a MIDI piano to a phone, play a few bars, and have a model carry on. The write-up walks through fourteen experiments and lands on a 125M-parameter transformer that keeps up in real time, around 108 notes per second on an iPhone 15. What moved the needle was not scale. Finding the right way to tokenize MIDI mattered most. A naive note-on token carrying both pitch and velocity pushes the vocabulary past 16,000 entries, while factoring the representation into a grammar shrinks it. Aggressive cleaning of the training data and a round of DPO post-training did the rest. Sound samples for Pallet Town, Terra's Theme and Für Elise are embedded in the post, and the resulting iPhone and iPad app, RollTab, is free.