A thirty-second music sidebar

If you don’t read music, here is everything this post needs. LilyPond is the plain-text notation format behind the Mutopia Project’s public-domain scores. Pitches are lowercase letters: c, e, g. Angle brackets group notes that sound at the same time, so <c e g> is the three notes C, E and G struck together, which is a C major chord, the first chord anyone learns. A number after a note or chord is its written length: 2 is a half note, 4 a quarter, and a trailing dot adds half again, so <c e g>2. is a dotted half-note chord. Doubled brackets, << … >>, mean something different: two independent voices playing at once, each with its own rhythm, which is how a piano score writes the left hand against the right.

I took piano lessons as a kid and played recitals, so I can read a score. That turned out to be exactly enough knowledge to believe this format would be easy to parse, and not enough to know why it isn’t.

What does LilyPond actually write?

A chord is one unit: an opening bracket, some pitches, a closing bracket, and the duration hangs off the close. The tokenizer in Étude produces exactly that shape, and the unit test says so in five lines:

#expect(try sut.tokenize("<c e g>") == [
    .chordStart,
    .note(NoteToken(name: "c")),
    .note(NoteToken(name: "e")),
    .note(NoteToken(name: "g")),
    .chordEnd(duration: nil),
])

Nothing clever. The point of this post is how the prototype got something this simple wrong, and what the fix teaches about tokenizers in general.

What did the prototype do with it?

The Python prototype that proved the pipeline on six pieces before the Swift rewrite tokenized like most quick scripts do. It split the text on whitespace, then noticed that chords had been torn apart and tried to glue them back together:

raw = text.split(); toks = []; i = 0
while i < len(raw):
    t = raw[i]
    if t.startswith('<') and '>' not in t:
        while '>' not in t and i+1 < len(raw): i += 1; t += ' ' + raw[i]
    toks.append(t); i += 1

Read the fourth line again. A token that starts with < and has no > in it must be the front of a chord, so keep swallowing words until one contains a >. For <c e g> that works: <c, then e, then g>, glued back into one string, which a regex later splits on spaces a second time to find the pitches.

The symptom in the bug catalog is blunt: chords lost notes or corrupted whatever followed. Structure was an afterthought bolted onto whitespace.

Why is fixing the repair loop the wrong answer?

Because the assumption fails again at the very next construct, one layer up. Satie’s first Gnossienne contains this fragment, verbatim:

<<af2 \new Voice{\voiceOne \once \hideNotes af4 }>> af4

That is a parallel-music opener glued straight onto a pitch. No space between << and af2. Feed it to the repair loop: the token <<af2 starts with < and contains no >, so the loop starts gluing. It swallows \new, Voice{\voiceOne, \once, \hideNotes, af4, and only stops at }>>, the bracket that closes the whole passage. An entire voice has just become one giant “chord” token, and everything downstream of it is wrong.

That is bug 005 in the catalog, and it is the same bug as 001 wearing a different hat. You could special-case <<. Then you would special-case the next thing. If a construct can contain the delimiter you split on, splitting was the wrong first move.

What does structural tokenizing look like?

It looks like reading the characters in order and letting the grammar decide. The Swift tokenizer treats < and > as tokens in their own right, and the pitches between them tokenize exactly like pitches anywhere else:

if c == "<" {
    // `<<` opens simultaneous music; a single `<` opens a chord.
    if i + 1 < chars.count, chars[i + 1] == "<", !inChord {
        tokens.append(.parallelStart)
        i += 2
    } else {
        tokens.append(.chordStart)
        inChord = true
        i += 1
    }
    continue
}

Whitespace never participates. <<af2 is a parallel opener followed by a pitch, because that is what the characters say, whether or not there is a space between them.

This landed as three commits on the same day, and their subject lines are the changelog:

09d0891 Tokenizes chords structurally, never by whitespace split (BUG-001)
51608b1 Tokenizes the chord-repeat mark with an optional duration
958b2f5 Distinguishes parallel markers from chord delimiters

The first one added twelve lines to the tokenizer and two test files. Run git log --oneline --reverse in the repo and they are lines 14 through 16. That is the “replayable history” claim from the launch post, in the small.

How do you keep the bug from coming back?

With a regression test that carries its own story. Here is the one for bug 001, lightly trimmed:

/// **BUG-001 — Chord tokens split on whitespace.**
///
/// *Symptom:* in the prototype, `<c e g>` came out as garbage ...
/// *Root cause:* whitespace is not a token boundary inside `<…>` ...
/// *Guard:* the tokenizer scans structurally ...
@Suite("BUG-001: chords tokenize structurally, never by whitespace split")
struct BUG001_ChordTokensSplitOnWhitespace {
    @Test("chord with every duration and ornament suffix", .tags(.regression), arguments: [
        ("<fis d b>2", DurationToken(2), [Token]()),
        ("<c e g>", nil, []),
        ("<d a fis d>2.", DurationToken(2, dots: 1), []),
        ("<d a>4", DurationToken(4), []),
        ("<c e g>2~", DurationToken(2), [.tie]),
        ("<c e g>4(", DurationToken(4), [.slurOpen]),
    ] as [(String, DurationToken?, [Token])])
    func chordSuffixes(source: String, duration: DurationToken?, trailing: [Token]) throws {
        let sut = makeSUT()
        let tokens = try sut.tokenize(source)
        #expect(tokens.first == .chordStart)
        #expect(tokens.dropLast(trailing.count).last == .chordEnd(duration: duration))
        #expect(Array(tokens.suffix(trailing.count)) == trailing)
        // Every token between the delimiters is a pitch — nothing was torn apart.
        let inner = tokens.dropFirst().prefix { $0 != .chordEnd(duration: duration) }
        #expect(inner.allSatisfy { if case .note = $0 { true } else { false } })
        #expect(!inner.isEmpty)
    }
}

Three things to steal from it.

The doc comment is the bug report. Symptom, root cause, guard. Someone who opens this file in two years learns what went wrong without running git blame. The same three headings appear in docs/lessons/bug-001-chords-split-on-whitespace.md, and every lesson in that folder is also a suite like this one. Seven lessons, five with their own regression suite, two more guarded in the acceptance layer.

The arguments are the corpus, not my imagination. The shapes in that list, the tie, the slur, the dotted half, are the ones the shipped pieces use. The test pins what the music actually writes, not what I imagined it might.

The last assertion names the bug. “Every token between the delimiters is a pitch, nothing was torn apart.” Compare that with a test called testChords2 that checks one expected array. Both pass today. Only one of them tells the next person what it is protecting.

Why test with the exact hostile fragment?

Bug 005’s regression test does not use a tidy example. It tokenizes the Gnossienne line above, backslashes and all, and asserts the full token list, fourteen tokens long. The lesson written at the top of that file is the one I would put on the wall: synthetic examples are tidy; real sources glue things together. A parser test built from a real file will catch the bug that a parser test built from your mental model of the format never will.

That principle scales up. It is the reason the acceptance layer builds whole pieces from their real Mutopia sources, and it is the reason the Clair de Lune story exists at all. That one is two posts of its own, and we are getting there.

Read, then try it

The repo is public: github.com/lismondbernard/etude.

Read Tests/EtudeKitTests/Regression/BUG001_ChordTokensSplitOnWhitespace.swiftfirst, then the lesson it points to in docs/lessons/.

Try it yourself: add ("<c e g>8.", DurationToken(8, dots: 1), []) to the argument list, a dotted eighth, and run the suite. It passes without touching the tokenizer, which is the point. Then find one construct in a parser you own that can contain the delimiter you split on, and write this test for it.

Next post: stop 1 of the course map. A unit suite worth imitating: makeSUT(), sample builders, and one musical rule per test.

It’s called Étude: an open-source iOS app that turns classical sheet music into MIDI files you can play, retempo, and export to GarageBand or a game engine. Under the app is a small Swift engine that parses LilyPond, the plain-text notation format the Mutopia Project uses to typeset public-domain scores, then validates the music and emits standard Type-1 MIDI. Seven pieces ship in the corpus: Bach, Petzold, Vivaldi, two Satie, and Debussy’s Clair de Lune, which has a story I’ll get to.

That’s the app. But the app isn’t really the point.

What this actually is

Étude is a piece of working software and a course in how to test software, shipped as the same repository. Clone it, run git log --oneline --reverse, and the history replays as the course: one behavior per commit, each commit a red→green→refactor step with a subject line that names the behavior. “Resolves relative octaves within a fourth of the previous note,” not “Update Resolver.swift.”

I borrowed this discipline from the Essential Developer curriculum I worked through years ago. The codebase you build there has a git history that reads like a TDD transcript, and Étude’s history now reads the same way. I mean that as a checkable claim, not a boast: the git history is a deliverable, and it publishes exactly as it was written. No squashing, no cleanup rebase. There’s even an architecture decision record in the repo (ADR-0005) explaining why, including the three duplicated commit subjects that are still sitting in the log, documented as findings rather than laundered away. A testing course that quietly edited its own history would be teaching the wrong lesson.

So the tests aren’t scaffolding that got deleted before shipping. The tests are the product, as much as the app is: unit tests, hand-rolled property-based tests, golden-file comparisons, invariant validators, regression tests reconstructed from real bugs, and UI tests with page objects. Each one earned its place by catching something concrete, including a UI test that caught a real observability bug the unit tests couldn’t see, on the day it was written.

Where to start reading

The README has a course map: eight stops, each naming the one test file that best teaches a technique, in reading order. The phases of the build are tagged (phase-1 through phase-6), so you can check out the repo as it stood at the end of any lesson. The docs/lessons/ folder holds seven write-ups of real bugs (symptom, root cause, the guard now in place), and every one of them is also a named regression test in the suite.

The piece that beat the prototype, and what closed it

None of this was speculative. A Python prototype proved the pipeline on six pieces before the Swift rewrite. Getting there meant fixing a catalog of genuinely instructive bugs: chord tokens split on whitespace into nonsense, a bass line that spiraled into sub-audible octaves because relative pitch got threaded through a repeat, register drift that a defensive “just clamp it” fix quietly hid instead of surfacing.

And then there’s Debussy. Clair de Lune defeated the prototype outright. Its dense polyphony produced voices of 91, 46, 57, and 54 beats where they should have been equal. The Swift engine solved that alignment problem, but a residual defect remained: part of the left hand drifted below the piano’s range. Here’s the part I care about. Rather than fake a fix, v0.1.0 shipped with that defect loud: a permanently visible known-issue test, a public GitHub issue, and the app’s own diagnostics screen showing the drift honestly. That’s the project’s one non-negotiable rule at work: never ship subtly wrong music. A correct excerpt beats an incorrect whole.

The issue is closed now, and the closing is the best lesson in the repo. Recovering the original Mutopia source of the score exposed two root causes: a hand-”correction” to the source that was itself wrong, and a resolver bug that the Swift engine had faithfully inherited from the prototype. Both engines agreed with each other, and both were wrong, which is exactly why agreement with your own prototype is worthless as evidence. Only an independent oracle, LilyPond’s own rendering of the original, broke the loop. That story (and the two other pieces it quietly corrected on the way) gets its own post; it’s the reason this series exists.

What happens next

Development continues in the open. Two issues are live right now. The first: in-app playback is silent on real devices, a lovely bug where every automated test passes because the test seam faked the hardware, and only a human with ears could catch it. The second: a feature to browse and download scores straight from Mutopia, which is where the parser meets the real world’s LilyPond instead of the corpus’s. The app is live on the App Store, because a real product raises the stakes: it has to actually work, not just demo well.

The code is public under Apache-2.0 so it can be read, learned from, and contributed to; the name and icon stay mine as the practical guard against clone submissions. If you contribute, the same commit discipline applies. CONTRIBUTING.md spells it out, and it’s enforced on human and AI-assisted work alike. (Étude was built with heavy use of Claude Code; the method, the architecture, and every recorded decision are mine, and the unedited history is the evidence of how the discipline held.)

Follow along

A word on who’s writing: I run Silent Spice, a one-person studio shipping iOS apps: a picture-matching vocabulary game in eight languages, an app for junior-tennis parents. Étude is me showing the working method behind those apps instead of just asserting it. I apply the same method when I audit other people’s iOS apps.

If you write software and testing has always felt like a chore bolted on afterward, this series is for you. Over the coming posts I’ll walk the course map stop by stop: real code, real test failures, and the decisions I got wrong along the way, which are still in the log because that’s the point. If you just want to turn a Satie piece into a MIDI file and drop it into your game, the app is for you too.

The repo is public: github.com/lismondbernard/etude. Start with the README’s course map, or just run git log --oneline --reverse and scroll.

— Next post: the tokenizer, and why <c e g> is harder than it looks.