Apple's Music Understanding in a DJ app: BPM, key, energy, and vocals on macOS 27

macOS 27 ships Music Understanding, a framework that analyses music on the device, offline. Bake'n Deck 1.4.0, released on 2026-10-30, uses it for two jobs: a check of rekordbox's BPM and key that marks disagreements without writing anything, and an energy and vocal tag such as E7V written onto the USB stick, where a DJ sees it on the player. This page is the implementation record: the Swift code, what the framework agreed with on a real DJ library, how raw results became decisions, and what went wrong on the way. Everything was measured on 2026-10-28 with macOS 27.0.1 on a MacBook Air M3, against one DJ's rekordbox library of 1,397 tracks (MP3, AAC, FLAC, and AIFF).

The framework in one paragraph

You hand a MusicUnderstandingSession an AVAsset or a stream of PCM buffers, and analyze(for:) returns whichever you ask for: key (tonic and mode per time range), rhythm (beat and bar times, plus an optional BPM), pace, structure (sections, segments, and phrases), instrument activity (vocal, drums, bass, and other, as time ranges), and loudness. Apple's documentation and WWDC26 session 253 cover the API. Two things are worth knowing before you design around it:

No result carries a confidence value. Whatever you build has to decide what to do with a wrong answer without being told which answers are shaky.
Pace is not 0 to 1. It is easy to read it as a normalised energy; the documentation for PaceResult calls it "a perceptual events-per-minute rate", independent of tempo. On this library the per-track mean ran from 4.8 to 36.1, so any 1 to 10 scale is yours to define.

What the app does with it, and what it does not

Check BPM and Key. In the Playlist steps, every track of the playlist goes through the framework and a macOS column appears next to rekordbox's key and BPM. Disagreements that matter for mixing are marked, with the reason in a tooltip. Nothing is written: neither side was reliably right (below), so the DJ checks the marked tracks by ear.
Energy and vocal tags on the stick. USB Export can put E7V (energy 7, vocals) or E7 at the start of each track's comment or title, on the stick only. The collection XML and rekordbox's own library are never changed. The comment shows on the player's INFO screen and the title in every list, so the tag is there in the booth, where there is no Mac.
Not used for gain. Headroom keeps its own BS.1770-4 meter, and the framework documents its peak only as "dB", not as true peak.
Not built: filling in missing keys, or cue points at section boundaries. With 22 of 100 keys far from rekordbox's, writing either into a library without a person checking them is not something the app should do, and placing cues is a decision that belongs to the DJ.
Earlier macOS. The deployment target stays at macOS 14. The framework is weak-linked automatically (Xcode emitted LC_LOAD_WEAK_DYLIB with no flag), every use sits behind @available(macOS 27, *), and on older systems the button, the column, and the tag option do not appear.

The code

The whole integration is about 250 lines of Swift in one file. These are the parts that carry the design. HeardTrack is a small Codable struct (key index, BPM, pace, and vocal share) that is cached per file, keyed by path and checked against size and mtime, and saved every 25 tracks so a long run that is quit keeps most of its work.

Swift: one session, four analyses

import AVFoundation

import MusicUnderstanding

@available(macOS 27, *)

static func listen(to path: String) async throws -> HeardTrack {

    let reader = try PCMReader(URL(fileURLWithPath: path))

    // One session for all four: cheaper than two passes.

    let result = try await MusicUnderstandingSession(audioProvider: AsyncStream { reader.next() })

        .analyze(for: [.key, .rhythm, .pace, .instrumentActivity])

    // A track can change key; the one held longest stands for it.

    var held: [KeyResult.KeySignature: Double] = [:]

    for range in result.key?.ranges ?? [] {

        held[range.value, default: 0] += range.range.duration.seconds

}

    let key = held.max { $0.value < $1.value }?.key

    // Pace: the mean over the track, weighted by each range's duration.

    let paces = result.pace?.ranges ?? []

    let paced = paces.reduce(0) { $0 + $1.range.duration.seconds }

    // Vocal: the share of the track inside the vocal ranges.

    let vocals = result.instrumentActivity.map { activity in

        (activity.ranges[.vocal] ?? []).reduce(0) { $0 + $1.duration.seconds }

}

    let duration = reader.seconds

    return HeardTrack(

        keyIndex: key.map(keyIndex),

        bpm: result.rhythm?.beatsPerMinute.map(Double.init).flatMap { $0 > 0 ? $0 : nil },

        pace: paced > 0 ? paces.reduce(0) { $0 + $1.value * $1.range.duration.seconds } / paced : nil,

        vocalShare: vocals.flatMap { duration > 0 ? min(1, $0 / duration) : nil }

)

}

The session reads PCM that the app decodes itself. The first version passed an AVAsset. With four sessions at once, the sandboxed app crashed once inside AudioToolbox (AudioQueueXPC_Bridge, a dispatch_sync onto its own queue, under AVAssetReader's offline mixer). The unsandboxed command-line probe never did. AVAudioFile decodes without an audio queue; on 14 tracks across every format the results were identical to the asset's, and about 290 tracks since have run without a crash.

Swift: PCM from AVAudioFile instead of an AVAsset

@available(macOS 27, *)

private final class PCMReader: @unchecked Sendable {

    private let file: AVAudioFile

    init(_ url: URL) throws {

        file = try AVAudioFile(forReading: url)

}

    var seconds: Double {

        Double(file.length) / file.processingFormat.sampleRate

}

    /// nil at the end, or where a damaged file stops decoding.

    func next() -> AVReadOnlyAudioPCMBuffer? {

        guard let buffer = AVAudioPCMBuffer(pcmFormat: file.processingFormat, frameCapacity: 65_536),

              (try? file.read(into: buffer)) != nil, buffer.frameLength > 0

        else { return nil }

        return AVReadOnlyAudioPCMBuffer(copying: buffer)

}

}

Four sessions at a time, no more.

Swift: four sessions in flight

static let width = 4   // six were no faster; eight once stalled

await withTaskGroup(of: (String, HeardTrack?).self) { group in

    var queue = pending.makeIterator()

    func next() {

        guard !cancel.isCancelled(), let path = queue.next() else { return }

        group.addTask { (path, try? await listen(to: path)) }

}

    for _ in 1...width { next() }

    while let (path, heard) = await group.next() {

        if let heard { found[path] = heard }   // and into the cache

        next()

}

}

BPM and key against rekordbox

100 random tracks from the library, analyze(for: [.key, .rhythm]), compared with rekordbox's AverageBpm and Tonality in the exported collection XML. There is no ground truth here, so this measures agreement, not accuracy.

BPM, macOS against rekordbox	Tracks
Within 0.5 BPM	70
Within 2 BPM	12
Half or double	9
Off by more than 2 BPM, not half or double	9

In every half or double case it was the framework hearing half time: 169.30 against 84.66, 160 against 80, 140 against 70, with rekordbox holding the value a DJ would expect. The other misses include 125 against 117.9, 92.0 against 70.0, and 144 against 48. A BPM of 0 is treated as no answer.

Key, macOS against rekordbox	Tracks	Flagged
Same key	61	no
Relative key (same number, A and B swapped)	11	no
One step on the wheel (same letter)	5	no
Further apart	22	yes
No key returned	1	no

Every track so far came back with a single key range, but the code still takes the key held longest in case one does not. 22 of 100 is too many to overwrite and too few to ignore, which is why the app marks and never writes. A BPM is marked when it is half or double rekordbox's (within 6%) or more than 2 BPM away: 18 of the 100.

Swift: tonic and mode to the Alphanumeric wheel

// 1A = 0, 1B = 1, ..., 12B = 23. The number follows the relative major;

// C major and A minor are 8.

static func keyIndex(_ key: KeyResult.KeySignature) -> Int {

    let minor = key.mode == .minor

    let major = (pitchClass(key.tonic) + (minor ? 3 : 0)) % 12

    return (major * 7 + 7) % 12 * 2 + (minor ? 0 : 1)

}

// Flag only keys further apart than the relative key (same number)

// or a one-step neighbour (same letter): those still mix.

let apart = abs(ours / 2 - theirs / 2)

let steps = min(apart, 12 - apart)

let keyAgrees = steps == 0 || (steps == 1 && ours % 2 == theirs % 2)

Pace as energy

172 tracks in reference sets: whole playlists the DJ already built, and tracks the DJ had tagged by floor in rekordbox's My Tag. analyze(for: [.pace, .instrumentActivity]), pace averaged over each track by duration.

Group	Tracks	Median pace	Q1 to Q3	Energy at the median
chill playlist	25	19.4	17.6 to 24.1	4
vocal pop playlist	25	18.9	15.4 to 23.7	4
house and trance playlist	25	23.9	20.1 to 25.6	5
tagged second floor	7	25.3	20.2 to 27.1	6
tagged main floor	17	27.7	23.5 to 32.2	6
hard techno playlist	25	32.5	29.2 to 34.3	8

A hard techno track out-paces a chill one 90% of the time (AUC 0.90); house out-paces chill 69% of the time, and main floor beats second floor 71% of the time (small sets).
The Spearman correlation with rekordbox's BPM is 0.47, so pace is not tempo in disguise. The lowest values were intros and ballads, the highest 140 to 146 BPM techno.
The scale is fixed rather than relative to the library, so a track keeps its number as the library grows: chill lands mostly at 3 to 5, house at 4 to 6, and hard techno at 7 to 9.

Swift: energy and the tag

// Fixed scale: pace 10 -> 1, 40 -> 10, clamped. A track keeps its

// number as the library grows; a scale relative to the library would not.

var energy: Int? {

    pace.map { min(10, max(1, Int((($0 - 10) / 30 * 9).rounded()) + 1)) }

}

var vocal: Bool? { vocalShare.map { $0 >= 0.15 } }

// "E7V", or "E7" without vocals: short ASCII that cannot be read as a key.

var tag: String? { energy.map { "E" + String($0) + (vocal == true ? "V" : "") } }

Vocals from instrument activity

The measure is the share of the track's duration inside the vocal ranges. The reference sets came from the DJ's own tags and titles.

Reference set	Tracks	Median vocal share	Range
tagged Vocal by the DJ	34	26.4%	1.0 to 88.3%
vocal pop playlist	25	71.6%	15.7 to 87.0%
tagged by component, without Vocal	35	0.0%	0 to 14.7%
karaoke versions (by title)	7	0.0%	0 to 2.6%
instrumental versions (by title)	4	0.0%	0 to 0%

AUC 0.98 between the vocal and instrumental references.
At 15%, all 46 instrumental references read instrumental and 51 of 59 vocal ones read vocal. The misses are tracks with only brief vocals (1 to 6% of the track).
At 5%, it catches 57 of 59 vocal tracks but marks 8 of 46 instrumentals as vocal (synth or sample parts at 10 to 15%).
The app uses 15%: a V on the player should mean vocals a DJ will hear, and a missing V on a track with one sampled line costs less than a false one.

Speed

Run	Seconds per track
Key and rhythm, one session at a time	4.4 (the first call 12, presumably loading the model)
Key and rhythm, four in flight, 24 tracks	1.9
Key and rhythm, four in flight, 100 tracks	3.0
Key and rhythm, six in flight	no faster than four
Key and rhythm, eight in flight	once stalled for minutes at near-zero CPU
Pace and instrument activity, four in flight	1.95
All four in one session, four in flight, 20 tracks	2.75

Ask for everything you need in one call: all four analyses in one session cost less than two separate passes. A 30-track playlist takes one to one and a half minutes the first time, and nothing the second.

Pitfalls

Feed PCM in a sandboxed app. The AVAsset path crashed once with four sessions in flight (above); audioProvider: with AVReadOnlyAudioPCMBuffer(copying:) has not.
Cap the parallelism. Four was the sweet spot on an M3; eight once stalled for minutes with the CPU idle, and nothing came back until it recovered.
Expect a slow first call. 12 s against 4.4 s afterwards, presumably model loading.
SwiftUI on a macOS 14 target. An if #available inside TableColumnBuilder needs macOS 14.4, so the app builds two Tables, one with the macOS column, under #available(macOS 27, *).
Map the key yourself. The framework returns a tonic (with enharmonic spellings as separate cases) and a mode; DJ software shows the Alphanumeric wheel. The mapping above puts C major and A minor on 8, as rekordbox does.
Keep derived tags unambiguous. On a player, 7V could be read as a key; E7V cannot.

Still open

Loudness. The framework's integrated, momentary, and short-term LUFS have not been compared against a reference BS.1770 meter, and whether its peak is a sample peak or a true peak is not known.
The tags on a player. Where they show comes from the CDJ manuals and from photos of the comment line on a CDJ-2000 and a CDJ-2000NXS2; a stick with the tags has not been read on a player yet.
Accuracy against other analysers. Only rekordbox was compared. Numbers against KeyFinder, Essentia, or hand-checked keys would tell more than agreement does.

The discussion is on hydrogenaudio. Measurements from other libraries are welcome there.

Where to find it

Check BPM and Key and the energy and vocal tags are in Bake'n Deck for Mac 1.4.0 on macOS 27. Every other feature works on macOS 14 or later. The app has no network access, so the analysis never leaves the Mac.

Back to Bake'n Deck →