Features

How Rock N Roller solves its harder problems: local CDLC, catalog search, practice speed and section loops, Demucs stems, left-handed / inverted highway layout, a 3D chart whose camera math and note clock stay locked to the audio callback, and a Chords + Lyrics sheet for following names with the words — from one C + raylib codebase on macOS, Windows, and WebAssembly.

1. Stack & platforms
2. Library that stays on your disk
3. SQLite on disk (macOS / Windows)
4. Catalog search & query language
5. Browser library access
6. Multithreaded wasm
7. Practice speed without pitch shift
8. Section loop
9. Stems & Demucs
10. Left-handed & string order
11. Long songs without blowing RAM
12. 3D highway
    1. World space & fret math
    2. Camera, frustum, trigonometry
    3. What a frame draws
    4. Notes in PSARC / SNG
    5. Load, audio, and raylib threads
    6. How the chart locks to audio
13. Chords + Lyrics
14. Keeping the UI smooth
15. What ships

Stack & platforms

The app is C17 (plus one C++17 wrapper for Signalsmith Stretch) on raylib 6.0. One CMake project builds macOS (Apple Silicon and Intel), Windows x64, and Emscripten wasm from the same commit — platform differences stay in small shims for paths, dialogs, and threading.

Heavy lifting is a short list of libraries: SQLite for state and the song catalog, an in-repo PSARC reader, vgmstream for Wwise audio (no FFmpeg), ogg/vorbis, and Signalsmith for time stretch. The UI is drawn in-app; there is no separate toolkit.

Library that stays on your disk

Opening every .psarc on every launch would be slow. Instead, a local SQLite DB caches each song's metadata (title, artist, tunings, arrangements, length) keyed by file size and mtime. Startup loads the catalog from the DB; a background rescan only re-parses files that changed. Profiles, playlists, play history, and settings live in the same database. Charts are Rocksmith's binary SNG — parsed once when you pick a song, not from XML at runtime.

SQLite on disk (macOS / Windows)

On desktop, the database is a normal file on your machine — not in the cloud, not inside the app bundle. The state directory is created on first launch; the DB file is rocknroller.db (plus SQLite WAL sidecars while the app is open).

That file holds the song scan cache, profiles, playlists, play history, library folder list, stems folder path, autoplay mode, chart layout, and per-profile cell settings (including left-handed / invert). Your PSARC and stem audio stay wherever you put them — only metadata and prefs are in the DB. Reset in Settings wipes this state, not your media files.

Catalog search & query language

A large CDLC library is useless if you can only scroll it. The catalog search bar accepts plain text or a small query language that filters on the metadata already in SQLite — no reopening PSARCs.

Bare words are a global substring match (case-insensitive) across title, artist, album, year, and tuning. Column filters use field="value" and can be mixed with free text:

Inside the quotes, comma means AND and | means OR. Examples:

• stems="true" tuning="E Standard | Drop D" — songs with stems whose tuning matches either string
• artist="Rush" instruments="l, b" — Rush charts that have both lead and bass
• instruments="l-E Standard|b-Drop D" — lead in E Standard or bass in Drop D
• official="false" stairway — CDLC only, plus free-text “stairway”

Clicking an album or artist card fills the bar with matching artist="…" / album="…" clauses. The same query powers the play queue when autoplay advances to the next song.

Browser library access

The browser has no filesystem, but the app needs a folder of your CDLC. The solution:

• Chromium's File System Access API grants read-only access to a local folder; the handle is remembered in IndexedDB so you don't re-pick every visit.
• The folder is mirrored into wasm MEMFS as zero-byte skeleton files with real names, sizes, and mtimes. Desktop scan/cache logic runs unchanged — and because the cache keys on size + mtime, cataloging never needs the file bytes.
• Opening a song stages the real PSARC on demand: a worker parks while the main thread copies bytes into a bounded LRU cache (512 MiB), then wakes. Large libraries stream through a fixed footprint.

Bytes go disk → page memory only. Nothing is uploaded. This (and folder picking) is why the web build needs Chrome or Edge.

Multithreaded wasm

The web build is the same C, compiled with Emscripten as multithreaded wasm: pthreads → Web Workers sharing memory via SharedArrayBuffer. That requires a cross-origin-isolated page (COOP/COEP headers). GitHub Pages can't send those, so a same-origin coi-serviceworker injects them on player URLs only (/latest/, /0.1.35/, … — one reload on first Play visit). Landing and docs stay un-isolated so embedded YouTube can load. The ~4 MB engine then streams from CDN; the site itself stays a thin shell.

Practice speed without pitch shift

Pitch is frequency. An octave up means the waveform repeats twice as often: 220 Hz (A3) → 440 Hz (A4) → 880 Hz (A5). Twelve equal-tempered semitones make that doubling:

f′ = f · 2ⁿ   (n = octaves)
one semitone = 2^1/12 ≈ 1.0595×

Playing a buffer slower without processing stretches each period T. Frequency is 1/T, so half speed makes every note an octave down and the guitar sounds like a bass. Practice mode needs 0.3×–3.0× with that frequency held still — the lick lasts longer (or shorter), but A stays A.

Inside the miniaudio callback the app does not resample. It asks Signalsmith Stretch (cheaper preset, splitComputation on) to emit a device-rate period from a different number of source frames:

in_frames = ceil(device_frames · rate)
stretch(in, in_frames) → out[device_frames] at the original pitch

At 0.5× it pulls half as much song audio as it plays to the DAC: the lick is twice as long. At 2× it pulls twice as much and compresses it into one period. At 1.0× the stretcher is bypassed (copy samples straight through) so there is no extra latency or FFT cost when you are not practicing.

Stretch keeps pitch by working in the frequency domain, not by scaling the waveform. Overlapping windows (~100 ms block, ~40 ms hop at 48 kHz) go through an STFT. Each bin is a sinusoid with a magnitude and a phase. Time-stretch walks the analysis windows through the input at rate while the synthesis hop stays at the device interval:

input_offset = output_index · (in_frames / out_frames)

A classic phase vocoder then advances each bin’s phase so its instantaneous frequency (how fast phase rotates) stays the same as in the source. The note’s harmonics keep their Hz; only how long they ring changes. Signalsmith adds a second prediction that locks phase across neighboring bins (vertical locking), which keeps transients and stacked guitar partials from smearing the way a naive vocoder does. Rock N Roller never transposes: the engine’s frequency multiplier stays 1, so it skips the spectral-peak remap used for pitch shift. Above a 2× stretch ratio it starts randomising some phases; 3.0× is inside that “less clean” region on purpose so fast-forward still works.

splitComputation spreads one STFT’s analyse / predict / synthesise steps across the callback period so a 1024-frame quantum does not spike the audio thread. The overlap-add still needs a look-ahead buffer; that output latency (synthesis delay plus one hop when split) is subtracted when publishing the heard clock, which is why the chart clock can lock gems to what you actually hear. Speed and seek reset the stretcher atomically inside the callback so a rate change cannot mix two phase histories.

Section loop

Rocksmith charts tag named sections (intro, verse, solo, …). Press P or the footer Loop button to loop the section under the playhead: the app seeks to that section’s start and repeats until you turn loop off. While loop is on, Enter or O jumps back to the loop start and unpauses — useful after you scrub away mid-practice.

Loop edges show on the seek bar and can be dragged to tighten the span (as long as the chart has sections). Combined with pitch-preserving speed and stem mutes, this is the usual “drill this four bars” workflow. See keybinds.

Stems & Demucs MLX

Stem mode mixes six isolated tracks (drums, bass, other, vocals, guitar, piano) with per-stem faders. Separation runs offline in the companion Demucs MLX app (macOS Apple Silicon via MLX, or Windows x64 via NVIDIA CUDA), then Rock N Roller matches {stems_root}/{psarc_basename}/ and streams the six MP3s. Full workflow, Mac vs Windows size/speed, folder layout, mixer, and ?demo=stems: stems page.

Left-handed & string order

Two independent highway layout flags, saved per profile and per cell size class (also bulk-set from catalog Settings):

Toggle them in Cell Settings while playing, or use Settings → “Set all to Left-handed / Right-handed” and “Set all to Default / Inverted” for every size class at once. Details: settings.

Long songs without blowing RAM

A long chart can be huge if kept fully renderer-ready. Songs over 10 minutes keep the parsed chart in a compact cache and only expose a 3-minute window to the highway. As you near the edge (with preload that scales with practice speed), the window slides and refills. Shorter songs drop the cache after the first fill.

Audio streams similarly: OGG from disk, stems as MP3 with seek points, and long WEM-only tracks decode in the background while the chart starts. Memory stays flat for a three-minute song or a forty-minute one.

3D highway

Each instrument pane is its own scissored 3D viewport (raylib rlgl). The neck is a real 3D scene — not a 2D strip with fake perspective. Gems live on a guitar-scaled fretboard, time maps to depth, and an off-axis camera looks down the runway without keystoning the fret wires. Zoom, runway grade, and lookahead are per-cell settings. Left-handed and invert flags remap X/Y before draw (see left-handed & string order). Target is 60 FPS with four panes up.

World space & fret math

The highway uses a right-handed world with a 24-fret neck:

Fret positions are the equal-tempered guitar formula (the same math as a real neck): each twelve-fret octave halves the remaining string length. With scale length L = 2.25 world units:

x(f) = L · (1 − 2^−f/12)   for fret f = 0…12

That is L - L / 2^(f/12). Consecutive fret gaps shrink toward the body the way a physical guitar does. Past fret 12 the remaining gaps would be too tight on screen, so the renderer stretches everything beyond the 12th fret by 1.65× around that anchor — high-fret phrases stay readable without breaking the low-fret spacing players already know.

A gem’s X is the midpoint of its fret cell; open notes (fret 0) are a wide slab centered on the chart’s current hand-position anchor (the SNG “where is the left hand” window), not on the nut. String Y is evenly spaced. Time becomes depth with a constant seconds-to-world scale TS ≈ 1.5 (200 × the neck’s unit K):

Δt = note.time − song_timer
z = −Δt · TS

A note 2 seconds away sits at z ≈ −3. When Δt ≤ 0 the gem is pinned at the strike line (z = 0) for the sustain, then fades. The visible window is about 0.5 s behind the playhead to lookahead seconds ahead (default 3 s, slider 1.5–6). Only that slice is batched.

Camera, frustum, trigonometry

The camera never pitches. Position and look-at share the same Y, so the view direction is straight down −Z — horizontal. Fret wires stay parallel to the window’s top edge; there is no keystone. The “looking down a rising highway” feel comes from two things: the camera sits above the neck, and the projection is an asymmetric (shift) frustum that slides the image down so the neck sits low in the pane with a horizon near the top.

Runway grade is that height multiplier. Default 1.0 matches the original elevation; the slider (0.35–2.2) scales cam_h only. Higher grade → larger θ → the runway converges toward the horizon faster. The camera still does not pitch.

Zoom is mostly a dolly: cam_dist is divided by the zoom factor (0.5×–2.5×). Closer camera, larger on-screen neck. A small FOV tweak then fits the residual.

Perspective uses the usual half-angle tangent. With vertical field of view fovy and near-plane distance n:

top = n · tan(fovy / 2)
right = top · (viewport_width / viewport_height)

A symmetric frustum would be [-right, right] × [-top, top]. Instead the renderer shifts the vertical window by a fraction s of top (a shift-lens / off-axis projection):

cy = top · s
frustum Y = [cy − top, cy + top]

s is not a user slider. After FOV is fitted, it is solved in one closed form so the near-bottom of the neck (fret-number row) pins to a fixed screen Y — about the bottom of the pane — regardless of zoom or grade. A uniform vertical shift moves every projected Y by the same amount, so it cancels out of the FOV height ratio and can be applied last. Pitching the camera would have keystoned the frets; shifting the frustum does not.

Half the world-space width visible at the camera’s distance (the quantity the wide-pane pan uses):

visible_half = tan(fovy / 2) · cam_dist · aspect

Two camera-follow modes sit on top of that fit. Center (tall / quarter cells) keeps the active lane in the middle. Wide (full-width cells) holds a deadband: the camera only pans when the required fret span approaches a frustum edge, then glides the minimum amount. Lane extents themselves are a box-filtered average of SNG hand anchors over a short time window, so the pan is piecewise-linear in song time — no hitch when the hand jumps a few frets.

Far down the runway, perspective would shrink a constant-size gem below a pixel. World size needed for roughly constant screen size grows linearly with depth, so open bars and thin wires use 1 + |z| / cam_dist as a keep-alive scale (capped so the spawn end does not balloon). Approaching gems also rotate: they start ~90° edge-on at lookahead and flatten to the neck as (Δt / ahead) · π/2 → 0.

What a frame draws

highway_3d_draw(instrument, song_timer, viewport, …) runs on the raylib thread once per pane, per frame:

1. Scissor to the cell; rebuild a per-frame cache (lane bounds, note batch) keyed on song_timer.
2. Binary-search the sorted note array for [song_timer − 0.5s, song_timer + ahead], including sustains that started earlier and still overlap (cap 512 visible notes).
3. Fit / reuse the camera; begin a custom 3D mode with the shifted frustum (not stock BeginMode3D, which cannot scissor per pane or shift the lens).
4. Draw in a fixed depth order: lane surface → hit bar → fret wires → glass chord frames → sustain ribbons (depth-write off) → string wires → gem halos / ghost repeats → solid gem bodies → open slabs (depth test off so far bars do not flicker against the faded lane).
5. Restore the 2D ortho matrix and overlay SDF fret numbers, chord names, and the chord-diagram HUD in screen space (world → clip → NDC → pane pixels, using the same shifted projection).

Gems are halo / body / edge, Rocksmith-style. Sustains are ribbons that can follow a bend envelope. Repeat-chord “ghost” gems travel inside the next identical frame at reduced alpha so they never depth-block the landing chord. Four panes share four small camera contexts so layout A does not steal layout B’s pan state.

60 FPS is a 16.7 ms budget for the whole window — every highway, the lyrics pane, headers, and the footer. The bottleneck is almost never “too many triangles.” It is GPU submissions: each time GL state changes (texture, shader, blend, depth-write, scissor, viewport, quad vs triangle), raylib’s rlgl must flush the vertex batch and start a new draw call. A naive loop that fully drew each gem (halo + body + edge + icon + label) would flush hundreds of times per pane.

The note batch exists to stop that. Visible notes are collected once into a Hw3dNoteBatch, then the same array is walked in several cheap passes. Within a pass, gems share one rlBegin(RL_QUADS) (or triangles) so four edge bars and many gem faces land in a single submission instead of a DrawCube per note. Technique icons go into a spritesheet atlas and are drawn after 3D with one texture. Lane/camera caches skip work when song_timer and the viewport have not changed in a way that matters. Depth-write still has to toggle between opaque gems and glass/ribbons — those flushes are deliberate, not accidental — but they happen once per layer, not once per gem.

That is why the full kitchen sink is hard. A four-quarter layout is four independent 3D viewports: four scissors, four projections, four depth clears, four note batches, four 3D→2D restores, four SDF fret-label passes and chord HUDs. Cost is roughly linear in pane count, and a dense chart at long lookahead can fill all 512 slots in each cell. Put lyrics (or Chords + Lyrics) in the remaining slot — the “3 highways + lyrics” template, or 3D + Lyrics as an inline strip on a wide cell — and you add a different tax on the same 16.7 ms: the SDF font shader, a DrawTextEx per syllable, karaoke bloom (the active word is drawn five times, center plus four offsets), and chord names drawn twice for weight. Every shader / 2D-vs-3D flip breaks the batch the highways just built. Four quarter highways alone already multiply the 3D path; four cells of highways plus a chords-and-lyrics pane is the worst of both worlds, which is why the 60 FPS target is aimed at that layout, not at a single full-window neck.

Notes in PSARC / SNG

A *_p.psarc is a PlayStation-style archive: magic PSAR, zlib block compression, a table of contents, and a names block. Official DLC sets archive flag 4 and AES-encrypts the TOC; CDLC usually does not. The reader decrypts when needed, inflates each entry on demand, and never needs the XML toolkit files at runtime.

Chart data lives in songs/bin/… as binary SNG (not XML):

• *lead*.sng, *rhythm*.sng, *bass*.sng
• *vocals*.sng for lyrics

An SNG file on disk is still encrypted. Header magic 0x4A, a 16-byte AES-CTR IV, then ciphertext with a platform key (PC for *_p.psarc). After decrypt, a u32 uncompressed size and a zlib stream unpack to the real chart blob: BPM, then one arrangement per dynamic-difficulty tier, each with its own notes and hand anchors, plus song sections (intro / verse / solo), chord templates, and (for vocals) timed lyric syllables.

Playback uses the arrangement whose difficulty equals the SNG metadata max — the fully authored chart, not a lower DD tier. Each note row is a packed binary record. The fields the highway actually stores:

Those rows are copied into a flat NoteEvent[], sorted by time then string. Chord SNG rows explode into one gem per sounding string (a “spoke”). Songs longer than 10 minutes keep the full parsed SNG in a compact cache and only expose a 3-minute window to the highway — see long songs. Audio is a separate extract: first *.ogg if present, else the largest audio/*.wem, decoded with vgmstream to a temp WAV.

Load, audio, and raylib threads

Three threads touch a song. They never share OpenGL, and the audio callback never waits on the renderer.

Picking a song spawns pick_load_thread. That worker is the only place that opens the PSARC, decrypts SNGs, and fills Instrument.notes. The frame loop keeps rendering the catalog (or a loading overlay) until pick_load_done flips; then main joins the worker (so the note arrays are visible) and music_reload opens miniaudio on the main thread. Device creation is not done on the loader — audio APIs want the thread that owns the player.

After that, a dedicated device callback (miniaudio, not raylib’s audio) runs at the hardware period (~1024 frames). It pulls source samples, optionally time-stretches them, writes the output buffer, and publishes a clock. Seek, speed, and volume from the UI are atomic stores plus a pending flag; the callback consumes them at the start of a period. No mutex, so a hitch in draw cannot underrun the DAC. Long WEM tracks decode on a fourth worker; the chart can start before the WAV exists, then the callback is swapped in when decode finishes (see keeping the UI smooth).

On web this is the same C: pthreads become Workers on SharedArrayBuffer (see multithreaded wasm). The “raylib thread” is still the browser’s main thread that owns WebGL.

How the chart locks to audio

The highway does not read the DAC clock every frame. Audio periods are coarser than 60 Hz and stretch adds latency, so a raw sample-counter would make gems stutter. Instead main keeps a chart clock that is anchored to audio and extrapolated with wall time between samples.

1. The callback tracks heard (how far the decoder has read into the file). When stretch is active it subtracts (latency_samples · rate) / sample_rate so the published time is “what you are hearing now,” not “what was just pulled into the stretcher.” At 1.0× stretch is bypassed and latency is zero.
2. That value is stored in published_time_sec (relaxed atomic). Main’s play_audio_time_played() is just an atomic load.
3. On play / resume / seek, main snapshots (wall_now, audio_t) as the anchor and sets song_timer = audio_t.
4. Each playing frame: song_timer = audio_t + speed · (GetTime() − wall_anchor). Gems move smoothly even if the callback has not run this frame.
5. About every 75 ms, or sooner if extrapolated time drifts more than 120 ms from the published clock (or the anchor is older than 100 ms), main re-reads the atomic and re-anchors. That corrects stretch, OS scheduling, and speed changes without a visible snap.

song_timer(t) = audio_anchor + speed · (wall(t) − wall_anchor)
resample when |song_timer − published_time| > 0.12 s

Seek writes a pending target into the callback; main holds song_timer on the requested time for a short grace so the highway does not jump to stale audio then back. If the published clock stops advancing while the device still claims to be playing (tab freeze, AudioContext stall), the chart clock holds at the last audio time instead of running ahead on wall time — that was the old “highway sawtooth” where gems raced then snapped back every resample. Speed changes reset the stretcher and re-anchor in the same frame so notes stay under the hit line at 0.3× and at 3.0×.

The draw call is then just geometry: highway_3d_draw(inst, song_timer, pane). Same song_timer drives lyrics, the seek bar, section loop, and every instrument cell, so four highways cannot drift apart.

Chords + Lyrics

A lyrics pane can show the chart’s named chords sitting on the words — Ultimate Guitar-style changes, timed to the same vocal line cache the karaoke panel already uses. This is a Lyrics cell viz, not the 3D highway: glass chord frames on the neck stay their usual pale rim; only the sheet’s alphanumeric names are color-coded.

Pick a layout that has a lyrics slot (e.g. 3 highways + lyrics, or Full with the cell source set to Lyrics). In that cell’s header, set viz to Chords + Lyrics (the other lyrics option is the plain karaoke panel). Lyric size and lyric-line count still come from Cell Settings.

Names are the chart’s chord-template labels (e.g. A7sus4), emitted on each change and restated after a stretch of silence so a verse does not stay unlabeled after a break. The chord track prefers the rhythm arrangement, then lead. If the song has no named chords (or no rhythm / lead notes), the pane falls back to the regular lyrics panel.

Each lyric row gets a chord band above the words. A change anchors above the first word that starts at or after that time; a chord in the second before a line’s first word is a pickup at the line start. Colors are kept apart from karaoke (yellow past / red sweep / white upcoming):

Instrument cells can still use 3D + Lyrics for an inline strip on the highway; that strip is lyrics only. Layout and viz details: settings.

Keeping the UI smooth

The main thread renders; workers handle library sync, song load, WEM decode, stem mixer open, and album art. They finish via atomic flags the frame loop polls — no blocking joins on the hot path. Speed, seek, and volume cross into the audio callback with atomics only, so the UI can't stall playback. The same model runs on web as the pthread pool above. How those three threads hand a song to the 3D highway (PSARC parse → device callback → chart clock) is under 3D highway.

What ships

Desktop is roughly an 8 MB binary; the web package is about 1.8 MB zipped (~4 MB wasm + JS + a ~420 KB preload of fonts/shaders/logo — songs never ride along). That size is mostly “link only what Rocksmith content needs” and keep assets out of the download, not a separate size project.