reaper-control/.sdd/design.md

Design: reggaeton-composer

Technical Approach

Extend the existing SongDefinition → FLPBuilder pipeline with a new selection + melodic-generation layer. SampleSelector wraps sample_index.json; melodic generators produce {notes, sample_path} dicts; SongDefinition gains a melodic_tracks field; FLPBuilder appends audio-clip channels after existing drum channels.

---

Architecture Decisions

Decision	Choice	Rejected	Rationale
Selector load strategy	Load full index once at __init__	Lazy / streaming	862 entries ≈ 50 MB max; random access needed for scoring
Note format	Reuse {"pos","len","key","vel"} from rhythm.py	New format	Already converted by _convert_rhythm_notes; zero friction
Schema extension	Add melodic_tracks: list[MelodicTrack] optional field	Separate class	Single source of truth; existing validate() extended, not replaced
Channel numbering	Melodic starts at ch 17	Dynamic	skeleton.py already defines EMPTY_SAMPLER_CHANNELS = {17,18,19}; expanding naturally
Melodic channel type	ChType = 21 → value 0 (sampler)	AudioClip type 1	Reference FLP uses sampler channels for .wav; skeleton.py already patches them via melodic_map param (already present in load() signature)
Builder integration	_build_melodic_channels() inserted between channel_bytes and arrangement	New Builder subclass	Minimal diff; builder is not subclassed anywhere
CLI	scripts/compose_track.py using argparse	Click	Existing scripts use plain argparse

---

Data Flow

data/sample_index.json
        │
        ▼
  SampleSelector.select(role, key, bpm, character, is_tonal)
        │  list[SampleEntry]
        ▼
  melodic.py generators
  generate_bass / generate_lead / generate_chords / generate_pad
        │  MelodicTrackDef {role, sample_path, notes[], channel_hint}
        ▼
  SongDefinition (extended)
  .melodic_tracks: list[MelodicTrack]
        │
        ▼
  FLPBuilder.build(song)
        ├── _build_header          (unchanged)
        ├── _build_all_patterns    (unchanged)
        ├── ChannelSkeletonLoader  (pass melodic_map → already supported)
        ├── _build_melodic_notes() (NEW — PatNotes for ch 17+)
        └── _build_arrangement     (unchanged)
        │
        ▼
  output/my_track.flp

---

File Changes

File	Action	Description
src/selector/__init__.py	Create	SampleSelector class
src/composer/melodic.py	Create	generate_bass/lead/chords/pad
src/flp_builder/schema.py	Modify	Add MelodicTrack, Note dataclasses; extend SongDefinition; extend validate() and from_json()
src/flp_builder/builder.py	Modify	Add _build_melodic_notes(); pass melodic_map to ChannelSkeletonLoader.load()
scripts/compose_track.py	Create	CLI entry point

---

Interfaces / Contracts

# src/selector/__init__.py
@dataclass
class SampleEntry:
    path: str          # original_path (absolute)
    role: str
    key: str | None
    bpm: float         # 0 = unknown
    character: str
    is_tonal: bool
    score: float = 0.0

class SampleSelector:
    def __init__(self, index_path: str | Path): ...
    def select(
        self,
        role: str,
        key: str | None = None,
        bpm: float | None = None,
        character: str | None = None,
        is_tonal: bool | None = None,
        limit: int = 10,
    ) -> list[SampleEntry]: ...

# src/composer/melodic.py
@dataclass
class MelodicTrackDef:
    role: str                        # "bass" | "lead" | "chords" | "pad"
    sample_path: str
    notes: list[dict]                # {"pos","len","key","vel"}
    channel_hint: int                # suggested channel index (17+)

def generate_bass(key, scale, bpm, bars, kick_pattern=None, density=0.7) -> MelodicTrackDef
def generate_lead(key, scale, bpm, bars, density=0.5) -> MelodicTrackDef
def generate_chords(key, scale, bpm, bars, progression=None, voicing="closed") -> MelodicTrackDef
def generate_pad(key, scale, bpm, bars) -> MelodicTrackDef

# src/flp_builder/schema.py (additions)
_KNOWN_ROLES = frozenset({"bass","lead","chords","pad","arp","fx"})

@dataclass
class Note:                   # alias for PatternNote — same fields
    pos: float
    len: float
    key: int
    vel: int

@dataclass
class MelodicTrack:
    role: str
    sample_path: str
    notes: list[Note]
    channel_index: int        # >= 17
    volume: float = 0.78      # 0.0–1.0
    pan: float = 0.0          # -1.0–1.0

# SongDefinition gets:
melodic_tracks: list[MelodicTrack] = field(default_factory=list)

---

Key Scoring Logic

# Key compatibility (circle-of-fifths aware)
COMPAT = {
    "exact":        1.0,
    "relative":     0.8,   # Am ↔ C, Gm ↔ Bb
    "dominant":     0.6,   # Am → Em
    "subdominant":  0.6,   # Am → Dm
    "parallel":     0.5,   # Am ↔ A
}

# Character groups (fuzzy matching)
CHAR_GROUPS = [
    {"warm","soft","lush"},
    {"boomy","deep"},
    {"sharp","crisp","bright"},
    {"aggressive","dark"},
]

# Combined score = key_score * 0.5 + bpm_score * 0.3 + char_score * 0.2

---

Tresillo / Dembow Bass Pattern

Bar positions (8 bars × 4 beats = 32 beats):
Tresillo: [0, 0.75, 1.5] per bar  (3+3+2 in 8th-note grid)
Kick-avoidance: skip notes within ±0.125 beats of a kick hit

---

Validation Extensions

SongDefinition.validate() additions:

1. melodic_track.role in _KNOWN_ROLES
2. Path(melodic_track.sample_path).exists()
3. melodic_track.channel_index >= 17
4. No duplicate channel_index across melodic tracks

---

Error Handling Strategy

Layer	Strategy
SampleSelector.select()	Returns [] on no match — callers must check; never raises on empty
generate_* functions	Raise ValueError if selector.select() returns empty (required sample missing)
SongDefinition.validate()	Accumulate all errors, raise ValueError with full list
FLPBuilder.build()	Propagates FileNotFoundError if sample_path missing at write time
CLI	sys.exit(1) with human-readable message on any ValueError / FileNotFoundError

---

Testing Strategy

Layer	What	Approach
Unit	SampleSelector scoring	Fixture with 5 hand-crafted entries; assert rank order
Unit	generate_bass tresillo	Assert note positions match [0, 0.75, 1.5] per bar
Unit	MelodicTrack schema validation	validate() returns expected errors for bad inputs
Integration	FLPBuilder with melodic tracks	Build to bytes, parse header, assert num_channels == 17 + n
Smoke	CLI end-to-end	compose_track.py --key Am --bpm 95 --output /tmp/test.flp; assert file exists and size > 0

---

Generation + Validation Pipeline

CLI: generate-song

A thin CLI wrapper (scripts/generate.py) delegates to compose.main() and optionally validates output.

Flags:

Flag	Type	Default	Description
--bpm	float	95	Tempo
--key	str	Am	Musical key
--output	str	output/song.rpp	Output path
--seed	int	42	Random seed (reproducibility)
--validate	flag	False	Run validate_rpp_output() after generation

BPM validation: Raises ValueError if bpm <= 0.

Data flow:

generate.py main()
  ├── argparse (--bpm, --key, --output, --seed, --validate)
  ├── import compose.main
  │   └── compose.main(args) → writes output.rpp
  └── if --validate:
      └── validate_rpp_output(output_path) → list[str]
          ├── count <TRACK> blocks
          ├── regex <SOURCE WAVE> for audio paths → verify exist
          ├── regex NOTE for MIDI clip notes
          ├── compute arrangement end beat
          ├── regex AUXRECV for send routing
          └── count VST/FXCHAIN for plugin chains

Validator: rpp_validator.py

src/validator/rpp_validator.py exports validate_rpp_output(rpp_path: str) -> list[str].

Returns [] for valid .rpp, or a list of error strings for any violation.

6 structural checks:

1. Track count — must be exactly 9 (<TRACK blocks)
2. Audio clip paths — all <SOURCE WAVE "..."> paths must exist on disk
3. MIDI note presence — MIDI items must contain at least one NOTE event
4. Arrangement duration — last item end must be ≥ 52 bars × 4 beats at given BPM
5. Send routing — each non-return track must have AUXRECV to a return track
6. Plugin chains — each non-return track must have <FXCHAIN> with VST entry

Perc Loop Fallback

build_perc_track() in compose.py silently skips missing 91bpm bellako percloop.wav files. No changes needed — this matches the spec requirement.

---

ReaScript Hybrid Pipeline (Phase 1 + Phase 2)

Overview

Two-phase architecture: Phase 1 generates .rpp offline; Phase 2 runs inside REAPER via ReaScript for FX verification, track calibration, rendering, and loudness measurement.

Phase 1 (offline, Python)          Phase 2 (inside REAPER, ReaScript)
─────────────────────────           ─────────────────────────────────
RPPBuilder.build()                 Main_openProject(rpp_path)
       │                                    │
       ▼                                    ▼
output/song.rpp                   TrackFX_GetCount + TrackFX_GetFXName
                                     → fx_errors (missing plugins)
       │                                    │
       │                           SetMediaTrackInfo_Value(VOLUME/PAN)
       │                           CreateTrackSend for each send
       │                                    │
       │                                    ▼
       │                           Main_RenderFile → output/song.wav
       │                                    │
       │                                    ▼
       │                           CalcMediaSrcLoudness
       │                           → integrated_lufs, short_term_lufs
       │                                    │
       │                                    ▼
       │                           write result.json
       ▼
run_in_reaper.py
  → generate phase2.py
  → write command.json  ──────────────────────────────►
  → poll result.json    ◄──────────────────────────────
  → print LUFS, write fx_errors.json

Bridge: JSON File Protocol

Communication via fl_control_command.json / fl_control_result.json in REAPER ResourcePath:

• No network dependency
• REAPER owns timing — avoids race conditions
• Human-readable JSON for debugging

Command JSON Schema

{
  "version": 1,
  "action": "calibrate" | "verify_fx" | "render",
  "rpp_path": "absolute path to .rpp",
  "render_path": "absolute path for rendered output (wav)",
  "timeout": 120,
  "track_calibration": [
    {
      "track_index": 0,
      "volume": 0.85,
      "pan": 0.0,
      "sends": [{"dest_track_index": 5, "level": 0.05}]
    }
  ]
}

Result JSON Schema

{
  "version": 1,
  "status": "ok" | "error" | "timeout",
  "message": "optional error message",
  "lufs": -14.2,
  "integrated_lufs": -14.2,
  "short_term_lufs": -12.1,
  "fx_errors": [{"track_index": 2, "fx_index": 1, "name": "", "expected": "Serum_2"}],
  "tracks_verified": 8
}

Key Files

File	Role
src/reaper_scripting/__init__.py	ReaScriptGenerator — generates self-contained Python ReaScript
src/reaper_scripting/commands.py	ReaScriptCommand, ReaScriptResult dataclasses + protocol
scripts/run_in_reaper.py	CLI: generate script → write command → poll result → output LUFS

ReaScript API Subset (Stable)

Main_openProject, TrackFX_GetCount, TrackFX_GetFXName, TrackFX_AddByName, SetMediaTrackInfo_Value, CreateTrackSend, Main_RenderFile, CalcMediaSrcLoudness, GetFunctionMetadata, and others listed in the spec.

Architecture Decisions

Decision	Choice	Rationale
JSON file protocol over python-reapy	fl_control_command.json / fl_control_result.json	No network dependency; REAPER owns timing
Self-contained ReaScript (no import json)	Hand-rolled JSON parser via string splitting	Maximum REAPER version compatibility
Separate commands.py for protocol	Protocol isolated from generator	Stable, testable in isolation
track_calibration JSON array	Stateless interface for volume/pan/sends	Retry-friendly; replay on REAPER crash

---

Open Questions

• skeleton.py EMPTY_SAMPLER_CHANNELS includes {17,18,19} — need to confirm that adding melodic channels beyond 19 doesn't require reference FLP changes or if we expand the sampler clone range.
• sample_index.json entries use original_path (absolute Windows paths). CLI on other machines will break. Decision needed: embed relative paths or make selector rebase paths against a configurable library_root.
• Should render_path default to the .rpp's folder with _rendered.wav suffix?
• Do we need to handle REAPER's __startup__.py registration automatically, or is manual Action registration acceptable for Phase 1?