# Design: reggaeton-composer

## Technical Approach

Extend the existing `SongDefinition → FLPBuilder` pipeline with a new selection +
melodic-generation layer. `SampleSelector` wraps `sample_index.json`; melodic
generators produce `{notes, sample_path}` dicts; `SongDefinition` gains a
`melodic_tracks` field; `FLPBuilder` appends audio-clip channels after existing
drum channels.

---

## Architecture Decisions

| Decision | Choice | Rejected | Rationale |
|----------|--------|----------|-----------|
| Selector load strategy | Load full index once at `__init__` | Lazy / streaming | 862 entries ≈ 50 MB max; random access needed for scoring |
| Note format | Reuse `{"pos","len","key","vel"}` from `rhythm.py` | New format | Already converted by `_convert_rhythm_notes`; zero friction |
| Schema extension | Add `melodic_tracks: list[MelodicTrack]` optional field | Separate class | Single source of truth; existing `validate()` extended, not replaced |
| Channel numbering | Melodic starts at ch 17 | Dynamic | `skeleton.py` already defines `EMPTY_SAMPLER_CHANNELS = {17,18,19}`; expanding naturally |
| Melodic channel type | `ChType = 21` → value `0` (sampler) | AudioClip type 1 | Reference FLP uses sampler channels for .wav; `skeleton.py` already patches them via `melodic_map` param (already present in `load()` signature) |
| Builder integration | `_build_melodic_channels()` inserted between channel_bytes and arrangement | New Builder subclass | Minimal diff; builder is not subclassed anywhere |
| CLI | `scripts/compose_track.py` using `argparse` | Click | Existing scripts use plain argparse |

---

## Data Flow

```
data/sample_index.json
        │
        ▼
  SampleSelector.select(role, key, bpm, character, is_tonal)
        │  list[SampleEntry]
        ▼
  melodic.py generators
  generate_bass / generate_lead / generate_chords / generate_pad
        │  MelodicTrackDef {role, sample_path, notes[], channel_hint}
        ▼
  SongDefinition (extended)
  .melodic_tracks: list[MelodicTrack]
        │
        ▼
  FLPBuilder.build(song)
        ├── _build_header          (unchanged)
        ├── _build_all_patterns    (unchanged)
        ├── ChannelSkeletonLoader  (pass melodic_map → already supported)
        ├── _build_melodic_notes() (NEW — PatNotes for ch 17+)
        └── _build_arrangement     (unchanged)
        │
        ▼
  output/my_track.flp
```

---

## File Changes

| File | Action | Description |
|------|--------|-------------|
| `src/selector/__init__.py` | Create | `SampleSelector` class |
| `src/composer/melodic.py` | Create | `generate_bass/lead/chords/pad` |
| `src/flp_builder/schema.py` | Modify | Add `MelodicTrack`, `Note` dataclasses; extend `SongDefinition`; extend `validate()` and `from_json()` |
| `src/flp_builder/builder.py` | Modify | Add `_build_melodic_notes()`; pass `melodic_map` to `ChannelSkeletonLoader.load()` |
| `scripts/compose_track.py` | Create | CLI entry point |

---

## Interfaces / Contracts

```python
# src/selector/__init__.py
@dataclass
class SampleEntry:
    path: str          # original_path (absolute)
    role: str
    key: str | None
    bpm: float         # 0 = unknown
    character: str
    is_tonal: bool
    score: float = 0.0

class SampleSelector:
    def __init__(self, index_path: str | Path): ...
    def select(
        self,
        role: str,
        key: str | None = None,
        bpm: float | None = None,
        character: str | None = None,
        is_tonal: bool | None = None,
        limit: int = 10,
    ) -> list[SampleEntry]: ...

# src/composer/melodic.py
@dataclass
class MelodicTrackDef:
    role: str                        # "bass" | "lead" | "chords" | "pad"
    sample_path: str
    notes: list[dict]                # {"pos","len","key","vel"}
    channel_hint: int                # suggested channel index (17+)

def generate_bass(key, scale, bpm, bars, kick_pattern=None, density=0.7) -> MelodicTrackDef
def generate_lead(key, scale, bpm, bars, density=0.5) -> MelodicTrackDef
def generate_chords(key, scale, bpm, bars, progression=None, voicing="closed") -> MelodicTrackDef
def generate_pad(key, scale, bpm, bars) -> MelodicTrackDef

# src/flp_builder/schema.py (additions)
_KNOWN_ROLES = frozenset({"bass","lead","chords","pad","arp","fx"})

@dataclass
class Note:                   # alias for PatternNote — same fields
    pos: float
    len: float
    key: int
    vel: int

@dataclass
class MelodicTrack:
    role: str
    sample_path: str
    notes: list[Note]
    channel_index: int        # >= 17
    volume: float = 0.78      # 0.0–1.0
    pan: float = 0.0          # -1.0–1.0

# SongDefinition gets:
melodic_tracks: list[MelodicTrack] = field(default_factory=list)
```

---

## Key Scoring Logic

```python
# Key compatibility (circle-of-fifths aware)
COMPAT = {
    "exact":        1.0,
    "relative":     0.8,   # Am ↔ C, Gm ↔ Bb
    "dominant":     0.6,   # Am → Em
    "subdominant":  0.6,   # Am → Dm
    "parallel":     0.5,   # Am ↔ A
}

# Character groups (fuzzy matching)
CHAR_GROUPS = [
    {"warm","soft","lush"},
    {"boomy","deep"},
    {"sharp","crisp","bright"},
    {"aggressive","dark"},
]

# Combined score = key_score * 0.5 + bpm_score * 0.3 + char_score * 0.2
```

---

## Tresillo / Dembow Bass Pattern

```
Bar positions (8 bars × 4 beats = 32 beats):
Tresillo: [0, 0.75, 1.5] per bar  (3+3+2 in 8th-note grid)
Kick-avoidance: skip notes within ±0.125 beats of a kick hit
```

---

## Validation Extensions

`SongDefinition.validate()` additions:
1. `melodic_track.role in _KNOWN_ROLES`
2. `Path(melodic_track.sample_path).exists()`
3. `melodic_track.channel_index >= 17`
4. No duplicate `channel_index` across melodic tracks

---

## Error Handling Strategy

| Layer | Strategy |
|-------|----------|
| `SampleSelector.select()` | Returns `[]` on no match — callers must check; never raises on empty |
| `generate_*` functions | Raise `ValueError` if `selector.select()` returns empty (required sample missing) |
| `SongDefinition.validate()` | Accumulate all errors, raise `ValueError` with full list |
| `FLPBuilder.build()` | Propagates `FileNotFoundError` if `sample_path` missing at write time |
| CLI | `sys.exit(1)` with human-readable message on any `ValueError` / `FileNotFoundError` |

---

## Testing Strategy

| Layer | What | Approach |
|-------|------|----------|
| Unit | `SampleSelector` scoring | Fixture with 5 hand-crafted entries; assert rank order |
| Unit | `generate_bass` tresillo | Assert note positions match `[0, 0.75, 1.5]` per bar |
| Unit | `MelodicTrack` schema validation | `validate()` returns expected errors for bad inputs |
| Integration | `FLPBuilder` with melodic tracks | Build to bytes, parse header, assert `num_channels == 17 + n` |
| Smoke | CLI end-to-end | `compose_track.py --key Am --bpm 95 --output /tmp/test.flp`; assert file exists and size > 0 |

---

## Generation + Validation Pipeline

### CLI: generate-song

A thin CLI wrapper (`scripts/generate.py`) delegates to `compose.main()` and optionally validates output.

**Flags:**
| Flag | Type | Default | Description |
|------|------|---------|-------------|
| `--bpm` | float | 95 | Tempo |
| `--key` | str | Am | Musical key |
| `--output` | str | output/song.rpp | Output path |
| `--seed` | int | 42 | Random seed (reproducibility) |
| `--validate` | flag | False | Run `validate_rpp_output()` after generation |

**BPM validation:** Raises `ValueError` if `bpm <= 0`.

**Data flow:**
```
generate.py main()
  ├── argparse (--bpm, --key, --output, --seed, --validate)
  ├── import compose.main
  │   └── compose.main(args) → writes output.rpp
  └── if --validate:
      └── validate_rpp_output(output_path) → list[str]
          ├── count <TRACK> blocks
          ├── regex <SOURCE WAVE> for audio paths → verify exist
          ├── regex NOTE for MIDI clip notes
          ├── compute arrangement end beat
          ├── regex AUXRECV for send routing
          └── count VST/FXCHAIN for plugin chains
```

### Validator: rpp_validator.py

`src/validator/rpp_validator.py` exports `validate_rpp_output(rpp_path: str) -> list[str]`.

Returns `[]` for valid `.rpp`, or a list of error strings for any violation.

**6 structural checks:**
1. **Track count** — must be exactly 9 (`<TRACK` blocks)
2. **Audio clip paths** — all `<SOURCE WAVE "...">` paths must exist on disk
3. **MIDI note presence** — MIDI items must contain at least one `NOTE` event
4. **Arrangement duration** — last item end must be ≥ 52 bars × 4 beats at given BPM
5. **Send routing** — each non-return track must have `AUXRECV` to a return track
6. **Plugin chains** — each non-return track must have `<FXCHAIN>` with `VST` entry

### Perc Loop Fallback

`build_perc_track()` in `compose.py` silently skips missing `91bpm bellako percloop.wav` files. No changes needed — this matches the spec requirement.

---

## ReaScript Hybrid Pipeline (Phase 1 + Phase 2)

### Overview

Two-phase architecture: Phase 1 generates `.rpp` offline; Phase 2 runs inside REAPER via ReaScript for FX verification, track calibration, rendering, and loudness measurement.

```
Phase 1 (offline, Python)          Phase 2 (inside REAPER, ReaScript)
─────────────────────────           ─────────────────────────────────
RPPBuilder.build()                 Main_openProject(rpp_path)
       │                                    │
       ▼                                    ▼
output/song.rpp                   TrackFX_GetCount + TrackFX_GetFXName
                                     → fx_errors (missing plugins)
       │                                    │
       │                           SetMediaTrackInfo_Value(VOLUME/PAN)
       │                           CreateTrackSend for each send
       │                                    │
       │                                    ▼
       │                           Main_RenderFile → output/song.wav
       │                                    │
       │                                    ▼
       │                           CalcMediaSrcLoudness
       │                           → integrated_lufs, short_term_lufs
       │                                    │
       │                                    ▼
       │                           write result.json
       ▼
run_in_reaper.py
  → generate phase2.py
  → write command.json  ──────────────────────────────►
  → poll result.json    ◄──────────────────────────────
  → print LUFS, write fx_errors.json
```

### Bridge: JSON File Protocol

Communication via `fl_control_command.json` / `fl_control_result.json` in REAPER ResourcePath:
- No network dependency
- REAPER owns timing — avoids race conditions
- Human-readable JSON for debugging

### Command JSON Schema

```json
{
  "version": 1,
  "action": "calibrate" | "verify_fx" | "render",
  "rpp_path": "absolute path to .rpp",
  "render_path": "absolute path for rendered output (wav)",
  "timeout": 120,
  "track_calibration": [
    {
      "track_index": 0,
      "volume": 0.85,
      "pan": 0.0,
      "sends": [{"dest_track_index": 5, "level": 0.05}]
    }
  ]
}
```

### Result JSON Schema

```json
{
  "version": 1,
  "status": "ok" | "error" | "timeout",
  "message": "optional error message",
  "lufs": -14.2,
  "integrated_lufs": -14.2,
  "short_term_lufs": -12.1,
  "fx_errors": [{"track_index": 2, "fx_index": 1, "name": "", "expected": "Serum_2"}],
  "tracks_verified": 8
}
```

### Key Files

| File | Role |
|------|------|
| `src/reaper_scripting/__init__.py` | `ReaScriptGenerator` — generates self-contained Python ReaScript |
| `src/reaper_scripting/commands.py` | `ReaScriptCommand`, `ReaScriptResult` dataclasses + protocol |
| `scripts/run_in_reaper.py` | CLI: generate script → write command → poll result → output LUFS |

### ReaScript API Subset (Stable)

`Main_openProject`, `TrackFX_GetCount`, `TrackFX_GetFXName`, `TrackFX_AddByName`, `SetMediaTrackInfo_Value`, `CreateTrackSend`, `Main_RenderFile`, `CalcMediaSrcLoudness`, `GetFunctionMetadata`, and others listed in the spec.

### Architecture Decisions

| Decision | Choice | Rationale |
|----------|--------|-----------|
| JSON file protocol over python-reapy | `fl_control_command.json` / `fl_control_result.json` | No network dependency; REAPER owns timing |
| Self-contained ReaScript (no `import json`) | Hand-rolled JSON parser via string splitting | Maximum REAPER version compatibility |
| Separate `commands.py` for protocol | Protocol isolated from generator | Stable, testable in isolation |
| `track_calibration` JSON array | Stateless interface for volume/pan/sends | Retry-friendly; replay on REAPER crash |

---

## Open Questions

- [ ] `skeleton.py` `EMPTY_SAMPLER_CHANNELS` includes `{17,18,19}` — need to confirm that adding melodic channels beyond 19 doesn't require reference FLP changes or if we expand the sampler clone range.
- [ ] `sample_index.json` entries use `original_path` (absolute Windows paths). CLI on other machines will break. Decision needed: embed relative paths or make selector rebase paths against a configurable `library_root`.
- [ ] Should `render_path` default to the .rpp's folder with `_rendered.wav` suffix?
- [ ] Do we need to handle REAPER's `__startup__.py` registration automatically, or is manual Action registration acceptable for Phase 1?
