reaper-control/.sdd/changes/audio-clip-playlist/design.md

Design: audio-clip-playlist

Context

The current FLPBuilder emits pattern-based playlist items (single 32-byte ArrangementItem) for all tracks. Melodic tracks with source_type="audio" need a different encoding: 3 consecutive 32-byte items per clip, with fixed indices 0x3FF0 / 0x8080 / 0x6440.

---

Architecture Decisions

Decision	Choice	Rationale
Polymorphism	to_items() returns list[bytes] (3 items) vs to_bytes() returns bytes (1 item)	Avoids union-type juggling; builder collects all items uniformly
source_type default	"pattern"	Existing callers unchanged; audio is opt-in
Audio clip item indices	Treat 0x8080/0x6440/0x3FF0 as constants	Per-proposal risk mitigation; verified against audio_clip_reference.flp
ChSamplePath correlation	Audio clip item precedes its ChSamplePath event in channel data	Builder writes items first, then channel events with sample paths already handled by ChannelSkeletonLoader

---

File Changes

File	Action
src/flp_builder/arrangement.py	Add AudioClipItem, constants AUDIO_CLIP_HEADER_INDEX, AUDIO_CLIP_DATA_INDEX
src/flp_builder/schema.py	Add source_type: str = "pattern" field to MelodicTrack
src/flp_builder/builder.py	Route on track.source_type; append AudioClipItem objects to arrangement items list

---

Binary Format

Each audio clip placement = 96 bytes (3 × 32):

Item 0 — PATTERN positioning

struct.pack("<IHHIHH HH 4B ff",
    position,      # bar * ppq * 4
    0x5000,        # pattern_base (distinct from drum pattern_base 0x5000)
    0x5000 + pattern_id,
    length,        # num_bars * ppq * 4
    track_rvidx,   # (max_tracks - 1) - track_index
    0,             # group
    0x0078,
    0x0040,        # flags (no mute for audio)
    64, 100, 128, 128,
    -1.0, -1.0,
)

Item 1 — AUDIO_CLIP_HEADER

00 00 00 00  00 00 F0 3F  00 00 00 00  00 00 00 00
00 03 00 00  00 50 05 00  03 00 00 F3  01 00 00

• item_index = 0x3FF0
• All other bytes constant (observed from reference FLP)

Item 2 — AUDIO_CLIP_DATA

78 00 40 00  40 64 80 80  00 00 80 BF  00 00 80 BF
02 00 00 00  00 00 00 00  00 00 00 00  00 00 00 00

• pattern_base = 0x6440
• item_index = 0x8080
• Remaining bytes constant

---

Data Flow

SongDefinition.melodic_tracks[i]
  ├── source_type == "pattern" → ArrangementItem (existing path)
  └── source_type == "audio"   → AudioClipItem.to_items() [3 × 32 bytes]
                                    │
                                    ▼
                           build_arrangement_section()
                           (all items concatenated into Playlist ID233 data)

---

AudioClipItem Interface

# src/flp_builder/arrangement.py

AUDIO_CLIP_HEADER_INDEX = 0x3FF0   # item_index for header item
AUDIO_CLIP_DATA_INDEX    = 0x8080   # item_index for data item
AUDIO_CLIP_DATA_BASE     = 0x6440   # pattern_base for data item

@dataclass
class AudioClipItem:
    """Playlist item referencing an audio clip via 3×32-byte structure."""

    pattern_id: int       # references the pattern containing audio notes
    bar: float            # start bar (0-based)
    num_bars: float       # length in bars
    track_index: int      # 0-based track index
    muted: bool = False

    def to_items(self, ppq: int = PPQ_DEFAULT, max_tracks: int = MAX_TRACKS_DEFAULT) -> list[bytes]:
        """Return 3 consecutive 32-byte items: [PATTERN, HEADER, DATA]."""
        ...

    # Convenience: to_bytes() returns concatenation for polymorphic use
    def to_bytes(self, ppq: int = PPQ_DEFAULT, max_tracks: int = MAX_TRACKS_DEFAULT) -> bytes:
        return b"".join(self.to_items(ppq, max_tracks))

---

Schema Change

# src/flp_builder/schema.py — MelodicTrack

@dataclass
class MelodicTrack:
    role: str
    sample_path: str
    notes: list[MelodicNote]
    channel_index: int
    volume: float = 0.85
    pan: float = 0.0
    source_type: str = "pattern"   # NEW: "pattern" | "audio"

SongDefinition.validate() also validates:

• source_type in ("pattern", "audio")
• If source_type == "audio": channel_index >= 17

---

Builder Routing

# src/flp_builder/builder.py — _build_arrangement

def _build_arrangement(self, song, track_data_template):
    items: list[ArrangementItem | AudioClipItem] = []

    # Pattern tracks → ArrangementItem (existing)
    for item in song.items:
        items.append(ArrangementItem(
            pattern_id=item.pattern,
            bar=item.bar,
            num_bars=item.bars,
            track_index=item.track - 1,
            muted=item.muted,
        ))

    # Melodic tracks → route on source_type
    drum_pattern_count = len(song.patterns)
    max_drum_track = max((it.track for it in song.items), default=1)

    for i, mt in enumerate(song.melodic_tracks):
        pattern_id = drum_pattern_count + i + 1
        track_index = max_drum_track + i  # 0-based

        if mt.source_type == "audio":
            items.append(AudioClipItem(
                pattern_id=pattern_id,
                bar=mt.bar,          # from MelodicTrack.bar (default 0.0)
                num_bars=mt.num_bars, # from MelodicTrack.num_bars (default 4.0)
                track_index=track_index,
                muted=False,
            ))
        else:  # "pattern"
            items.append(ArrangementItem(
                pattern_id=pattern_id,
                bar=mt.bar or 0.0,
                num_bars=mt.num_bars or 4.0,
                track_index=track_index,
                muted=False,
            ))

    return build_arrangement_section(items, track_data_template, ppq=song.meta.ppq)

Note: MelodicTrack currently has no bar/num_bars fields. The proposal scope does not include adding them; for now audio clip items use bar=0.0, num_bars=4.0 as placeholders until a future change extends MelodicTrack with timeline placement fields.

---

ChSamplePath Correlation

AudioClipItem places a reference on the playlist. The actual sample file path is carried by ChSamplePath events (ID 196) in the channel data, already handled by ChannelSkeletonLoader.load(melodic_map=...) which patches sampler channels with the correct .wav paths.

No additional correlation is needed: the channel index on the melodic track (e.g., ch17) maps to the same channel that carries the ChSamplePath event. The builder ensures channel events precede arrangement events in the FLP binary.

---

Validation Strategy

Binary diff against audio_clip_reference.flp:

1. Generate a minimal FLP with one audio clip item
2. Extract playlist bytes (ID 233 data, after encode_data_event wrapper)
3. Compare first 96 bytes of playlist data against reference
4. Specifically verify:
   • Bytes 0–31: pattern positioning (PATTERN_BASE = 0x5000)
   • Bytes 32–63: header (item_index = 0x3FF0)
   • Bytes 64–95: data (pattern_base = 0x6440, item_index = 0x8080)

If mismatch at 0x8080/0x6440: derive values from reference via build_track_data_template-style extraction, then update constants.

---

Open Questions

Question	Status
MelodicTrack.bar/num_bars fields needed?	Not in scope; placeholders used. Future change to add placement fields.
Index pool (0x8080/0x6440) per-project vs constant?	Treated as constants per proposal risk mitigation. If FLP fails to open, switch to derivation.