AbletonMCP_AI/QWEN.md

# QWEN.md - AbletonMCP_AI v3.2 (Score → Render)

> **Context**: MCP-based system for controlling Ableton Live 12 from AI agents.
> **Architecture**: Compose-then-Render v3.2 (**STRICT SESSION VIEW**).
> **Team**: Qwen (verify/debug/architecture) + Kimi (fast coding).

## CRITICAL RULES (READ FIRST)

1. **NEVER touch `libreria/` or `librerias/`** - User's sample library. NEVER delete, move, or modify. These are read-only.
2. **NEVER delete project files** - Overwrite, don't delete then create.
3. **NEVER create debug .md files in project root** - All docs go in `AbletonMCP_AI/docs/`.
4. **STRICT SESSION VIEW ONLY** - Arrangement View and its commands (`create_arrangement_*`) are DISCARDED for this sprint. All production goes to scenes and clip slots.
5. **NEVER modify Ableton's built-in scripts** - `_Framework`, `_APC`, `_Komplete_Kontrol`, etc. are not yours.
6. **ALWAYS compile after changes**: `python -m py_compile "<file_path>"`
7. **ALWAYS restart Ableton Live** after changes to `__init__.py` (no hot-reload for Remote Scripts).

## Project Overview

**AbletonMCP_AI** is an AI-powered music production system that lets you create complete professional tracks in Ableton Live using **natural language prompts only**. It uses the Model Context Protocol (MCP) to bridge AI agents with Ableton Live's Python API.

### How It Works

```
AI Agent (OpenCode/Claude/Kimi)
  ↓ Natural language prompts
SongScore Engine (Pure Python Data Model)
  ↓ JSON score representation
Score Renderer (Session View Translator)
  ↓ JSON commands via TCP socket
LiveBridge (TCP → Ableton Live API)
  ↓
Ableton Live 12 Suite → Session View Scenes & Clip Slots
```

### Key Architecture Components

| Component | File | Purpose |
|-----------|------|---------|
| **Remote Script** | `AbletonMCP_AI/__init__.py` | Ableton Control Surface. TCP server on port 9877. Handles all Live API calls. |
| **Score Engine** | `mcp_server/score_engine.py` | [Sprint 9] JSON data model for songs. Decoupled from Ableton logic. |
| **Score Renderer** | `mcp_server/score_renderer.py` | [Sprint 9] Translates JSON Score to Session View Scenes/Clips. |
| **AI Loop** | `mcp_server/ai_loop.py` | [Sprint 9] Autonomous production loop (Anthropic-compatible). |
| **Metadata Store** | `mcp_server/engines/metadata_store.py` | SQLite database of pre-analyzed sample features. No numpy required for queries. |
| **Sample Selector** | `mcp_server/engines/sample_selector.py` | Smart sample selection with coherence scoring. |
| **Mixing Engine** | `mcp_server/engines/mixing_engine.py` | Professional mixing chains (EQ, compression). |
| **LiveBridge** | `mcp_server/engines/live_bridge.py` | Direct Ableton Live API execution engine. |

### Directory Structure

```
MIDI Remote Scripts/
├── AbletonMCP_AI/                 # Main project
│   ├── __init__.py               # Remote Script entry point
│   ├── runtime.py                # TCP server runtime
│   ├── README.md                 # Project documentation
│   ├── docs/                     # Sprints, skills, API reference
│   ├── examples/                 # Usage examples
│   ├── presets/                  # Saved configurations (.json)
│   └── mcp_server/
│       ├── server.py             # MCP FastMCP server (130+ tools)
│       ├── score_engine.py       # SongScore model
│       ├── score_renderer.py     # Session View renderer
│       ├── ai_loop.py            # AI production loop
│       ├── scores/               # [NEW] JSON songs folder
│       └── engines/              # Specialized production engines
├── libreria/                     # User samples (READ-ONLY, git-ignored)
├── librerias/                    # Organized samples (READ-ONLY, git-ignored)
├── mcp_wrapper.py                # MCP server launcher
├── AGENTS.md                     # Agent instructions
├── CLAUDE.md                     # Claude-specific docs
└── QWEN.md                       # This file
```

## Building and Running

### Compile Check (ALWAYS after edits)

```powershell
python -m py_compile "C:\ProgramData\Ableton\Live 12 Suite\Resources\MIDI Remote Scripts\AbletonMCP_AI\__init__.py"
python -m py_compile "C:\ProgramData\Ableton\Live 12 Suite\Resources\MIDI Remote Scripts\AbletonMCP_AI\mcp_server\server.py"
python -m py_compile "C:\ProgramData\Ableton\Live 12 Suite\Resources\MIDI Remote Scripts\mcp_wrapper.py"
```

### Verify Ableton is Listening

```powershell
netstat -an | findstr 9877
```

Expected output: `TCP  127.0.0.1:9877  0.0.0.0:0  LISTENING`

### Test MCP Server Directly

```powershell
python "C:\ProgramData\Ableton\Live 12 Suite\Resources\MIDI Remote Scripts\mcp_wrapper.py"
```

### Restart Ableton (After __init__.py Changes)

1. **Kill all Ableton processes:**
   ```powershell
   Get-Process | Where-Object { $_.ProcessName -like "*Ableton*" } | ForEach-Object { Stop-Process -Id $_.Id -Force }
   ```

2. **Delete recovery files:**
   ```powershell
   # Check both locations
   Remove-Item "$env:APPDATA\Ableton\Live*\Preferences\CrashRecoveryInfo.cfg" -ErrorAction SilentlyContinue
   Remove-Item "$env:LOCALAPPDATA\Ableton\Live*\CrashRecoveryInfo.cfg" -ErrorAction SilentlyContinue
   ```

3. **Start Ableton Live** and verify TCP 9877 is listening.

### OpenCode MCP Configuration

Located in `~/.config/opencode/opencode.json`:

```json
{
  "mcp": {
    "ableton-live-mcp": {
      "type": "local",
      "command": ["python", "C:\\ProgramData\\Ableton\\Live 12 Suite\\Resources\\MIDI Remote Scripts\\mcp_wrapper.py"],
      "enabled": true,
      "timeout": 300000
    }
  }
}
```

### Session View First Workflow (v3.1)

Primary production workflow:

1. **Generate in Session View:**
   ```python
   ableton-live-mcp_produce_13_scenes(
       genre="reggaeton",
       tempo=95,
       key="Am"
   )
   ```

2. **Verify MIDI instruments loaded:**
   ```python
   ableton-live-mcp_validate_session()
   # If needed: ableton-live-mcp_fix_session_midi_tracks()
   ```

3. **Test scenes:**
   ```python
   ableton-live-mcp_fire_scene(scene_index=4)  # Jump to Chorus
   ableton-live-mcp_start_playback()
   ```

4. **Record to Arrangement (manual):**
   - User presses **F9** in Ableton Live
   - Or use: `ableton-live-mcp_record_to_arrangement(duration_bars=70)

## Available MCP Tools (114+)

### Project Info
- `get_session_info` - Tempo, tracks, scenes, playback state
- `get_tracks` / `get_scenes` - List all elements
- `get_arrangement_clips` - Timeline content
- `get_master_info` - Master track settings
- `health_check` - Verify all systems operational

### Transport & Settings
- `start_playback` / `stop_playback` / `toggle_playback`
- `set_tempo` (20-300 BPM) / `set_time_signature` / `set_metronome`

### Tracks & Mixing
- `create_midi_track` / `create_audio_track`
- `set_track_name` / `set_track_volume` / `set_track_pan`
- `set_track_mute` / `set_track_solo`
- `set_master_volume`
- `create_bus_track` / `route_track_to_bus`
- `configure_eq` / `configure_compressor` / `setup_sidechain`

### Clip Creation
- `create_clip` - MIDI clips in Session View
- `add_notes_to_clip` - Add MIDI note data
- `create_arrangement_audio_pattern` - Load audio files to timeline
- `load_sample_to_clip` / `load_sample_to_drum_rack`

### AI Generation (Key Tools)
- `generate_intelligent_track` - One-prompt complete track
- `generate_expansive_track` - 12+ samples per category
- `build_song` - Full arrangement with sections
- `produce_13_scenes` - **Sprint 7**: 13 scenes, 20 tracks, 100+ samples
- `produce_reggaeton` - Complete reggaeton production
- `produce_from_reference` - Match reference audio style

### BPM & Coherence (Sprint 7)
- `analyze_all_bpm` - Analyze 800+ samples with librosa
- `select_bpm_coherent_pool` - Select samples matching target BPM ±tolerance
- `warp_clip_to_bpm` - Auto-warp audio to project tempo (Complex Pro)
- `validate_session` - Verify MIDI tracks have instruments
- `fix_session_midi_tracks` - Auto-load instruments by track name

### Score → Render Pipeline (Sprint 9)
- `new_score` / `get_score` - Score lifecycle
- `compose_from_template` - Quick song generation
- `compose_audio_track` / `compose_midi_track` - Direct composition
- `compose_pattern` - MIDI pattern application
- `save_score` / `load_score` - JSON persistence
- `render_score` - Inject score into Session View (Scene-by-scene)
- `render_all_scores` - Batch autonomous production

See `AbletonMCP_AI/docs/API_REFERENCE_PRO.md` for complete documentation.

## Development Conventions

### Coding Style
- **Python 3.7+** compatible (uses `from __future__ import` for Python 2/3 compatibility in `__init__.py`)
- **All-in-one `__init__.py`** - Ableton's discovery mechanism only reads this file, so all Remote Script code lives here
- **One TCP connection per command** - MCP server opens a new TCP connection to Ableton for each tool call, sends JSON, gets response, closes
- **No `request_refresh()` in `update_display()`** - Causes CPU loop that blocks Ableton

### File Organization
- `__init__.py`: ONLY Ableton Live API code (ControlSurface subclass)
- `mcp_server/server.py`: ONLY MCP tool definitions and TCP client logic
- `mcp_server/engines/`: Music logic (sample selection, generation, mixing)
- **No cross-imports** from `__init__.py` into engines (Ableton's Python environment is isolated)

### Testing Practices
- Always compile-check after edits: `python -m py_compile "<file>"`
- Run `health_check()` after Ableton restart to verify connectivity
- Test new tools individually before integrating
- Use `netstat -an | findstr 9877` to verify TCP port availability

### Error Handling
- **No silent failures** - Errors must be explicit and actionable
- **Musical timing** - All timing uses bars/beats, not wall-clock
- **Coherence scoring** - Sample compatibility threshold at 0.90+

## Sample Library

### Location
- `libreria/` - User's raw samples (git-ignored, READ-ONLY)
- `librerias/` - Organized/analyzed samples (git-ignored, READ-ONLY)

### Expected Structure
```
libreria/reggaeton/
├── kick/
├── snare/
├── hihat/
├── bass/
├── chords/
├── melody/
├── fx/
└── drumloops/
```

### Metadata Store
- SQLite database at `AbletonMCP_AI/mcp_server/engines/sample_metadata.db`
- 800+ total samples (735+ analyzed with BPM, key, spectral features)
- **SentimientoLatino2025 collection**: 658 samples (26 kicks, 26 snares, 34 drumloops, 34 percs, 24 fx, 84 oneshots)
- Librosa-powered BPM analysis for accurate tempo detection
- Spectral embeddings (MFCC) for coherence matching
- Analysis cached on first scan, reused forever

## Key Skills

### Skill 1: Reinicio Correcto de Ableton
**File:** `AbletonMCP_AI/docs/skill_reinicio_ableton.md`

3-step process to cleanly restart Ableton:
1. Kill all Ableton processes
2. Delete recovery files (`CrashRecoveryInfo.cfg`, `CrashDetection.cfg`, `Undo.cfg`)
3. Start Ableton + verify TCP 9877

**When to use:** After modifying `__init__.py`, when changes don't reflect, after crashes.

### Skill 2: Producción Senior de Audio
**File:** `AbletonMCP_AI/docs/skill_produccion_audio.md`

Professional production workflow with 5 automatic injection methods:
- M1: `track.insert_arrangement_clip()` (Live 12+ direct)
- M2: `track.create_audio_clip()` (Live 11+ direct)
- M3: `arrangement_clips.add_new_clip()` (Live 12+ API)
- M4: Session → `duplicate_clip_to_arrangement` (legacy)
- M5: Session → Recording (universal fallback)

**Zero manual configuration** - System chooses automatically.

### Skill 3: Session View Máster (Sprint 7)
**Status:** ✅ Completed 2026-04-13

Complete Session View production system:
- **13 scenes**: Intro → Verse A/B/C → Pre-Chorus → Chorus A/B/C → Bridge → Build Up → Final Chorus → Outro → End
- **20 tracks**: 14 audio + 6 MIDI (Kick layers, Snare layers, Drum Loop, Piano/Chords, Lead, Bass)
- **100+ samples**: Unique per scene with energy-based selection
- **BPM coherence**: Librosa analysis + spectral embeddings
- **Humanization**: Per-instrument profiles with timing/velocity variation
- **Warp automation**: Complex Pro for non-matching samples

**Usage:**
```python
ableton-live-mcp_produce_13_scenes(
    genre="reggaeton",
    tempo=95,
    key="Am",
    auto_play=True
)
# Then press F9 in Ableton to record to Arrangement
```

## EQ and Compressor Presets (Agente 10)

### EQ Presets
| Category | Preset | Description |
|----------|--------|-------------|
| Drums | `kick`, `kick_sub`, `kick_punch` | Kick variations |
| Drums | `snare`, `snare_body`, `snare_crack` | Snare variations |
| Bass | `bass`, `bass_clean`, `bass_dirty` | Bass variations |
| Synth | `synth`, `synth_air`, `pad_warm` | Synth/pad variations |
| Vocal | `vocal_presence` | 3-5kHz presence boost |
| Master | `master`, `master_tame` | Master EQ variations |

### Compressor Presets
| Category | Preset | Description |
|----------|--------|-------------|
| Drums | `kick_punch`, `parallel_drum` | Drum compression |
| Bass | `bass_glue` | Glue compression |
| Vocal | `aggressive_vocal` | Vocal compression |
| Bus | `buss_glue`, `buss_tight`, `glue_light`, `glue_heavy` | Bus compression |
| Master | `master_loud` | Loud master |
| FX | `pumping_sidechain`, `transparent_leveling` | Special effects |

## Known Issues & Workarounds

### Issue 1: MIDI Instrument Loading (Async Timing)
**Status:** ⚠️ Workaround available  
**Problem:** `browser.load_item()` is asynchronous; devices may not appear immediately after call  
**Fix Applied:** Polling loop with 3-second timeout, 15 attempts × 200ms  
**Workaround:** If automatic loading fails, use `insert_device` manually or verify in Ableton UI  
**Note:** Track will show `device_count=0` until instrument actually loads

### Issue 2: analyze_library Cache Attribute
**Status:** ✅ Fixed  
**Problem:** Typo in server.py line 738: `analyzer._cache_file` vs `analyzer.cache_path`  
**Fix:** Corrected to `analyzer.cache_path`  
**Verification:** `analyze_all_bpm` tool now functional

### Issue 3: Drum Loop BPM Mismatch
**Status:** ✅ Auto-handled  
**Problem:** "100bpm gata only drumloop" vs project at 95 BPM  
**Solution:** `warp_clip_to_bpm` automatically applies Complex Pro warp mode  
**Result:** Seamless tempo matching without pitch shift artifacts

## Troubleshooting

| Problem | Solution |
|---------|----------|
| Connection refused | Check Ableton has AbletonMCP_AI loaded in Preferences → Link/Tempo/MIDI → Control Surfaces |
| Port 9877 blocked | Run: `netstat -an \| findstr 9877` |
| Changes not reflecting | Restart Ableton (delete `CrashRecoveryInfo.cfg` first) |
| Sample selection empty | Verify `libreria/reggaeton/` has .wav files |
| Timeout on generation | Check Ableton log for errors |
| MCP server won't start | Run `mcp_wrapper.py` manually to see error output |

## Project Statistics

| Metric | Value |
|--------|-------|
| Total Files | 125+ |
| Lines of Code | ~110,000 |
| Python Engines | 53+ |
| MCP Tools | 114+ |
| Documentation | 32+ pages |
| Sample Library | 800+ total, 735+ analyzed |
| Presets | 7+ saved |
| Sprints Completed | 7 |

## What NOT to Modify

- `libreria/` - User samples (read-only)
- `librerias/` - Organized samples (read-only)
- `_Framework/`, `_APC/`, `_Komplete_Kontrol/`, etc. - Ableton's built-in scripts
- Any directory not under `AbletonMCP_AI/`

## Workflow

**Kimi** codes features → **Qwen** verifies/compiles/debugs/assigns next sprint

All sprints saved to `AbletonMCP_AI/docs/sprint_N_description.md`

---

## 🗺️ Roadmap & Future Work (TODO)

### **Critical Priority (Sprint 8)**

#### 1. MIDI Instrument Loading - Robust Solution
**Status:** ⚠️ Partial - Polling implemented but unreliable  
**Problem:** `browser.load_item()` is async, no callback when device actually loads  
**Current workaround:** 3-second polling loop  
**Needed solution:**
- [ ] Implement device presence verification with retry logic (10 attempts × 500ms)
- [ ] Add fallback: if Wavetable fails, try Operator, then Analog, then Simpler
- [ ] Create "Instrument Rack" preset approach - load rack with default chain
- [ ] Alternative: Use `live.object` API if available for direct device creation
- [ ] Max for Live bridge (last resort) - create M4L device that receives OSC commands

**Acceptance Criteria:**
- `insert_device` returns `device_inserted: true` AND `device_count > 0` in track
- Works for: Wavetable, Operator, Analog, Electric, Tension, Collision
- Max 5 seconds total wait time

#### 2. BPM Analyzer Integration
**Status:** ✅ Engine created, NOT integrated into production pipeline  
**Files ready:** `bpm_analyzer.py`, `spectral_coherence.py`  
**Integration needed:**
- [ ] Run `analyze_all_bpm()` on full library (800 samples) - takes ~30 min
- [ ] Store results in `metadata_store` table `samples_bpm`
- [ ] Modify `produce_13_scenes` to use BPM-coherent samples by default
- [ ] Add `force_bpm_coherence` parameter to all production tools
- [ ] Create `get_bpm_recommendations()` tool for user queries

**Acceptance Criteria:**
- All 800 samples have BPM in database
- Producing at 95 BPM uses only 90-100 BPM samples (±5 tolerance)
- Samples outside tolerance auto-warp with Complex Pro

#### 3. Single Drum Loop Architecture
**Status:** 📝 Planned  
**Current:** Multiple drum loops rotate across scenes  
**Desired:** ONE drum loop stretched 1:30 min + harmony variations  
**Implementation:**
- [ ] Create `extend_loop_to_duration()` function
- [ ] Use `clip.loop_end` to extend without re-triggering
- [ ] Disable sample rotation for drumloop category
- [ ] Add harmony layers (piano, pads) that change per scene
- [ ] Keep drum loop constant, vary harmony/progressions

**Acceptance Criteria:**
- Single drum loop plays continuously for full song duration
- Harmony/progressions change per scene (Intro≠Verse≠Chorus)
- No audible cuts/glitches in drum loop

---

### **High Priority (Sprint 9)**

#### 4. Max for Live Integration (Optional)
**Status:** 📋 Evaluated, not implemented  
**Use case:** If Python `browser.load_item()` remains unreliable  
**Approach:**
- [ ] Create simple M4L device "InstrumentLoader" that listens to OSC
- [ ] Python sends OSC message: `/loadinstrument track_index, instrument_name`
- [ ] M4L device uses `live.object` to insert device directly (more reliable)
- [ ] M4L confirms back via OSC when done

**Pros:** More reliable device insertion  
**Cons:** Requires M4L license, additional complexity  
**Decision:** Only implement if Python solution fails consistently

#### 5. Arrangement Recording Automation
**Status:** 📝 Planned - Currently manual (F9)  
**Goal:** Auto-record Session View to Arrangement  
**Implementation:**
- [ ] `arrangement_overdub` + scene firing + time-based stop
- [ ] Or: `duplicate_clip_to_arrangement` for each clip (if API available)
- [ ] Create `auto_record_session(duration_bars=70)` tool
- [ ] Post-recording: verify all clips appeared in Arrangement

**Current workaround:** User presses F9 manually

---

### **Medium Priority (Backlog)**

#### 6. Advanced Warp Modes
- [ ] Auto-detect best warp mode (Complex Pro vs Beats vs Tones)
- [ ] Per-sample warp configuration stored in metadata
- [ ] Real-time warp quality monitoring

#### 7. Vocal Placeholder Tracks
- [ ] Create empty audio track labeled "VOCALS" for user recording
- [ ] Add sidechain ducking from vocals to music
- [ ] Pre-configure compressor for vocal riding

#### 8. Stem Export Automation
- [ ] `render_stems()` with track groups (Drums, Bass, Music, FX)
- [ ] Individual stems + mixed stem option
- [ ] Naming convention: `ProjectName_StemName.wav`

#### 9. Reference Track Matching
- [ ] Finish `produce_from_reference()` implementation
- [ ] Spectral analysis of reference vs generated
- [ ] Auto-adjust EQ/compression to match reference

#### 10. Batch Production
- [ ] `batch_produce(count=5)` - Generate 5 variations of same prompt
- [ ] Each with different random seed for samples
- [ ] Compare and rank by coherence score

---

### **Bug Fixes Needed**

| Bug | Severity | Status | Notes |
|-----|----------|--------|-------|
| `device_count` stays 0 after `insert_device` | **Critical** | Workaround | Polling helps but not 100% |
| `analyze_library` needs OpenCode restart | Low | Fixed | Cache path typo corrected |
| Humanization needs numpy | Medium | Broken | `apply_human_feel` fails without numpy |
| Time stretch clip API mismatch | Medium | Broken | Signature mismatch in `get_notes` |
| `duplicate_project` renames tracks weirdly | Low | Working | Cosmetic issue only |

---

### **Performance Optimizations**

- [ ] Parallel sample analysis (4 threads for 800 samples)
- [ ] Lazy loading of heavy engines (librosa, sklearn)
- [ ] Cache embeddings as binary blobs not JSON
- [ ] Incremental BPM analysis (only new samples)

---

### **Documentation TODO**

- [ ] Create `docs/sprint_8_midi_loading.md` - Technical deep dive
- [ ] Create `docs/sprint_8_bpm_integration.md` - BPM system guide  
- [ ] Update `API_REFERENCE_PRO.md` with 5 new tools
- [ ] Create troubleshooting guide for MIDI issues
- [ ] Video/gif demos of Session View workflow

---

## Current Sprint Assignment

**Sprint 9 (Active):** Score → Render Pipeline (Compose-then-Render)
**Goal:** 50+ songs generated and rendered autonomously via ai_loop.py
**Status:** ✅ Completed 2026-04-14 (Strict Session View Implementation)

**Key Dev:** Refer to `docs/SYSTEM_SCORE_RENDER.md` for JSON schema and rendering logic.