AbletonMCP_AI/QWEN.md

QWEN.md - AbletonMCP_AI v3.0 (Senior Architecture)

Context: MCP-based system for controlling Ableton Live 12 from AI agents. Architecture: Senior v3.0 (Arrangement-first workflow). Team: Qwen (verify/debug/architecture) + Kimi (fast coding).

CRITICAL RULES (READ FIRST)

1. NEVER touch libreria/ or librerias/ - User's sample library. NEVER delete, move, or modify. These are read-only.
2. NEVER delete project files - Overwrite, don't delete then create.
3. NEVER create debug .md files in project root - All docs go in AbletonMCP_AI/docs/.
4. NEVER use rmdir /s /q except for __pycache__ - Can accidentally delete the whole project.
5. NEVER modify Ableton's built-in scripts - _Framework, _APC, _Komplete_Kontrol, etc. are not yours.
6. ALWAYS compile after changes: python -m py_compile "<file_path>"
7. ALWAYS restart Ableton Live after changes to __init__.py (no hot-reload for Remote Scripts).

Project Overview

AbletonMCP_AI is an AI-powered music production system that lets you create complete professional tracks in Ableton Live using natural language prompts only. It uses the Model Context Protocol (MCP) to bridge AI agents with Ableton Live's Python API.

How It Works

AI Agent (OpenCode/Claude/Kimi)
  ↓ Natural language prompts
MCP Server (FastMCP, stdio transport)
  ↓ JSON commands via TCP socket
50+ Production Engines (drums, bass, melody, mixing, etc.)
  ↓ Real-time clip creation
LiveBridge (TCP → Ableton Live API)
  ↓
Ableton Live 12 Suite → Arrangement View

Key Architecture Components

Component	File	Purpose
Remote Script	AbletonMCP_AI/__init__.py	Ableton Control Surface (~9752 lines). Starts TCP server on port 9877. Handles all Live API calls.
MCP Server	AbletonMCP_AI/mcp_server/server.py	FastMCP server (~6745 lines). Defines 114+ MCP tools. Communicates with Ableton via TCP.
BPM Analyzer	AbletonMCP_AI/mcp_server/engines/bpm_analyzer.py	Librosa-based BPM detection for 800+ samples.
Spectral Coherence	AbletonMCP_AI/mcp_server/engines/spectral_coherence.py	MFCC embeddings for sample similarity.
Session Orchestrator	AbletonMCP_AI/mcp_server/engines/session_orchestrator.py	MIDI instrument validation and auto-loading.
Launcher	mcp_wrapper.py	Entry point for MCP stdio transport. Imports and runs the server.
Integration	AbletonMCP_AI/mcp_server/integration.py	Senior Architecture coordinator. Wires all components together.
LiveBridge	AbletonMCP_AI/mcp_server/engines/live_bridge.py	Direct Ableton Live API execution. Creates clips, writes automation, routes tracks.
Arrangement Recorder	AbletonMCP_AI/mcp_server/engines/arrangement_recorder.py	State machine for Session→Arrangement recording. 7 states, musical quantization.
Metadata Store	AbletonMCP_AI/mcp_server/engines/metadata_store.py	SQLite database of pre-analyzed sample features. No numpy required for queries.
Sample Selector	AbletonMCP_AI/mcp_server/engines/sample_selector.py	Smart sample selection with coherence scoring.
Mixing Engine	AbletonMCP_AI/mcp_server/engines/mixing_engine.py	Professional mixing chains (EQ, compression, bus routing).
Song Generator	AbletonMCP_AI/mcp_server/engines/song_generator.py	Track generation from prompts.

Directory Structure

MIDI Remote Scripts/
├── AbletonMCP_AI/                 # Main project
│   ├── __init__.py               # Remote Script entry point
│   ├── runtime.py                # TCP server runtime
│   ├── README.md                 # Project documentation
│   ├── docs/                     # Sprints, skills, API reference
│   ├── examples/                 # Usage examples
│   ├── presets/                  # Saved configurations (.json)
│   └── mcp_server/
│       ├── server.py             # MCP FastMCP server
│       ├── integration.py        # Senior Architecture coordinator
│       ├── test_arrangement.py   # Verification tests
│       └── engines/              # 65+ production engines
│           ├── sample_selector.py
│           ├── song_generator.py
│           ├── arrangement_recorder.py
│           ├── live_bridge.py
│           ├── mixing_engine.py
│           ├── metadata_store.py
│           ├── massive_selector.py
│           ├── coherence_system.py
│           ├── bpm_analyzer.py          # Sprint 7: Librosa BPM detection
│           ├── spectral_coherence.py    # Sprint 7: MFCC embeddings
│           └── session_orchestrator.py  # Sprint 7: MIDI validation
│           └── ... (50+ more)
├── libreria/                     # User samples (READ-ONLY, git-ignored)
├── librerias/                    # Organized samples (READ-ONLY, git-ignored)
├── mcp_wrapper.py                # MCP server launcher
├── AGENTS.md                     # Agent instructions
├── CLAUDE.md                     # Claude-specific docs
└── QWEN.md                       # This file

Building and Running

Compile Check (ALWAYS after edits)

python -m py_compile "C:\ProgramData\Ableton\Live 12 Suite\Resources\MIDI Remote Scripts\AbletonMCP_AI\__init__.py"
python -m py_compile "C:\ProgramData\Ableton\Live 12 Suite\Resources\MIDI Remote Scripts\AbletonMCP_AI\mcp_server\server.py"
python -m py_compile "C:\ProgramData\Ableton\Live 12 Suite\Resources\MIDI Remote Scripts\mcp_wrapper.py"

Verify Ableton is Listening

netstat -an | findstr 9877

Expected output: TCP 127.0.0.1:9877 0.0.0.0:0 LISTENING

Test MCP Server Directly

python "C:\ProgramData\Ableton\Live 12 Suite\Resources\MIDI Remote Scripts\mcp_wrapper.py"

Restart Ableton (After init.py Changes)

1. Kill all Ableton processes:

   Get-Process | Where-Object { $_.ProcessName -like "*Ableton*" } | ForEach-Object { Stop-Process -Id $_.Id -Force }
   

2. Delete recovery files:

   # Check both locations
   Remove-Item "$env:APPDATA\Ableton\Live*\Preferences\CrashRecoveryInfo.cfg" -ErrorAction SilentlyContinue
   Remove-Item "$env:LOCALAPPDATA\Ableton\Live*\CrashRecoveryInfo.cfg" -ErrorAction SilentlyContinue
   

3. Start Ableton Live and verify TCP 9877 is listening.

OpenCode MCP Configuration

Located in ~/.config/opencode/opencode.json:

{
  "mcp": {
    "ableton-live-mcp": {
      "type": "local",
      "command": ["python", "C:\\ProgramData\\Ableton\\Live 12 Suite\\Resources\\MIDI Remote Scripts\\mcp_wrapper.py"],
      "enabled": true,
      "timeout": 300000
    }
  }
}

Session View First Workflow (v3.1)

Primary production workflow:

1. Generate in Session View:

   ableton-live-mcp_produce_13_scenes(
       genre="reggaeton",
       tempo=95,
       key="Am"
   )
   

2. Verify MIDI instruments loaded:

   ableton-live-mcp_validate_session()
   # If needed: ableton-live-mcp_fix_session_midi_tracks()
   

3. Test scenes:

   ableton-live-mcp_fire_scene(scene_index=4)  # Jump to Chorus
   ableton-live-mcp_start_playback()
   

4. Record to Arrangement (manual):

   • User presses F9 in Ableton Live
   • Or use: `ableton-live-mcp_record_to_arrangement(duration_bars=70)

Available MCP Tools (114+)

Project Info

• get_session_info - Tempo, tracks, scenes, playback state
• get_tracks / get_scenes - List all elements
• get_arrangement_clips - Timeline content
• get_master_info - Master track settings
• health_check - Verify all systems operational

Transport & Settings

• start_playback / stop_playback / toggle_playback
• set_tempo (20-300 BPM) / set_time_signature / set_metronome

Tracks & Mixing

• create_midi_track / create_audio_track
• set_track_name / set_track_volume / set_track_pan
• set_track_mute / set_track_solo
• set_master_volume
• create_bus_track / route_track_to_bus
• configure_eq / configure_compressor / setup_sidechain

Clip Creation

• create_clip - MIDI clips in Session View
• add_notes_to_clip - Add MIDI note data
• create_arrangement_audio_pattern - Load audio files to timeline
• load_sample_to_clip / load_sample_to_drum_rack

AI Generation (Key Tools)

• generate_intelligent_track - One-prompt complete track
• generate_expansive_track - 12+ samples per category
• build_song - Full arrangement with sections
• produce_13_scenes - Sprint 7: 13 scenes, 20 tracks, 100+ samples
• produce_reggaeton - Complete reggaeton production
• produce_from_reference - Match reference audio style

BPM & Coherence (Sprint 7)

• analyze_all_bpm - Analyze 800+ samples with librosa
• select_bpm_coherent_pool - Select samples matching target BPM ±tolerance
• warp_clip_to_bpm - Auto-warp audio to project tempo (Complex Pro)
• validate_session - Verify MIDI tracks have instruments
• fix_session_midi_tracks - Auto-load instruments by track name

Advanced

• create_riser / create_downlifter / create_impact - FX generation
• automate_filter / generate_curve_automation - Parameter automation
• humanize_track - Velocity/timing variations
• apply_professional_mix - Complete mix chain

See AbletonMCP_AI/docs/API_REFERENCE_PRO.md for complete documentation.

Development Conventions

Coding Style

• Python 3.7+ compatible (uses from __future__ import for Python 2/3 compatibility in __init__.py)
• All-in-one __init__.py - Ableton's discovery mechanism only reads this file, so all Remote Script code lives here
• One TCP connection per command - MCP server opens a new TCP connection to Ableton for each tool call, sends JSON, gets response, closes
• No request_refresh() in update_display() - Causes CPU loop that blocks Ableton

File Organization

• __init__.py: ONLY Ableton Live API code (ControlSurface subclass)
• mcp_server/server.py: ONLY MCP tool definitions and TCP client logic
• mcp_server/engines/: Music logic (sample selection, generation, mixing)
• No cross-imports from __init__.py into engines (Ableton's Python environment is isolated)

Testing Practices

• Always compile-check after edits: python -m py_compile "<file>"
• Run health_check() after Ableton restart to verify connectivity
• Test new tools individually before integrating
• Use netstat -an | findstr 9877 to verify TCP port availability

Error Handling

• No silent failures - Errors must be explicit and actionable
• Musical timing - All timing uses bars/beats, not wall-clock
• Coherence scoring - Sample compatibility threshold at 0.90+

Sample Library

Location

• libreria/ - User's raw samples (git-ignored, READ-ONLY)
• librerias/ - Organized/analyzed samples (git-ignored, READ-ONLY)

Expected Structure

libreria/reggaeton/
├── kick/
├── snare/
├── hihat/
├── bass/
├── chords/
├── melody/
├── fx/
└── drumloops/

Metadata Store

• SQLite database at AbletonMCP_AI/mcp_server/engines/sample_metadata.db
• 800+ total samples (735+ analyzed with BPM, key, spectral features)
• SentimientoLatino2025 collection: 658 samples (26 kicks, 26 snares, 34 drumloops, 34 percs, 24 fx, 84 oneshots)
• Librosa-powered BPM analysis for accurate tempo detection
• Spectral embeddings (MFCC) for coherence matching
• Analysis cached on first scan, reused forever

Key Skills

Skill 1: Reinicio Correcto de Ableton

File: AbletonMCP_AI/docs/skill_reinicio_ableton.md

3-step process to cleanly restart Ableton:

1. Kill all Ableton processes
2. Delete recovery files (CrashRecoveryInfo.cfg, CrashDetection.cfg, Undo.cfg)
3. Start Ableton + verify TCP 9877

When to use: After modifying __init__.py, when changes don't reflect, after crashes.

Skill 2: Producción Senior de Audio

File: AbletonMCP_AI/docs/skill_produccion_audio.md

Professional production workflow with 5 automatic injection methods:

• M1: track.insert_arrangement_clip() (Live 12+ direct)
• M2: track.create_audio_clip() (Live 11+ direct)
• M3: arrangement_clips.add_new_clip() (Live 12+ API)
• M4: Session → duplicate_clip_to_arrangement (legacy)
• M5: Session → Recording (universal fallback)

Zero manual configuration - System chooses automatically.

Skill 3: Session View Máster (Sprint 7)

Status: ✅ Completed 2026-04-13

Complete Session View production system:

• 13 scenes: Intro → Verse A/B/C → Pre-Chorus → Chorus A/B/C → Bridge → Build Up → Final Chorus → Outro → End
• 20 tracks: 14 audio + 6 MIDI (Kick layers, Snare layers, Drum Loop, Piano/Chords, Lead, Bass)
• 100+ samples: Unique per scene with energy-based selection
• BPM coherence: Librosa analysis + spectral embeddings
• Humanization: Per-instrument profiles with timing/velocity variation
• Warp automation: Complex Pro for non-matching samples

Usage:

ableton-live-mcp_produce_13_scenes(
    genre="reggaeton",
    tempo=95,
    key="Am",
    auto_play=True
)
# Then press F9 in Ableton to record to Arrangement

EQ and Compressor Presets (Agente 10)

EQ Presets

Category	Preset	Description
Drums	kick, kick_sub, kick_punch	Kick variations
Drums	snare, snare_body, snare_crack	Snare variations
Bass	bass, bass_clean, bass_dirty	Bass variations
Synth	synth, synth_air, pad_warm	Synth/pad variations
Vocal	vocal_presence	3-5kHz presence boost
Master	master, master_tame	Master EQ variations

Compressor Presets

Category	Preset	Description
Drums	kick_punch, parallel_drum	Drum compression
Bass	bass_glue	Glue compression
Vocal	aggressive_vocal	Vocal compression
Bus	buss_glue, buss_tight, glue_light, glue_heavy	Bus compression
Master	master_loud	Loud master
FX	pumping_sidechain, transparent_leveling	Special effects

Known Issues & Workarounds

Issue 1: MIDI Instrument Loading (Async Timing)

Status: ⚠️ Workaround available
Problem: browser.load_item() is asynchronous; devices may not appear immediately after call
Fix Applied: Polling loop with 3-second timeout, 15 attempts × 200ms
Workaround: If automatic loading fails, use insert_device manually or verify in Ableton UI
Note: Track will show device_count=0 until instrument actually loads

Issue 2: analyze_library Cache Attribute

Status: ✅ Fixed
Problem: Typo in server.py line 738: analyzer._cache_file vs analyzer.cache_path
Fix: Corrected to analyzer.cache_path
Verification: analyze_all_bpm tool now functional

Issue 3: Drum Loop BPM Mismatch

Status: ✅ Auto-handled
Problem: "100bpm gata only drumloop" vs project at 95 BPM
Solution: warp_clip_to_bpm automatically applies Complex Pro warp mode
Result: Seamless tempo matching without pitch shift artifacts

Troubleshooting

Problem	Solution
Connection refused	Check Ableton has AbletonMCP_AI loaded in Preferences → Link/Tempo/MIDI → Control Surfaces
Port 9877 blocked	Run: netstat -an | findstr 9877
Changes not reflecting	Restart Ableton (delete CrashRecoveryInfo.cfg first)
Sample selection empty	Verify libreria/reggaeton/ has .wav files
Timeout on generation	Check Ableton log for errors
MCP server won't start	Run mcp_wrapper.py manually to see error output

Project Statistics

Metric	Value
Total Files	125+
Lines of Code	~110,000
Python Engines	53+
MCP Tools	114+
Documentation	32+ pages
Sample Library	800+ total, 735+ analyzed
Presets	7+ saved
Sprints Completed	7

What NOT to Modify

• libreria/ - User samples (read-only)
• librerias/ - Organized samples (read-only)
• _Framework/, _APC/, _Komplete_Kontrol/, etc. - Ableton's built-in scripts
• Any directory not under AbletonMCP_AI/

Workflow

Kimi codes features → Qwen verifies/compiles/debugs/assigns next sprint

All sprints saved to AbletonMCP_AI/docs/sprint_N_description.md

---

🗺️ Roadmap & Future Work (TODO)

Critical Priority (Sprint 8)

1. MIDI Instrument Loading - Robust Solution

Status: ⚠️ Partial - Polling implemented but unreliable
Problem: browser.load_item() is async, no callback when device actually loads
Current workaround: 3-second polling loop
Needed solution:

• Implement device presence verification with retry logic (10 attempts × 500ms)
• Add fallback: if Wavetable fails, try Operator, then Analog, then Simpler
• Create "Instrument Rack" preset approach - load rack with default chain
• Alternative: Use live.object API if available for direct device creation
• Max for Live bridge (last resort) - create M4L device that receives OSC commands

Acceptance Criteria:

• insert_device returns device_inserted: true AND device_count > 0 in track
• Works for: Wavetable, Operator, Analog, Electric, Tension, Collision
• Max 5 seconds total wait time

2. BPM Analyzer Integration

Status: ✅ Engine created, NOT integrated into production pipeline
Files ready: bpm_analyzer.py, spectral_coherence.py
Integration needed:

• Run analyze_all_bpm() on full library (800 samples) - takes ~30 min
• Store results in metadata_store table samples_bpm
• Modify produce_13_scenes to use BPM-coherent samples by default
• Add force_bpm_coherence parameter to all production tools
• Create get_bpm_recommendations() tool for user queries

Acceptance Criteria:

• All 800 samples have BPM in database
• Producing at 95 BPM uses only 90-100 BPM samples (±5 tolerance)
• Samples outside tolerance auto-warp with Complex Pro

3. Single Drum Loop Architecture

Status: 📝 Planned
Current: Multiple drum loops rotate across scenes
Desired: ONE drum loop stretched 1:30 min + harmony variations
Implementation:

• Create extend_loop_to_duration() function
• Use clip.loop_end to extend without re-triggering
• Disable sample rotation for drumloop category
• Add harmony layers (piano, pads) that change per scene
• Keep drum loop constant, vary harmony/progressions

Acceptance Criteria:

• Single drum loop plays continuously for full song duration
• Harmony/progressions change per scene (Intro≠Verse≠Chorus)
• No audible cuts/glitches in drum loop

---

High Priority (Sprint 9)

4. Max for Live Integration (Optional)

Status: 📋 Evaluated, not implemented
Use case: If Python browser.load_item() remains unreliable
Approach:

• Create simple M4L device "InstrumentLoader" that listens to OSC
• Python sends OSC message: /loadinstrument track_index, instrument_name
• M4L device uses live.object to insert device directly (more reliable)
• M4L confirms back via OSC when done

Pros: More reliable device insertion
Cons: Requires M4L license, additional complexity
Decision: Only implement if Python solution fails consistently

5. Arrangement Recording Automation

Status: 📝 Planned - Currently manual (F9)
Goal: Auto-record Session View to Arrangement
Implementation:

• arrangement_overdub + scene firing + time-based stop
• Or: duplicate_clip_to_arrangement for each clip (if API available)
• Create auto_record_session(duration_bars=70) tool
• Post-recording: verify all clips appeared in Arrangement

Current workaround: User presses F9 manually

---

Medium Priority (Backlog)

6. Advanced Warp Modes

• Auto-detect best warp mode (Complex Pro vs Beats vs Tones)
• Per-sample warp configuration stored in metadata
• Real-time warp quality monitoring

7. Vocal Placeholder Tracks

• Create empty audio track labeled "VOCALS" for user recording
• Add sidechain ducking from vocals to music
• Pre-configure compressor for vocal riding

8. Stem Export Automation

• render_stems() with track groups (Drums, Bass, Music, FX)
• Individual stems + mixed stem option
• Naming convention: ProjectName_StemName.wav

9. Reference Track Matching

• Finish produce_from_reference() implementation
• Spectral analysis of reference vs generated
• Auto-adjust EQ/compression to match reference

10. Batch Production

• batch_produce(count=5) - Generate 5 variations of same prompt
• Each with different random seed for samples
• Compare and rank by coherence score

---

Bug Fixes Needed

Bug	Severity	Status	Notes
device_count stays 0 after insert_device	Critical	Workaround	Polling helps but not 100%
analyze_library needs OpenCode restart	Low	Fixed	Cache path typo corrected
Humanization needs numpy	Medium	Broken	apply_human_feel fails without numpy
Time stretch clip API mismatch	Medium	Broken	Signature mismatch in get_notes
duplicate_project renames tracks weirdly	Low	Working	Cosmetic issue only

---

Performance Optimizations

• Parallel sample analysis (4 threads for 800 samples)
• Lazy loading of heavy engines (librosa, sklearn)
• Cache embeddings as binary blobs not JSON
• Incremental BPM analysis (only new samples)

---

Documentation TODO

• Create docs/sprint_8_midi_loading.md - Technical deep dive
• Create docs/sprint_8_bpm_integration.md - BPM system guide
• Update API_REFERENCE_PRO.md with 5 new tools
• Create troubleshooting guide for MIDI issues
• Video/gif demos of Session View workflow

---

Current Sprint Assignment

Sprint 8 (Active): MIDI Instrument Loading + BPM Integration
Owner: Qwen + Kimi
Goal: MIDI tracks sound without manual intervention
Deadline: TBD (user decides priority)

Next: Sprint 9 (Max for Live or Arrangement Recording)