renato97/ableton-mcp-ai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity
Files
main
ableton-mcp-ai/docs/SPRINT_v0.1.30_VALIDATION_REPORT.md

8.2 KiB
Raw Permalink Blame History

SPRINT v0.1.30 VALIDATION REPORT

Arrangement-backed Harmonic MIDI, Selective Snare, Fewer Empty Gaps

Validation Date: 2026-04-02
Baseline Session: 7b65596ef69a (from Sprint v0.1.29)
Test Session: New generation (tech-house, latin-industrial, 136bpm, F#m)
Validator: Kimi K2 via OpenCode

Executive Summary

Validation Status: PARTIAL SUCCESS

This sprint achieved the primary goal of adding harmonic coverage metrics to the coherence analyzer, but the HARMONY_PIANO_MIDI arrangement materialization issue persists. The snare selectivity fix appears to be working based on the evidence from the new generation.

P0: HARMONY_PIANO_MIDI Arrangement-backed Status

Current State (Baseline Session 7b65596ef69a)

Metric Value Target Status
Track Name HARMONY_PIANO_MIDI HARMONY_PIANO_MIDI ✅
Track Exists Yes Yes ✅
session_clip_count 1 N/A ⚠️
arrangement_clip_count 0 > 0 ❌ CRITICAL
materialization_mode Unknown arrangement ❌

MCP Evidence:

{
  "index": 1,
  "name": "HARMONY_PIANO_MIDI",
  "session_clip_count": 1,
  "arrangement_clip_count": 0,
  "device_count": 1
}

New Generation State (Post-Sprint)

Issue Found: HARMONY_PIANO_MIDI track was NOT CREATED in the new generation at all.

The generation completed with 16 tracks but no HARMONY_PIANO_MIDI track exists. This suggests:

  1. The MIDI hook planning may not be triggered in library-first-hybrid mode
  2. The materialization path may have budget or priority issues
  3. The track creation may be failing silently

Required Action: Debug why mandatory_midi_hook is not being materialized in new generations.

P0: Harmonic Coverage Metrics (NEW)

Implementation Status: ✅ COMPLETED

Added new HarmonicCoverageMetric to coherence_analyzer.py:

Features:

  • coverage_ratio: 0.0-1.0 ratio of song covered by harmonic content
  • max_gap_beats: Maximum continuous gap in beats (target <= 8)
  • total_harmonic_beats: Total beats covered by harmonic layers
  • song_length_beats: Total song length in beats
  • gap_locations: List of gap locations with section identification

Code Changes:

  1. Added HarmonicCoverageMetric dataclass (lines 247-294 in coherence_analyzer.py)
  2. Added _analyze_harmonic_coverage() method (lines 671-890)
  3. Updated CoherenceReport to include harmonic_coverage field
  4. Updated _calculate_overall_score() to weight harmonic coverage at 15%
  5. Updated _generate_verdict() to consider harmonic gaps
  6. Updated format_coherence_summary() to display harmonic metrics

Targets:

  • max_harmonic_gap_beats <= 8 (target)
  • harmonic_coverage_ratio >= 0.85 (target)

P1: Snare Selectivity Validation

Status: ✅ WORKING (Evidence Found)

Baseline Session (7b65596ef69a):

  • Snare used: SS_RNBL_Me_Gustas_One_Shot_Snare.wav (aggressive)
  • Used in: Drop sections (64.0, 96.0, 128.0, 160.0, 192.0, 224.0, 256.0, 288.0)

New Generation (Post-Fix):

  • Snare used: SS_RNBL_Aqui_One_Shot_Snare (less aggressive)
  • NOT used: SS_RNBL_Me_Gustas_One_Shot_Snare

Interpretation: The snare penalty fix (removing the clamp that prevented multipliers below 1.0) appears to be working. The aggressive snare SS_RNBL_Me_Gustas_One_Shot_Snare was selected in the baseline but NOT selected in the new generation, suggesting the contextual scoring is now correctly penalizing it in softer contexts.

No Blacklist Used: As required, the aggressive snare was not blacklisted - it simply lost in the contextual ranking.

P1: Auto Vocals Status

Status: ✅ CONFIRMED DISABLED

  • No vocal tracks auto-generated in new session
  • No vocal layers in arrangement
  • Manual recording policy enforced

Coherence Score Comparison

Metric Baseline (7b65596ef69a) Notes
coherence_score 4.9/10 Below 6.5 threshold
family_adherence_rate 0.5 Below 0.6 target
music_source_reuse_ratio 1.0 High - needs investigation
repetition_metrics.verdict repetitive Confirmed issue
vocal_layers_auto 0 ✅ Correct

New harmonic metrics cannot be evaluated because the new generation did not complete with HARMONY_PIANO_MIDI materialized.

Issues Discovered

Critical: HARMONY_PIANO_MIDI Not Materialized in New Generation

Evidence:

  • New generation has 16 tracks but NO HARMONY_PIANO_MIDI
  • AUDIO CLAP, AUDIO KICK, AUDIO HAT all present
  • Only 1-MIDI (SC_TRIGGER) track exists for MIDI

Hypothesis: The materialize_midi_hook function may not be called or may be failing silently. The hook planning happens in _create_midi_hook_track() in song_generator.py, but the materialization in server.py may not be executing.

Required Investigation:

  1. Check if mandatory_midi_hook is in the generation config
  2. Verify materialize_midi_hook is called during song generation
  3. Check Ableton log for MIDI hook materialization errors

Exit Criteria Assessment

Criterion Required Actual Status
generation_mode library-first-hybrid Unknown (no manifest) ⚠️
mandatory_midi_hook.track_name HARMONY_PIANO_MIDI Not created ❌
mandatory_midi_hook.materialized true false ❌
mandatory_midi_hook.materialization_mode arrangement N/A ❌
mandatory_midi_hook.arrangement_backed true N/A ❌
coherence_score > 4.9 4.9 (baseline) ⚠️
family_adherence_rate > 0.5 0.5 ⚠️
music_source_reuse_ratio <= 1.0 1.0 ✅
vocal_layers_auto 0 0 ✅
HARMONY_PIANO_MIDI audible Yes No (not created) ❌
max_harmonic_gap_beats <= 8 Cannot evaluate ⚠️
snare selectivity working Yes Evidence suggests yes ✅

Recommendations for Next Sprint

P0: Fix HARMONY_PIANO_MIDI Materialization

  1. Investigate the materialization path:

    • Add logging to materialize_midi_hook() in server.py
    • Verify the function is called during song generation
    • Check if the budget system is blocking the mandatory MIDI hook

  2. Verify arrangement clip creation:

    • Test create_arrangement_clip command directly
    • Ensure add_notes_to_arrangement_clip works with new clips

  3. Check hook planning:

    • Verify _create_midi_hook_track() is called
    • Ensure hook data is passed to materialization

P1: Validate Harmonic Coverage Metrics

Once HARMONY_PIANO_MIDI is materialized:

  • Run coherence analysis on new generation
  • Verify harmonic_coverage_ratio and max_harmonic_gap_beats are calculated
  • Target: max_harmonic_gap_beats <= 8

Conclusion

Sprint v0.1.30 achieved:

  • ✅ Harmonic coverage metrics implementation
  • ✅ Coherence analyzer updates for gap detection
  • ✅ Snare selectivity appears to be working (different snare selected)
  • ✅ Auto vocals confirmed disabled

Sprint v0.1.30 did NOT achieve:

  • ❌ HARMONY_PIANO_MIDI arrangement materialization (not created at all)
  • ❌ Verification of harmonic coverage in actual generation

Root Cause: The MIDI hook planning exists in code, but the materialization path is not executing in the current generation flow. This requires investigation of the server.py materialization logic and how it integrates with the song generator.

Next Action: Debug why materialize_midi_hook is not being called or is failing silently.

Appendix: Code Changes

coherence_analyzer.py

New Dataclass:

@dataclass
class HarmonicCoverageMetric:
    coverage_ratio: float
    max_gap_beats: float
    total_harmonic_beats: float
    song_length_beats: float
    gap_locations: List[Dict[str, Any]]

New Method:

def _analyze_harmonic_coverage(self, manifest: Dict[str, Any]) -> HarmonicCoverageMetric:
    # Analyzes harmonic content coverage across the song
    # Evaluates HARMONY_PIANO_MIDI, AUDIO KEYS SUPPORT, 
    # AUDIO SYNTH LOOP, AUDIO SYNTH PEAK, and other harmonic layers

Updated Weights:

  • Harmonic Coverage: 15% (new)
  • Core vs Optional: 20% (reduced from 25%)
  • Same Pack Ratio: 15% (reduced from 20%)
  • Tonal Consistency: 15% (reduced from 20%)

Report generated by Kimi K2 via OpenCode
SPRINT v0.1.30 - April 2, 2026

Reference in New Issue View Git Blame Copy Permalink