renato97/free-ide-proxy
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

feat(proxy): auto-discovery de modelos + backend Kimchi

Browse Source 
- Nueva interfaz Discoverer con Refresh() por backend
- Kilo: full auto (isFree + metadata completa del upstream)
- OpenCode: heurística -free para descubrir nuevos free
- Kimchi: agrega todos los IDs no-curados del upstream
- discoveryLoop cada 10 min en el Proxy
- Backends stateful (mutex + catálogo dinámico)
- CREDENCIALES.md y KIMCHI_ANALISIS.md documentan el setup
This commit is contained in:
renato97
2026-07-12 19:58:30 -03:00
parent 05a52d29e2
commit 46127431a3
8 changed files with 1041 additions and 29 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CREDENCIALES.md
+136
View File
@@ -0,0 +1,136 @@
# Free IDE Proxy — Credenciales y Modelos
## Endpoint
```
http://127.0.0.1:6446
```
## Clientes compatibles
Cualquier cliente OpenAI o Anthropic. Apunta la base URL a `http://127.0.0.1:6446`.
### Claude Code
```bash
export ANTHROPIC_BASE_URL="http://127.0.0.1:6446"
export ANTHROPIC_MODEL="claude-sonnet-4-6" # se mapea a deepseek-v4-flash-free
```
Claude Code agrega `/v1/messages` automáticamente. No pongas `/v1` en la base URL.
### OpenAI SDK
```python
from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:6446/v1", api_key="...")
response = client.chat.completions.create(
model="deepseek-v4-flash-free",
messages=[{"role": "user", "content": "Hello"}]
)
```
### curl
```bash
curl http://127.0.0.1:6446/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "deepseek-v4-flash-free",
"messages": [{"role": "user", "content": "Hello"}],
"stream": true
}'
```
```bash
curl http://127.0.0.1:6446/v1/messages \
-H "Content-Type: application/json" \
-d '{
"model": "deepseek-v4-flash-free",
"max_tokens": 1000,
"messages": [{"role": "user", "content": "Hello"}]
}'
```
## Autenticación
Actualmente: **DESHABILITADA** (sin API key).
Para activar:
```bash
# Por flag
./free-ide-proxy -api-key "tu-clave-secreta"
# O por env var
export OPENCODE_PROXY_KEY="tu-clave-secreta"
```
Headers aceptados: `Authorization: Bearer <key>` o `x-api-key: <key>`
---
## Modelos disponibles
> **Auto-discovery activo.** El proxy descubre modelos automáticamente cada 10 min desde los `/models` de cada upstream. La lista de abajo es la **base curada** (metadata confiable + aliases); los modelos adicionales descubiertos aparecen en `/v1/models` en runtime. Consultá siempre el endpoint para la lista vigente:
> ```bash
> curl -s http://127.0.0.1:6446/v1/models | python3 -m json.tool
> ```
>
> **Estrategia por backend:**
> - **Kilo** → full auto: parsea `isFree:true` + metadata completa (ctx, max_tokens, vision). Modelos nuevos aparecen solos.
> - **OpenCode** → heurística: curados + cualquier ID con token `free` (sufijo `-free`/`:free`).
> - **Kimchi** → todos los IDs del upstream (cuenta por crédito, todos usables).
### OpenCode Zen (`opencode.ai/zen/v1`)
| Modelo ID | Aliases | Contexto | Output | Tags |
|-----------|---------|----------|--------|------|
| `deepseek-v4-flash-free` | `deepseek`, `deepseek-v4`, `ds` | 1M | 384K | ⭐ recomendado, razonamiento, propósito general |
| `big-pickle` | `pickle` | 200K | 32K | legacy, refactors, mantenimiento |
| `mimo-v2.5-free` | `mimo`, `mimo-v2.5`, `xiaomi` | 1M | 32K | multimodal, imágenes → código, MIT |
| `north-mini-code-free` | `north`, `north-mini`, `cohere` | 256K | 64K | respuesta rápida, scripts |
| `nemotron-3-ultra-free` | `nemotron`, `nemotron-3`, `nvidia` | 1M | 16K | razonamiento, Mamba-2 + MoE |
### Kilo Gateway (`api.kilo.ai/api/gateway`)
| Modelo ID | Aliases | Contexto | Output | Tags |
|-----------|---------|----------|--------|------|
| `stepfun/step-3.7-flash:free` | `stepfun`, `stepfun-free` | 256K | 32K | flash, tareas generales |
| `poolside/laguna-m.1:free` | `poolside`, `poolside-free`, `laguna` | 256K | 32K | código |
| `nvidia/nemotron-3-ultra-550b-a55b:free` | — | 1M | 16K | razonamiento, MoE |
| `openrouter/free` | `openrouter` | 256K | 32K | selección automática |
> Kilo además **auto-descubre** modelos free adicionales (ej: `kilo-auto/free`, `tencent/hy3:free`, `cohere/north-mini-code:free`, `nvidia/nemotron-3-super-120b-a12b:free`, etc.) con metadata completa del upstream.
### Kimchi Dev (`llm.kimchi.dev/openai/v1`)
| Modelo ID | Aliases | Contexto | Output | Tags |
|-----------|---------|----------|--------|------|
| `deepseek-v4-flash` | `kimchi`, `kimchi/deepseek` | 1M | 1M | ⭐ rápido, razonamiento |
| `glm-5.2-fp8` | `kimchi/glm` | 1M | 1M | GLM, FP8 |
| `kimi-k2.7` | `kimchi/kimi` | 262K | 262K | Moonshot, visión |
| `minimax-m3` | `kimchi/minimax` | 1M | 1M | MiniMax, visión |
| `nemotron-3-ultra-fp4` | `kimchi/nemotron` | 1M | 1M | NVIDIA, FP4 |
> Kimchi auto-descubre **todos** los IDs del upstream (cuenta por crédito). Adicionales típicos: `qwen3-coder-next-fp8`, `nemotron-3-super-fp4`, `kimi-k2.5/k2.6`, `minimax-m2.5/m2.7`, `smollm2-*`. Requiere header `User-Agent: kimchi/0.1.50` (sino el upstream devuelve 402).
### Alias de modelos Claude
| Alias Claude | Se resuelve a |
|--------------|---------------|
| `claude-sonnet-4-6` | `deepseek-v4-flash-free` |
| `claude-sonnet-4` | `deepseek-v4-flash-free` |
| `claude-3.5-sonnet` | `deepseek-v4-flash-free` |
| `claude-3-haiku` | `deepseek-v4-flash-free` |
| `claude-opus` | `deepseek-v4-flash-free` |
## Notas
- **Auto-discovery**: cada 10 min el proxy refresca el catálogo desde los `/models` de cada upstream. Los modelos descubiertos se agregan a `/v1/models` en runtime; la base curada (metadata + aliases) siempre se preserva.
- **Razonamiento**: DeepSeek, Nemotron y StepFun necesitan `max_tokens >= 500`. Con valores chicos devuelven contenido vacío.
- **Rate limiting**: si un modelo devuelve 429, se oculta de `/v1/models` y se reintenta cada 30 min.
- **Modelos nuevos**: los descubiertos rutean a su backend correcto automáticamente. Los totalmente desconocidos se reenvían al backend default (OpenCode Zen) como pass-through.
- **Session rotation**: las sesiones rotan cada 30 min.
- **Claude Code**: el proxy también acepta `/v1/v1/messages` por el bug de doble path.
KIMCHI_ANALISIS.md
+275
View File
@@ -0,0 +1,275 @@
# Kimchi Dev — Análisis Completo para Integración en Free IDE Proxy
## 1. Instalación
```bash
which kimchi
# /home/ren/.local/bin/kimchi
```
## 2. Configuración Local
### Archivos relevantes
| Archivo | Propósito |
|---------|-----------|
| `~/.config/kimchi/config.json` | Config principal: apiKey, skillPaths, deviceId, onboarding |
| `~/.config/kimchi/harness/models.json` | Definiciones de proveedores y modelos |
| `~/.config/kimchi/harness/settings.json` | Settings: defaultProvider, defaultModel, multiModel, etc. |
| `~/.config/kimchi/harness/auth.json` | Auth por proveedor (oauth, access token) |
### `~/.config/kimchi/config.json`
```json
{
"skillPaths": [
".config/kimchi/harness/skills",
".config/opencode/skills"
],
"migrationState": "done",
"apiKey": "castai_v1_befd8c666ce652d47cc08accf8c5accc0a5aaca9e1f006392df69ac123198f9b_4f8f2d40",
"onboarding": {
"sessionModeWizardSeenAt": "2026-06-29T13:46:41.351Z"
},
"deviceId": "9fb28c0e-dd65-4bac-8c90-c98ac1c26382"
}
```
### `~/.config/kimchi/harness/auth.json`
```json
{
"kimchi-dev": {
"type": "oauth",
"access": "castai_v1_befd8c666ce652d47cc08accf8c5accc0a5aaca9e1f006392df69ac123198f9b_4f8f2d40",
"refresh": "",
"expires": 9007199254740991
}
}
```
### `~/.config/kimchi/harness/settings.json`
```json
{
"quietStartup": true,
"theme": "kimchi-minimal",
"retry": { "maxRetries": 10 },
"lastChangelogVersion": "0.1.50",
"defaultProvider": "kimchi-dev",
"defaultModel": "kimi-k2.7",
"defaultThinkingLevel": "medium",
"multiModel": true,
"rtkAutoInstallCheckedAt": 1782740800268
}
```
## 3. Endpoint del API
### Configuración del proveedor (desde `models.json`)
```json
"kimchi-dev": {
"baseUrl": "https://llm.kimchi.dev/openai/v1",
"apiKey": "$KIMCHI_API_KEY",
"api": "openai-completions",
"authHeader": true,
"headers": {
"User-Agent": "kimchi/0.1.50"
},
"models": [
...
]
}
```
### Resumen del endpoint
| Campo | Valor |
|-------|-------|
| **Base URL** | `https://llm.kimchi.dev/openai/v1` |
| **Chat Completions** | `POST https://llm.kimchi.dev/openai/v1/chat/completions` |
| **Models listing** | `GET https://llm.kimchi.dev/openai/v1/models` |
| **API format** | OpenAI Chat Completions estándar |
| **Auth** | `Authorization: Bearer <access_token>` (`authHeader: true`) |
| **User-Agent** | `kimchi/0.1.50` |
| **Provider upstream** | castai (ai-enabler) |
### Auth token
- **Access token (hardcodeado):** `castai_v1_befd8c666ce652d47cc08accf8c5accc0a5aaca9e1f006392df69ac123198f9b_4f8f2d40`
- **Tipo:** oauth
- **Expira:** `9007199254740991` (nunca, es un token estático)
- **Env var equivalente:** `KIMCHI_API_KEY` (el provider usa `${KIMCHI_API_KEY}` en `apiKey`)
### Verificación con curl
```bash
# Listar modelos
curl -s https://llm.kimchi.dev/openai/v1/models \
-H "Authorization: Bearer castai_v1_befd8c666ce652d47cc08accf8c5accc0a5aaca9e1f006392df69ac123198f9b_4f8f2d40" \
-H "User-Agent: kimchi/0.1.50"
# Chat completion
curl -s https://llm.kimchi.dev/openai/v1/chat/completions \
-H "Authorization: Bearer castai_v1_befd8c666ce652d47cc08accf8c5accc0a5aaca9e1f006392df69ac123198f9b_4f8f2d40" \
-H "User-Agent: kimchi/0.1.50" \
-H "Content-Type: application/json" \
-d '{
"model": "deepseek-v4-flash",
"messages": [{"role": "user", "content": "Decime solo SI"}],
"max_tokens": 500
}'
```
## 4. Modelos
### Categorización
Hay **13 modelos** que responde el upstream, pero **5 están curados** (con metadata completa en `models.json`). Los otros 8 aparecen en `/models` pero no tienen definición local.
### Los 5 curados (metadata de `models.json`)
| Modelo ID | Contexto | Max Tokens | Razonamiento | Vision | Tags |
|-----------|----------|-----------|:---:|:---:|------|
| `deepseek-v4-flash` | 1,048,576 (1M) | 1,048,576 (1M) | ✅ Sí | ❌ | DeepSeek V4, 284B params, propósito general |
| `glm-5.2-fp8` | 1,048,576 (1M) | 1,048,576 (1M) | ✅ Sí | ❌ | GLM, FP8 cuantizado, rendimiento |
| `kimi-k2.7` | 262,144 (262K) | 262,144 (262K) | ✅ Sí | ✅ Sí | Moonshot AI, multimodal, visión |
| `minimax-m3` | 1,048,576 (1M) | 1,048,576 (1M) | ✅ Sí | ✅ Sí | MiniMax, multimodal, texto+imagen |
| `nemotron-3-ultra-fp4` | 1,048,576 (1M) | 1,048,576 (1M) | ✅ Sí | ❌ | NVIDIA, 550B params, FP4 |
Costos: todos gratuitos (0 input, 0 output, 0 cache).
### Los 8 extras del upstream (sin metadata)
| Modelo ID | Notas |
|-----------|-------|
| `kimi-k2.5` | Versión anterior de kimi |
| `kimi-k2.6` | Versión anterior de kimi |
| `minimax-m2.5` | Versión anterior de minimax |
| `minimax-m2.7` | Versión anterior de minimax |
| `nemotron-3-super-fp4` | Variante "super" de nemotron, posiblemente más grande |
| `qwen3-coder-next-fp8` | 🔥 **Interesante**: modelo de código Qwen, FP8 |
| `smollm2-135m` | Toy — 135M params |
| `smollm2-360m` | Toy — 360M params |
### IDs en el upstream
El upstream responde con estos IDs exactos. No tienen sufijo `-free` (a diferencia de opencode). Esto evita colisiones con los modelos existentes del proxy.
### Comparativa con modelos existentes del proxy
| Modelo | En opencode backend | En kimchi backend |
|--------|:-:|:-:|
| deepseek-v4 | `deepseek-v4-flash-free` | `deepseek-v4-flash` (sin `-free`, distinto) |
| nemotron-3 | `nemotron-3-ultra-free` | `nemotron-3-ultra-fp4` (distinto, cuantizado FP4) |
Son IDs diferentes, no hay colisión.
## 5. Estructura del Proxy Relevante para Integración
### Interfaz Backend (`internal/proxy/backend.go`)
```go
type Backend interface {
Name() string
Models() []ModelInfo
Resolve(model string) (resolved string, ok bool)
ChatURL() string
Headers(requestID, sessionID string) map[string]string
}
```
### Registro en `NewProxy` (`internal/proxy/models.go:61`)
```go
func NewProxy(apiKey string) *Proxy {
oc := OpenCodeBackend{}
kl := KiloBackend{}
// KimchiBackend{} acá
p := &Proxy{
backends: []Backend{kl, oc}, // ← agregar KimchiBackend
...
}
}
```
### Orden de backends
Más específicos primero, para que rutas exactas ganen sobre alias:
```
KimchiBackend → KiloBackend → OpenCodeBackend (default)
```
### Patrón de implementación: `kilo.go` (plantilla exacta)
`kilo.go` es el template perfecto:
1. Const con la base URL
2. Struct vacío `KiloBackend struct{}`
3. Slice `kiloFreeModels []ModelInfo` con los modelos
4. Map `kiloAliases map[string]string`
5. Implementar `Name()`, `Models()`, `Resolve()`, `ChatURL()`, `Headers()`
### Manejo del API key
3 opciones identificadas:
| Opción | Cómo | Pros | Contras |
|--------|------|------|---------|
| **A) Env var `KIMCHI_API_KEY`** | `os.Getenv("KIMCHI_API_KEY")` en `Headers()` | Sin secretos en git, clave en systemd `Environment=` | Requiere configurar env var |
| **B) Hardcodear** | Const string igual que `"public"` en opencode | Simple | **La key queda en el repo** |
| **C) Leer config.json** | Parsear `~/.config/kimchi/config.json` en startup | Auto, no requiere config extra | Acopla proxy a ruta fija del disco |
**Recomendación: Opción A (env var)**. La key se configura solo en `~/.config/systemd/user/free-ide-proxy.service`, fuera del repo.
### `ModelByID` y routing de visión
El proxy tiene `ModelByID()` en `opencode.go` que solo revisa `zenModels`. Si se integran modelos con visión de kimchi (`kimi-k2.7`, `minimax-m3`), `ModelByID` no los encontrará. Habría que extender `ModelByID()` para que revise todos los backends, o crear un map global por ID.
## 6. Posibles Alias para el Backend Kimchi
Cuidando de no pisar alias existentes de opencode/kilo:
| Alias propuesto | Resuelve a |
|----------------|------------|
| `kimchi/deepseek` | `deepseek-v4-flash` |
| `kimchi/glm` | `glm-5.2-fp8` |
| `kimchi/kimi` | `kimi-k2.7` |
| `kimchi/minimax` | `minimax-m3` |
| `kimchi/nemotron` | `nemotron-3-ultra-fp4` |
| `kimchi/qwen` | `qwen3-coder-next-fp8` |
| `kimchi` | `deepseek-v4-flash` (alias default) |
Sin alias cortos sueltos (`deepseek`, `nemotron`) para no pisar los de opencode.
## 7. Archivos a Modificar
| Archivo | Acción |
|---------|--------|
| `internal/proxy/kimchi.go` | **CREAR** — backend completo |
| `internal/proxy/models.go` | **EDITAR** — registrar `KimchiBackend{}` en `NewProxy()` |
| `internal/proxy/opencode.go` | **EDITAR** — extender `ModelByID()` global o crear `AllModelsByID()` |
| `~/.config/systemd/user/free-ide-proxy.service` | **EDITAR** — agregar `Environment=KIMCHI_API_KEY=...` |
| `CREDENCIALES.md` | **EDITAR** — agregar sección de kimchi |
## 8. Comandos de Build y Deploy
```bash
# Build
cd /home/ren/ide_proxy
go build -o free-ide-proxy .
# Restart service
systemctl --user daemon-reload
systemctl --user restart free-ide-proxy
# Check logs
journalctl --user -u free-ide-proxy -f
# Test
curl -s http://127.0.0.1:6446/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model":"deepseek-v4-flash","messages":[{"role":"user","content":"Hola"}],"max_tokens":500}'
```
internal/proxy/anthropic.go
+4 -1
View File
@@ -665,7 +665,10 @@ func (p *Proxy) ForwardAnthropicMessages(w http.ResponseWriter, r *http.Request,
// a vision-capable free model). Kilo models are forwarded as-is. // a vision-capable free model). Kilo models are forwarded as-is.
hasImages := requestHasImages(&ar) hasImages := requestHasImages(&ar)
if hasImages && backend.Name() == "opencode" { if hasImages && backend.Name() == "opencode" {
mi := ModelByID(resolved) var mi *ModelInfo
if oc, ok := backend.(*OpenCodeBackend); ok {
mi = oc.ModelByID(resolved)
}
if mi == nil || !mi.SupportsVision { if mi == nil || !mi.SupportsVision {
fmt.Printf("[anthropic] model=%s lacks vision, auto-routing to mimo-v2.5-free\n", resolved) fmt.Printf("[anthropic] model=%s lacks vision, auto-routing to mimo-v2.5-free\n", resolved)
resolved = "mimo-v2.5-free" resolved = "mimo-v2.5-free"
internal/proxy/backend.go
+15
View File
@@ -11,6 +11,8 @@ type Backend interface {
Name() string Name() string
// Models returns the free models this backend exposes, for /v1/models listing. // Models returns the free models this backend exposes, for /v1/models listing.
// Implementations may return a dynamic (discovered) catalogue; before the
// first successful Refresh() they fall back to the curated list.
Models() []ModelInfo Models() []ModelInfo
// Resolve maps a client-requested model name to the upstream model ID. // Resolve maps a client-requested model name to the upstream model ID.
@@ -25,3 +27,16 @@ type Backend interface {
// request to the upstream (auth, client-id, request/session tracking, etc.). // request to the upstream (auth, client-id, request/session tracking, etc.).
Headers(requestID, sessionID string) map[string]string Headers(requestID, sessionID string) map[string]string
} }
// Discoverer is an optional capability a Backend may implement to
// auto-discover its model catalogue from the upstream /models endpoint.
//
// The Proxy runs a discovery loop that calls Refresh() periodically. Each
// implementation owns its parsing and free-model filtering logic, and merges
// discovered models with its curated base layer (curated metadata always wins
// for known IDs).
type Discoverer interface {
// Refresh fetches the upstream model list and updates the backend's
// dynamic catalogue. Safe to call concurrently with Models()/Resolve().
Refresh() error
}
internal/proxy/kilo.go
+181 -15
View File
@@ -1,5 +1,15 @@
package proxy package proxy
import (
"encoding/json"
"fmt"
"io"
"net/http"
"strings"
"sync"
"time"
)
// KiloGatewayBase is the Kilo Code public gateway base URL. // KiloGatewayBase is the Kilo Code public gateway base URL.
// Confirmed working without auth for free models (HTTP 200 on both // Confirmed working without auth for free models (HTTP 200 on both
// GET /api/gateway/models and POST /api/gateway/chat/completions). // GET /api/gateway/models and POST /api/gateway/chat/completions).
@@ -7,14 +17,20 @@ const KiloGatewayBase = "https://api.kilo.ai/api/gateway"
// KiloBackend forwards to Kilo Code's free gateway models. // KiloBackend forwards to Kilo Code's free gateway models.
// //
// Kilo is an opencode-based CLI whose gateway exposes a handful of free models // Kilo is an opencode-based CLI whose gateway exposes free models (flagged
// (suffixed ":free") reachable without authentication, in standard OpenAI // isFree:true in GET /api/gateway/models) reachable without authentication,
// Chat Completions format. // in standard OpenAI Chat Completions format. This backend is a Discoverer:
type KiloBackend struct{} // it auto-discovers its full free catalogue with rich metadata from the
// upstream /models endpoint.
type KiloBackend struct {
mu sync.RWMutex
discovered []ModelInfo
}
// kiloFreeModels are the free models advertised by the Kilo gateway. // kiloCuratedModels is the curated base layer: aliases + trusted metadata.
// Sourced from GET /api/gateway/models filtering isFree:true. // Discovered models matching these IDs keep this metadata (curated wins).
var kiloFreeModels = []ModelInfo{ // Discovered-only models use the metadata parsed from the upstream response.
var kiloCuratedModels = []ModelInfo{
{ {
ID: "stepfun/step-3.7-flash:free", ID: "stepfun/step-3.7-flash:free",
Name: "StepFun Step 3.7 Flash (Free)", Name: "StepFun Step 3.7 Flash (Free)",
@@ -60,17 +76,34 @@ var kiloAliases = map[string]string{
} }
// Name returns the backend identifier. // Name returns the backend identifier.
func (KiloBackend) Name() string { return "kilo" } func (*KiloBackend) Name() string { return "kilo" }
// Models returns the Kilo free models. // Models returns the Kilo free models (discovered catalogue, or curated
func (KiloBackend) Models() []ModelInfo { return kiloFreeModels } // fallback before the first successful Refresh()).
func (b *KiloBackend) Models() []ModelInfo {
b.mu.RLock()
defer b.mu.RUnlock()
if len(b.discovered) == 0 {
return kiloCuratedModels
}
return b.discovered
}
// Resolve maps a requested name to a Kilo free model ID. // Resolve maps a requested name to a Kilo model ID. Checks curated aliases,
func (KiloBackend) Resolve(model string) (string, bool) { // curated IDs, then the discovered catalogue so newly discovered models route
// correctly instead of falling through to pass-through.
func (b *KiloBackend) Resolve(model string) (string, bool) {
if mapped, ok := kiloAliases[model]; ok { if mapped, ok := kiloAliases[model]; ok {
return mapped, true return mapped, true
} }
for _, m := range kiloFreeModels { for _, m := range kiloCuratedModels {
if m.ID == model {
return model, true
}
}
b.mu.RLock()
defer b.mu.RUnlock()
for _, m := range b.discovered {
if m.ID == model { if m.ID == model {
return model, true return model, true
} }
@@ -79,13 +112,146 @@ func (KiloBackend) Resolve(model string) (string, bool) {
} }
// ChatURL is the Kilo gateway chat completions endpoint. // ChatURL is the Kilo gateway chat completions endpoint.
func (KiloBackend) ChatURL() string { return KiloGatewayBase + "/chat/completions" } func (*KiloBackend) ChatURL() string { return KiloGatewayBase + "/chat/completions" }
// Headers returns minimal headers for the Kilo gateway (no auth needed for free). // Headers returns minimal headers for the Kilo gateway (no auth needed for free).
func (KiloBackend) Headers(_, _ string) map[string]string { func (*KiloBackend) Headers(_, _ string) map[string]string {
return map[string]string{ return map[string]string{
"Content-Type": "application/json", "Content-Type": "application/json",
"Accept": "text/event-stream, application/json", "Accept": "text/event-stream, application/json",
"User-Agent": "kilo/7.3.54", "User-Agent": "kilo/7.3.54",
} }
} }
// ---------- Auto-discovery ----------
// Refresh fetches GET /api/gateway/models, keeps only isFree:true entries,
// and merges them with the curated base layer. Curated metadata wins for
// known IDs; discovered-only models use upstream metadata.
func (b *KiloBackend) Refresh() error {
req, err := http.NewRequest("GET", KiloGatewayBase+"/models", nil)
if err != nil {
return err
}
req.Header.Set("User-Agent", "kilo/7.3.54")
req.Header.Set("Accept", "application/json")
client := &http.Client{Timeout: 30 * time.Second}
resp, err := client.Do(req)
if err != nil {
return fmt.Errorf("kilo discovery: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode != 200 {
return fmt.Errorf("kilo discovery: upstream status %d", resp.StatusCode)
}
body, err := io.ReadAll(resp.Body)
if err != nil {
return err
}
parsed := parseKiloModels(body)
merged := mergeKiloCatalogue(kiloCuratedModels, parsed)
b.mu.Lock()
b.discovered = merged
b.mu.Unlock()
return nil
}
// kiloModelsResponse is the subset of the Kilo /models schema we care about.
type kiloModelsResponse struct {
Data []struct {
ID string `json:"id"`
Name string `json:"name"`
Description string `json:"description"`
IsFree bool `json:"isFree"`
ContextLen int `json:"context_length"`
TopProvider struct {
ContextLength int `json:"context_length"`
MaxCompletionTokens *int `json:"max_completion_tokens"`
} `json:"top_provider"`
Architecture struct {
InputModalities []string `json:"input_modalities"`
} `json:"architecture"`
Pricing struct {
Prompt string `json:"prompt"`
Completion string `json:"completion"`
} `json:"pricing"`
} `json:"data"`
}
// parseKiloModels converts the upstream response into ModelInfo entries,
// keeping only free models (isFree:true).
func parseKiloModels(body []byte) []ModelInfo {
var resp kiloModelsResponse
if err := json.Unmarshal(body, &resp); err != nil {
return nil
}
var out []ModelInfo
for _, m := range resp.Data {
if !m.IsFree {
continue
}
ctx := m.ContextLen
if ctx == 0 {
ctx = m.TopProvider.ContextLength
}
maxOut := 0
if m.TopProvider.MaxCompletionTokens != nil {
maxOut = *m.TopProvider.MaxCompletionTokens
}
vision := false
for _, mod := range m.Architecture.InputModalities {
if mod == "image" {
vision = true
}
}
owner := ownerFromID(m.ID)
desc := m.Description
if desc == "" {
desc = fmt.Sprintf("%s (free). Detectado automáticamente.", m.Name)
}
out = append(out, ModelInfo{
ID: m.ID,
Name: m.Name,
OwnedBy: owner,
ContextWindow: ctx,
MaxOutputTokens: maxOut,
Description: desc,
SupportsVision: vision,
})
}
return out
}
// mergeKiloCatalogue merges curated + discovered. Curated entries are always
// present (with their metadata). Discovered entries not in curated are added.
func mergeKiloCatalogue(curated, discovered []ModelInfo) []ModelInfo {
byID := make(map[string]bool, len(curated)+len(discovered))
merged := make([]ModelInfo, 0, len(curated)+len(discovered))
// Curated first (trusted metadata + aliases still resolve).
for _, m := range curated {
if !byID[m.ID] {
merged = append(merged, m)
byID[m.ID] = true
}
}
// Discovered-only appended with upstream metadata.
for _, m := range discovered {
if !byID[m.ID] {
merged = append(merged, m)
byID[m.ID] = true
}
}
return merged
}
// ownerFromID extracts a provider name from a "provider/model" id.
func ownerFromID(id string) string {
if i := strings.Index(id, "/"); i >= 0 {
return id[:i]
}
return "unknown"
}
internal/proxy/kimchi.go
+232
View File
@@ -0,0 +1,232 @@
package proxy
import (
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"sync"
"time"
)
// KimchiBaseURL is the upstream Kimchi Dev API base URL.
const KimchiBaseURL = "https://llm.kimchi.dev/openai/v1"
// KimchiBackend forwards to Kimchi Dev's free models.
//
// It is a Discoverer: it polls GET /openai/v1/models and merges the IDs with
// the curated base layer. Kimchi's /models returns only IDs (no metadata, no
// free flag). Because the Kimchi account is credit-based, every advertised
// model is treated as usable: all discovered IDs not already curated are added
// with minimal metadata (option B). Curated IDs keep their trusted metadata.
type KimchiBackend struct {
mu sync.RWMutex
discovered []ModelInfo
}
// kimchiCuratedModels is the curated base layer with full metadata.
var kimchiCuratedModels = []ModelInfo{
{
ID: "deepseek-v4-flash",
Name: "DeepSeek V4 Flash (Kimchi)",
OwnedBy: "deepseek",
ContextWindow: 1_048_576,
MaxOutputTokens: 1_048_576,
Description: "DeepSeek V4, 284B params, propósito general, razonamiento.",
},
{
ID: "glm-5.2-fp8",
Name: "GLM 5.2 FP8 (Kimchi)",
OwnedBy: "zhipu",
ContextWindow: 1_048_576,
MaxOutputTokens: 1_048_576,
Description: "GLM-5.2 cuantizado FP8, razonamiento, rendimiento.",
},
{
ID: "kimi-k2.7",
Name: "Kimi K2.7 (Kimchi)",
OwnedBy: "moonshot",
ContextWindow: 262_144,
MaxOutputTokens: 262_144,
Description: "Moonshot AI, multimodal con visión, razonamiento.",
SupportsVision: true,
},
{
ID: "minimax-m3",
Name: "MiniMax M3 (Kimchi)",
OwnedBy: "minimax",
ContextWindow: 1_048_576,
MaxOutputTokens: 1_048_576,
Description: "MiniMax M3, multimodal texto+imagen, razonamiento.",
SupportsVision: true,
},
{
ID: "nemotron-3-ultra-fp4",
Name: "Nemotron 3 Ultra FP4 (Kimchi)",
OwnedBy: "nvidia",
ContextWindow: 1_048_576,
MaxOutputTokens: 1_048_576,
Description: "NVIDIA Nemotron 3 Ultra 550B, FP4 cuantizado, razonamiento.",
},
}
// kimchiAliases lets clients use shorter names for Kimchi models.
// Uses prefixed aliases to avoid colliding with opencode backend aliases.
var kimchiAliases = map[string]string{
"kimchi": "deepseek-v4-flash",
"kimchi/deepseek": "deepseek-v4-flash",
"kimchi/glm": "glm-5.2-fp8",
"kimchi/kimi": "kimi-k2.7",
"kimchi/minimax": "minimax-m3",
"kimchi/nemotron": "nemotron-3-ultra-fp4",
"deepseek-v4-flash": "deepseek-v4-flash",
"glm-5.2-fp8": "glm-5.2-fp8",
"kimi-k2.7": "kimi-k2.7",
"minimax-m3": "minimax-m3",
"nemotron-3-ultra-fp4": "nemotron-3-ultra-fp4",
}
// Name returns the backend identifier.
func (*KimchiBackend) Name() string { return "kimchi" }
// Models returns the Kimchi free models (discovered, or curated fallback).
func (b *KimchiBackend) Models() []ModelInfo {
b.mu.RLock()
defer b.mu.RUnlock()
if len(b.discovered) == 0 {
return kimchiCuratedModels
}
return b.discovered
}
// Resolve maps a requested name to a Kimchi model ID.
func (b *KimchiBackend) Resolve(model string) (string, bool) {
if mapped, ok := kimchiAliases[model]; ok {
return mapped, true
}
for _, m := range kimchiCuratedModels {
if m.ID == model {
return model, true
}
}
b.mu.RLock()
defer b.mu.RUnlock()
for _, m := range b.discovered {
if m.ID == model {
return model, true
}
}
return "", false
}
// ChatURL is the Kimchi Dev chat completions endpoint.
func (*KimchiBackend) ChatURL() string { return KimchiBaseURL + "/chat/completions" }
// Headers returns the headers required for Kimchi Dev auth.
// Reads the API key from the KIMCHI_API_KEY env var.
// Falls back to the hardcoded key if the env var is empty.
func (*KimchiBackend) Headers(_, _ string) map[string]string {
key := os.Getenv("KIMCHI_API_KEY")
if key == "" {
key = "castai_v1_befd8c666ce652d47cc08accf8c5accc0a5aaca9e1f006392df69ac123198f9b_4f8f2d40"
}
return map[string]string{
"Authorization": "Bearer " + key,
"Content-Type": "application/json",
"Accept": "text/event-stream, application/json",
"User-Agent": "kimchi/0.1.50",
}
}
// ---------- Auto-discovery ----------
// Refresh fetches GET /openai/v1/models and merges the IDs with the curated
// base layer. All discovered IDs are treated as usable (credit-based account),
// so every non-curated ID is added with minimal metadata.
func (b *KimchiBackend) Refresh() error {
req, err := http.NewRequest("GET", KimchiBaseURL+"/models", nil)
if err != nil {
return err
}
key := os.Getenv("KIMCHI_API_KEY")
if key == "" {
key = "castai_v1_befd8c666ce652d47cc08accf8c5accc0a5aaca9e1f006392df69ac123198f9b_4f8f2d40"
}
req.Header.Set("Authorization", "Bearer "+key)
req.Header.Set("User-Agent", "kimchi/0.1.50")
req.Header.Set("Accept", "application/json")
client := &http.Client{Timeout: 30 * time.Second}
resp, err := client.Do(req)
if err != nil {
return fmt.Errorf("kimchi discovery: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode != 200 {
return fmt.Errorf("kimchi discovery: upstream status %d", resp.StatusCode)
}
body, err := io.ReadAll(resp.Body)
if err != nil {
return err
}
ids := parseKimchiModelIDs(body)
merged := mergeKimchiCatalogue(kimchiCuratedModels, ids)
b.mu.Lock()
b.discovered = merged
b.mu.Unlock()
return nil
}
// kimchiModelsResponse is the minimal Kimchi /models schema.
type kimchiModelsResponse struct {
Data []struct {
ID string `json:"id"`
OwnedBy string `json:"owned_by"`
} `json:"data"`
}
// parseKimchiModelIDs returns the list of model IDs advertised upstream.
func parseKimchiModelIDs(body []byte) []struct{ ID, OwnedBy string } {
var resp kimchiModelsResponse
if err := json.Unmarshal(body, &resp); err != nil {
return nil
}
out := make([]struct{ ID, OwnedBy string }, 0, len(resp.Data))
for _, m := range resp.Data {
out = append(out, struct{ ID, OwnedBy string }{m.ID, m.OwnedBy})
}
return out
}
// mergeKimchiCatalogue keeps all curated models plus every discovered ID
// (Kimchi is credit-based, so all advertised models are usable).
func mergeKimchiCatalogue(curated []ModelInfo, discovered []struct{ ID, OwnedBy string }) []ModelInfo {
byID := make(map[string]bool, len(curated)+len(discovered))
merged := make([]ModelInfo, 0, len(curated)+len(discovered))
for _, m := range curated {
if !byID[m.ID] {
merged = append(merged, m)
byID[m.ID] = true
}
}
for _, d := range discovered {
if byID[d.ID] {
continue
}
owner := d.OwnedBy
if owner == "" {
owner = "castai"
}
merged = append(merged, ModelInfo{
ID: d.ID,
Name: d.ID + " (Kimchi)",
OwnedBy: owner,
Description: "Detectado automáticamente.",
})
byID[d.ID] = true
}
return merged
}
internal/proxy/models.go
+47 -3
View File
@@ -21,6 +21,11 @@ type ModelInfo struct {
SupportsVision bool `json:"supports_vision"` SupportsVision bool `json:"supports_vision"`
} }
// discoveryInterval is how often backends re-fetch their upstream /models
// catalogue. Kept shorter than the health interval because model lists rotate
// more frequently than rate-limit windows.
const discoveryInterval = 10 * time.Minute
// healthCheckInterval is how often caged models get re-probed. // healthCheckInterval is how often caged models get re-probed.
const healthCheckInterval = 30 * time.Minute const healthCheckInterval = 30 * time.Minute
@@ -59,15 +64,20 @@ type session struct {
// the catch-all opencode aliases. The default backend is the fallback for // the catch-all opencode aliases. The default backend is the fallback for
// unknown models (pass-through as-is). // unknown models (pass-through as-is).
func NewProxy(apiKey string) *Proxy { func NewProxy(apiKey string) *Proxy {
oc := OpenCodeBackend{} // Backends are stateful (they hold a mutex + discovered catalogue), so we
kl := KiloBackend{} // store pointers to avoid copying the mutex when the Backend interface
// value is passed around.
oc := &OpenCodeBackend{}
kl := &KiloBackend{}
km := &KimchiBackend{}
p := &Proxy{ p := &Proxy{
apiKey: apiKey, apiKey: apiKey,
backends: []Backend{kl, oc}, backends: []Backend{km, kl, oc},
defaultBackend: oc, defaultBackend: oc,
sessions: make(map[string]*session), sessions: make(map[string]*session),
health: make(map[string]*modelHealth), health: make(map[string]*modelHealth),
} }
go p.discoveryLoop()
go p.healthLoop() go p.healthLoop()
return p return p
} }
@@ -142,6 +152,40 @@ func randomHex(n int) string {
return hex.EncodeToString(b) return hex.EncodeToString(b)
} }
// ---------- Model discovery ----------
// discoveryLoop runs an initial full refresh on startup, then re-fetches each
// backend's upstream /models catalogue every discoveryInterval. Backends that
// do not implement Discoverer are skipped (they stay on their curated list).
func (p *Proxy) discoveryLoop() {
p.refreshAllBackends()
ticker := time.NewTicker(discoveryInterval)
defer ticker.Stop()
for range ticker.C {
p.refreshAllBackends()
}
}
// refreshAllBackends calls Refresh() on every backend that implements
// Discoverer. Failures are logged but do not stop the loop; a failed refresh
// leaves the backend on its previous (curated or last-good) catalogue.
func (p *Proxy) refreshAllBackends() {
var names []string
for _, b := range p.backends {
d, ok := b.(Discoverer)
if !ok {
continue
}
if err := d.Refresh(); err != nil {
fmt.Printf("[discovery] backend '%s' refresh failed: %v\n", b.Name(), err)
continue
}
names = append(names, b.Name())
}
total := len(p.Models())
fmt.Printf("[discovery] refresh complete: %d backends, %d models\n", len(names), total)
}
// ---------- Model health / rate-limit tracking ---------- // ---------- Model health / rate-limit tracking ----------
// markRateLimited cages a model so it disappears from /v1/models until it recovers. // markRateLimited cages a model so it disappears from /v1/models until it recovers.
internal/proxy/opencode.go
+151 -10
View File
@@ -1,12 +1,32 @@
package proxy package proxy
import (
"encoding/json"
"fmt"
"io"
"net/http"
"strings"
"sync"
"time"
)
// ZenBaseURL is the upstream OpenCode Zen API base URL. // ZenBaseURL is the upstream OpenCode Zen API base URL.
const ZenBaseURL = "https://opencode.ai/zen/v1" const ZenBaseURL = "https://opencode.ai/zen/v1"
// OpenCodeBackend forwards to OpenCode's free Zen models. // OpenCodeBackend forwards to OpenCode's free Zen models.
type OpenCodeBackend struct{} //
// It is a Discoverer: it polls GET /zen/v1/models and merges the IDs with the
// curated base layer. OpenCode's /models returns only IDs (no metadata, no
// free flag), so discovery is heuristic: curated models always pass with full
// metadata; additionally any ID carrying a "free" token (suffix -free / :free)
// is added with minimal metadata. This catches newly-rotated free models
// (e.g. hy3-free) while avoiding paid models.
type OpenCodeBackend struct {
mu sync.RWMutex
discovered []ModelInfo
}
// zenModels are the free models exposed by OpenCode Zen (no API key required). // zenModels is the curated base layer: full metadata + alias anchors.
var zenModels = []ModelInfo{ var zenModels = []ModelInfo{
{ {
ID: "big-pickle", ID: "big-pickle",
@@ -88,13 +108,20 @@ var modelAliases = map[string]string{
} }
// Name returns the backend identifier. // Name returns the backend identifier.
func (OpenCodeBackend) Name() string { return "opencode" } func (*OpenCodeBackend) Name() string { return "opencode" }
// Models returns the OpenCode Zen free models. // Models returns the OpenCode Zen free models (discovered, or curated fallback).
func (OpenCodeBackend) Models() []ModelInfo { return zenModels } func (b *OpenCodeBackend) Models() []ModelInfo {
b.mu.RLock()
defer b.mu.RUnlock()
if len(b.discovered) == 0 {
return zenModels
}
return b.discovered
}
// Resolve maps a requested name to an OpenCode Zen model ID. // Resolve maps a requested name to an OpenCode Zen model ID.
func (OpenCodeBackend) Resolve(model string) (string, bool) { func (b *OpenCodeBackend) Resolve(model string) (string, bool) {
if mapped, ok := modelAliases[model]; ok { if mapped, ok := modelAliases[model]; ok {
return mapped, true return mapped, true
} }
@@ -103,14 +130,21 @@ func (OpenCodeBackend) Resolve(model string) (string, bool) {
return model, true return model, true
} }
} }
b.mu.RLock()
defer b.mu.RUnlock()
for _, m := range b.discovered {
if m.ID == model {
return model, true
}
}
return "", false return "", false
} }
// ChatURL is the OpenCode Zen chat completions endpoint. // ChatURL is the OpenCode Zen chat completions endpoint.
func (OpenCodeBackend) ChatURL() string { return ZenBaseURL + "/chat/completions" } func (*OpenCodeBackend) ChatURL() string { return ZenBaseURL + "/chat/completions" }
// Headers builds the headers OpenCode Zen expects (x-opencode-* + public bearer). // Headers builds the headers OpenCode Zen expects (x-opencode-* + public bearer).
func (OpenCodeBackend) Headers(requestID, sessionID string) map[string]string { func (*OpenCodeBackend) Headers(requestID, sessionID string) map[string]string {
return map[string]string{ return map[string]string{
"Authorization": "Bearer public", "Authorization": "Bearer public",
"User-Agent": "opencode/1.15.0 ai-sdk/provider-utils/4.0.23 runtime/bun/1.3.13", "User-Agent": "opencode/1.15.0 ai-sdk/provider-utils/4.0.23 runtime/bun/1.3.13",
@@ -123,14 +157,121 @@ func (OpenCodeBackend) Headers(requestID, sessionID string) map[string]string {
} }
} }
// ---------- Auto-discovery ----------
// Refresh fetches GET /zen/v1/models and merges the IDs with the curated base
// layer. Curated models keep their metadata; additionally any ID carrying a
// "free" token is added with minimal metadata (heuristic free detection,
// because OpenCode's /models has no free flag).
func (b *OpenCodeBackend) Refresh() error {
req, err := http.NewRequest("GET", ZenBaseURL+"/models", nil)
if err != nil {
return err
}
req.Header.Set("Authorization", "Bearer public")
req.Header.Set("x-opencode-client", "cli")
req.Header.Set("Accept", "application/json")
client := &http.Client{Timeout: 30 * time.Second}
resp, err := client.Do(req)
if err != nil {
return fmt.Errorf("opencode discovery: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode != 200 {
return fmt.Errorf("opencode discovery: upstream status %d", resp.StatusCode)
}
body, err := io.ReadAll(resp.Body)
if err != nil {
return err
}
ids := parseZenModelIDs(body)
merged := mergeZenCatalogue(zenModels, ids)
b.mu.Lock()
b.discovered = merged
b.mu.Unlock()
return nil
}
// zenModelsResponse is the minimal OpenCode Zen /models schema.
type zenModelsResponse struct {
Data []struct {
ID string `json:"id"`
OwnedBy string `json:"owned_by"`
} `json:"data"`
}
// parseZenModelIDs returns the list of model IDs advertised upstream.
func parseZenModelIDs(body []byte) []struct{ ID, OwnedBy string } {
var resp zenModelsResponse
if err := json.Unmarshal(body, &resp); err != nil {
return nil
}
out := make([]struct{ ID, OwnedBy string }, 0, len(resp.Data))
for _, m := range resp.Data {
out = append(out, struct{ ID, OwnedBy string }{m.ID, m.OwnedBy})
}
return out
}
// looksFreeHeuristic reports whether a model ID looks free. Used because
// OpenCode's /models endpoint has no isFree flag.
func looksFreeHeuristic(id string) bool {
low := strings.ToLower(id)
return strings.Contains(low, "-free") || strings.Contains(low, ":free") || strings.HasSuffix(low, "free")
}
// mergeZenCatalogue keeps all curated models (with full metadata) plus any
// discovered ID that looks free and isn't already curated.
func mergeZenCatalogue(curated []ModelInfo, discovered []struct{ ID, OwnedBy string }) []ModelInfo {
byID := make(map[string]bool, len(curated)+len(discovered))
merged := make([]ModelInfo, 0, len(curated)+len(discovered))
for _, m := range curated {
if !byID[m.ID] {
merged = append(merged, m)
byID[m.ID] = true
}
}
for _, d := range discovered {
if byID[d.ID] {
continue
}
if !looksFreeHeuristic(d.ID) {
continue
}
owner := d.OwnedBy
if owner == "" {
owner = "opencode-zen"
}
merged = append(merged, ModelInfo{
ID: d.ID,
Name: d.ID + " (Free)",
OwnedBy: owner,
Description: "Detectado automáticamente (heurística free).",
})
byID[d.ID] = true
}
return merged
}
// ModelByID returns the OpenCode Zen ModelInfo for an ID, or nil if not found. // ModelByID returns the OpenCode Zen ModelInfo for an ID, or nil if not found.
// Used by the Anthropic path for vision auto-routing. // Used by the Anthropic path for vision auto-routing. Checks curated first,
func ModelByID(id string) *ModelInfo { // then the discovered catalogue.
func (b *OpenCodeBackend) ModelByID(id string) *ModelInfo {
for _, m := range zenModels { for _, m := range zenModels {
if m.ID == id { if m.ID == id {
return &m return &m
} }
} }
b.mu.RLock()
defer b.mu.RUnlock()
for i := range b.discovered {
if b.discovered[i].ID == id {
return &b.discovered[i]
}
}
return nil return nil
} }