renato97/free-ide-proxy
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

feat: multi-backend proxy with auto rate-limit jailing

Browse Source 
Refactor from monolithic proxy to multi-backend architecture
(Backend interface: OpenCode Zen + Kilo gateway).

- Add auto rate-limit jailing: models returning HTTP 429 are
  hidden from /v1/models immediately and re-probed every 30 min
- Backend interface supports model aliases, custom headers, and
  per-backend routing
- Full Anthropic Messages API support with OpenAI format conversion
- 9 free models across two backends with Claude name aliases
This commit is contained in:
Renato
2026-06-26 19:08:01 +02:00
parent 0a4289cafc
commit 05a52d29e2
9 changed files with 652 additions and 245 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+4
View File
@@ -1,6 +1,10 @@
*.exe *.exe
*.exe~ *.exe~
opencode-proxy opencode-proxy
free-ide-proxy
/opencode-proxy
/free-ide-proxy
deploy/ deploy/
vendor/ vendor/
*.log *.log
.atl/
AGENTS.md
+31 -16
View File
@@ -1,13 +1,13 @@
# OpenCode Zen Proxy # Free IDE Proxy
Proxy translating OpenAI and Anthropic Messages API requests to OpenCode Zen's free model API. OpenAI/Anthropic-compatible proxy that aggregates the **free** models from multiple opencode-based editors (currently OpenCode Zen and Kilo Code) under a single endpoint. Each upstream lives behind a `Backend` implementation and the proxy routes by model name.
## Build & Run ## Build & Run
```bash ```bash
go build -o opencode-proxy . go build -o free-ide-proxy .
./opencode-proxy # no auth ./free-ide-proxy # no auth
./opencode-proxy -api-key "secret" # with auth ./free-ide-proxy -api-key "secret" # with auth
``` ```
Auth reads `OPENCODE_PROXY_KEY` env var; `-api-key` flag takes precedence. Auth reads `OPENCODE_PROXY_KEY` env var; `-api-key` flag takes precedence.
@@ -16,15 +16,18 @@ Defaults: `127.0.0.1:6446`. Use `-host 0.0.0.0` for external access.
## Architecture ## Architecture
``` ```
Client → internal/server/ (HTTP, auth, routing) → internal/proxy/ (model resolve, session mgmt, forwarding, Anthropic↔OpenAI conversion) → opencode.ai/zen/v1 Client → internal/server/ (HTTP, auth, routing) → internal/proxy/ (model resolve, backend router, session mgmt, forwarding, Anthropic↔OpenAI conversion) → Backend → upstream
``` ```
- **Pure stdlib Go 1.22** — no router, no deps beyond stdlib - **Pure stdlib Go 1.22** — no router, no deps beyond stdlib
- **Entrypoint:** `main.go` wires `proxy.NewProxy(key)` into `server.NewServer(p)` - **Entrypoint:** `main.go` wires `proxy.NewProxy(key)` into `server.NewServer(p)`
- **Multi-backend:** `internal/proxy/backend.go` defines the `Backend` interface. The `Proxy` holds an ordered list and routes each request to the first backend that recognises the model. Unknown models fall back to the default backend (opencode) and are forwarded as-is.
- **Backends:** `opencode.go` (OpenCode Zen, `opencode.ai/zen/v1`, `Bearer public`) and `kilo.go` (Kilo gateway, `api.kilo.ai/api/gateway`, no auth). Add a new file implementing `Backend` and register it in `NewProxy` (`models.go`) to support another upstream.
- **Auth middleware** in `server.go:88` checks `Authorization: Bearer` or `x-api-key` header - **Auth middleware** in `server.go:88` checks `Authorization: Bearer` or `x-api-key` header
- **Model aliases** in `internal/proxy/models.go:59` — short names, company names, and Claude model names → full IDs - **Model aliases** live in each backend (`opencode.go` and `kilo.go`) — short names, company names, and Claude model names → full IDs
- **Session rotation:** session IDs rotate every 30 min per user key (`proxy.SessionID()` at `models.go:154`) - **Session rotation:** session IDs rotate every 30 min per user key (`proxy.SessionID()`)
- **Pass-through:** unknown model names are forwarded as-is, not rejected - **Auto rate-limit jailing:** models returning HTTP 429 are automatically hidden from `/v1/models` and re-probed every 30 min until they recover. See `models.go` (`modelHealth`, `healthLoop`, `pingModel`, `markRateLimited`, `markActive`, `isRateLimited`, `Models()` filtering).
- **Pass-through:** unknown model names are forwarded as-is to the default backend, not rejected
## Endpoints ## Endpoints
@@ -45,7 +48,9 @@ Fully implemented in `internal/proxy/anthropic.go`. Converts Anthropic → OpenA
**Claude model name aliases** are defined in `models.go:77-88` — Claude Code validates model names client-side, so you must set `ANTHROPIC_MODEL` to a valid Claude name like `claude-sonnet-4-6` (maps to `deepseek-v4-flash-free`). **Claude model name aliases** are defined in `models.go:77-88` — Claude Code validates model names client-side, so you must set `ANTHROPIC_MODEL` to a valid Claude name like `claude-sonnet-4-6` (maps to `deepseek-v4-flash-free`).
## Available Models (from `models.go`) ## Available Models (from `opencode.go` + `kilo.go`)
### OpenCode Zen backend (`opencode.go` → `opencode.ai/zen/v1`, `Bearer public`)
| Model ID | Aliases | Context | Max Output | | Model ID | Aliases | Context | Max Output |
|----------|---------|---------|------------| |----------|---------|---------|------------|
@@ -55,18 +60,28 @@ Fully implemented in `internal/proxy/anthropic.go`. Converts Anthropic → OpenA
| `north-mini-code-free` | `north`, `north-mini`, `cohere` | 256K | 64K | | `north-mini-code-free` | `north`, `north-mini`, `cohere` | 256K | 64K |
| `nemotron-3-ultra-free` | `nemotron`, `nemotron-3`, `nvidia` | 1M | 16K | | `nemotron-3-ultra-free` | `nemotron`, `nemotron-3`, `nvidia` | 1M | 16K |
**Source of truth is `models.go`, not `README.md`.** The README may list models that differ from the code (e.g. `minimax-m2.5-free`, `kimi-k2.5-free`, `gpt-5-nano`, `nemotron-3-super-free`, `qwen3.6-plus-free` are NOT in the code). The code controls what works. ### Kilo gateway backend (`kilo.go` → `api.kilo.ai/api/gateway`, no auth)
| Model ID | Aliases | Context | Max Output |
|----------|---------|---------|------------|
| `stepfun/step-3.7-flash:free` | `stepfun`, `stepfun-free` | 256K | 32K |
| `poolside/laguna-m.1:free` | `poolside`, `poolside-free`, `laguna` | 256K | 32K |
| `nvidia/nemotron-3-ultra-550b-a55b:free` | — | 1M | 16K |
| `openrouter/free` | `openrouter` | 256K | 32K |
**Source of truth is the code (`opencode.go`, `kilo.go`), not `README.md`.** The README may list models that differ from the code (e.g. `minimax-m2.5-free`, `kimi-k2.5-free`, `gpt-5-nano`, `nemotron-3-super-free`, `qwen3.6-plus-free` are NOT in the code). The code controls what works.
## Quirks & Gotchas ## Quirks & Gotchas
- **No tests exist.** Any change is untested unless you add them. - **No tests exist.** Any change is untested unless you add them.
- **No CI/CD, no Makefile, no lint config.** All manual. - **No CI/CD, no Makefile, no lint config.** All manual.
- **In-memory only.** Session state dies on restart. No database. - **In-memory only.** Session and health state dies on restart. The health map auto-rebuilds via initial scan.
- **`deploy/start-proxy.bat`** and **`deploy/CREDENCIALES.md`** contain a hardcoded API key — do not commit them. - **`deploy/start-proxy.bat`** and **`deploy/CREDENCIALES.md`** contain a hardcoded API key — do not commit them.
- **`opencode-proxy.exe~`** in root is a backup artifact; ignore. - **`opencode-proxy.exe~`** in root is a backup artifact; ignore.
- Upstream Zen API expects `x-opencode-request`, `x-opencode-session`, `x-opencode-client`, `x-opencode-project` headers (set in `proxy.go:107`). - Upstream Zen API expects `x-opencode-request`, `x-opencode-session`, `x-opencode-client`, `x-opencode-project` headers (set in `opencode.go` `OpenCodeBackend.Headers`). Kilo needs no auth — its gateway returns HTTP 200 for free models with no `Authorization` header.
- **Server logs request bodies to stdout** (truncated to 500 chars in `server.go:265-271`). Verbose by design. - **Server logs request bodies to stdout** (truncated to 500 chars in `server.go:265-271`). Verbose by design.
- **Health endpoint** returns hardcoded version `"1.0.0"` in `server.go:126`. Update both `main.go:12` and `server.go:126` when bumping. - **Health endpoint** returns hardcoded version `"1.0.0"` in `server.go:126`. Update both `main.go:12` and `server.go:126` when bumping.
- **Upstream timeout:** `proxy.go:167` sets 10-minute HTTP client timeout — adjust if Zen models are slow on first call. - **Upstream timeout:** `proxy.go` sets a 10-minute HTTP client timeout — adjust if Zen models are slow on first call.
- **Unknown models are forwarded as-is** (`models.go:140`) — useful when new models appear upstream before code is updated. - **Unknown models are forwarded as-is** via the default backend (opencode) — useful when new models appear upstream before code is updated.
- **Reasoning models** (DeepSeek, Nemotron) need `max_tokens` ≥ 500 — first tokens go to reasoning, not visible content. - **Reasoning models** (DeepSeek, Nemotron, StepFun) need `max_tokens` ≥ 500 — first tokens go to reasoning, not visible content. With `max_tokens:30` they return empty content; that is model behaviour, not a proxy bug.
- **Rate limit jailing:** `proxy.go:164` and `anthropic.go:713` check for 429 and call `markRateLimited` immediately. `healthLoop()` in `models.go` does a full scan on startup, then re-checks caged models every 30 min. `Models()` filters out rate-limited models — they reappear once recovered.
README.md
+96 -67
View File
@@ -1,30 +1,36 @@
# OpenCode Zen Proxy # Free IDE Proxy
OpenAI-compatible proxy for [OpenCode](https://opencode.ai)'s free Zen models. Use DeepSeek V4 Flash, MiniMax M2.5, Kimi K2.5, and more — for free — in any OpenAI-compatible tool (Cursor, Claude Code, Continue, etc.). OpenAI/Anthropic-compatible proxy that aggregates **free** models from [OpenCode Zen](https://opencode.ai) and [Kilo Code](https://kilo.ai) under a single endpoint. Use DeepSeek V4 Flash, StepFun, Nemotron, OpenRouter free, and more — for free — in any OpenAI-compatible tool (Cursor, Claude Code, Continue, Cline, etc.).
## How it works ## How it works
``` ```
Your tool → opencode-proxy (localhost:6446) → opencode.ai/zen/v1 Your tool → free-ide-proxy (localhost:6446) → { opencode.ai/zen/v1 | api.kilo.ai/api/gateway }
``` ```
The proxy injects the required `x-opencode-*` headers that OpenCode Zen's free API expects. Your tools talk standard OpenAI protocol to the proxy; the proxy handles the rest. The proxy routes by model name to the matching backend, injecting the required headers. OpenCode Zen expects `x-opencode-*` headers with `Bearer public`; Kilo's gateway needs no auth for free models. Your tools talk standard OpenAI protocol to the proxy; the proxy handles the rest.
### Auto rate-limit jailing
When an upstream starts returning HTTP 429 (rate limit) for a model, the proxy automatically **cages** it — the model disappears from `/v1/models` immediately. A background checker re-probes caged models every 30 minutes; if they recover, they reappear.
This means your client just needs to query `/v1/models` before each request and pick an available model. No hardcoded model names, no manual retries.
## Quick Start ## Quick Start
```bash ```bash
# Build # Build
go build -o opencode-proxy . go build -o free-ide-proxy .
# Run (no auth) # Run (no auth)
./opencode-proxy ./free-ide-proxy
# Run with API key protection # Run with API key protection
./opencode-proxy -api-key "your-secret-key" ./free-ide-proxy -api-key "your-secret-key"
# Or via env var # Or via env var
export OPENCODE_PROXY_KEY="your-secret-key" export OPENCODE_PROXY_KEY="your-secret-key"
./opencode-proxy ./free-ide-proxy
``` ```
The server starts on `http://127.0.0.1:6446`. The server starts on `http://127.0.0.1:6446`.
@@ -44,9 +50,12 @@ Also reads `OPENCODE_PROXY_KEY` environment variable if `-api-key` is not set.
| Method | Path | Description | | Method | Path | Description |
|--------|------|-------------| |--------|------|-------------|
| `GET` | `/health` | Health check | | `GET` | `/health` | Health check (version, active model count, auth status) |
| `GET` | `/v1/models` | List available free models | | `GET` | `/v1/models` | List currently **active** (not rate-limited) free models |
| `POST` | `/v1/chat/completions` | Chat completions (streaming + non-streaming) | | `GET` | `/v1/models/{id}` | Single model detail (404 if rate-limited) |
| `POST` | `/v1/chat/completions` | OpenAI Chat Completions (streaming + non-streaming) |
| `POST` | `/v1/messages` | Anthropic Messages API |
| `POST` | `/v1/v1/messages` | Workaround for Claude Code double-path bug |
### Authentication ### Authentication
@@ -56,58 +65,60 @@ If `-api-key` is set, all requests require one of:
## Available Models ## Available Models
| Model ID | Alias | Notes | ### OpenCode Zen (5 models) — `opencode.ai/zen/v1`
|----------|-------|-------|
| `deepseek-v4-flash-free` | `deepseek`, `deepseek-v4` | Solid, recommended |
| `big-pickle` | `pickle` | Stealth model (= DeepSeek V4 Flash) |
| `minimax-m2.5-free` | `minimax`, `m2.5` | Strong coding model |
| `kimi-k2.5-free` | `kimi`, `k2.5` | Best free model |
| `gpt-5-nano` | `nano`, `gpt5` | OpenAI-powered free |
| `nemotron-3-super-free` | `nemotron` | Hit or miss |
| `qwen3.6-plus-free` | `qwen` | Intermittent |
All support streaming, tool calls, and system messages. | Model ID | Aliases | Context | Max Output |
|----------|---------|---------|------------|
| `deepseek-v4-flash-free` | `deepseek`, `deepseek-v4`, `ds` | 1M | 384K |
| `big-pickle` | `pickle` | 200K | 32K |
| `mimo-v2.5-free` | `mimo`, `mimo-v2.5`, `xiaomi` | 1M | 32K |
| `north-mini-code-free` | `north`, `north-mini`, `cohere` | 256K | 64K |
| `nemotron-3-ultra-free` | `nemotron`, `nemotron-3`, `nvidia` | 1M | 16K |
**API auth:** `Bearer public` (no real token needed).
### Kilo gateway (4 models) — `api.kilo.ai/api/gateway`
| Model ID | Aliases | Context | Max Output |
|----------|---------|---------|------------|
| `stepfun/step-3.7-flash:free` | `stepfun`, `stepfun-free` | 256K | 32K |
| `poolside/laguna-m.1:free` | `poolside`, `poolside-free`, `laguna` | 256K | 32K |
| `nvidia/nemotron-3-ultra-550b-a55b:free` | — | 1M | 16K |
| `openrouter/free` | `openrouter` | 256K | 32K |
**API auth:** None needed for free models.
All models support streaming, tool calls, and system messages. Reasoning models (DeepSeek, Nemotron, StepFun) need `max_tokens` ≥ 500 or they return empty content (first tokens go to reasoning).
## Claude Code
Claude Code appends `/v1/messages` to `ANTHROPIC_BASE_URL` automatically. Set:
```bash
export ANTHROPIC_BASE_URL=http://127.0.0.1:6446
export ANTHROPIC_MODEL=claude-sonnet-4-6 # maps to deepseek-v4-flash-free
```
The proxy also handles `/v1/v1/messages` as a fallback for the double-path issue.
Claude model name aliases (Claude Code validates model names client-side):
| Claude Model | Maps to |
|-------------|---------|
| `claude-sonnet-4-6`, `claude-sonnet-4-5`, `claude-sonnet-4` | `deepseek-v4-flash-free` |
| `claude-opus-4-8`, `claude-opus-4-5`, `claude-opus-4` | `deepseek-v4-flash-free` |
| `claude-haiku-4-5`, `claude-haiku-4` | `north-mini-code-free` |
| `claude-3.5-sonnet` | `north-mini-code-free` |
| `claude-3.5-haiku` | `big-pickle` |
## Tool Configuration ## Tool Configuration
### Cursor / Continue / Cline ### Cursor / Continue / Cline
- **Base URL**: `http://127.0.0.1:6446/v1` - **Base URL**: `http://127.0.0.1:6446/v1`
- **API Key**: your proxy key (or `public` if no auth) - **API Key**: your proxy key (or empty if no auth)
- **Model**: `deepseek-v4-flash-free` (or any alias) - **Model**: `deepseek-v4-flash-free` (or any alias)
### Claude Code
Claude Code uses the Anthropic Messages API natively. For now, use an OpenAI-compatible bridge or configure via:
```bash
# In Claude Code, use as custom provider
claude config set provider_base_url http://127.0.0.1:6446/v1
```
### OpenCode CLI
Add to `~/.config/opencode/opencode.json`:
```json
{
"provider": {
"free": {
"name": "free",
"type": "openai",
"apiKey": "public",
"baseURL": "http://127.0.0.1:6446/v1",
"models": {
"free/deepseek": {
"id": "deepseek-v4-flash-free",
"name": "free/deepseek"
}
}
}
}
}
```
### Any OpenAI SDK ### Any OpenAI SDK
```python ```python
@@ -115,7 +126,7 @@ from openai import OpenAI
client = OpenAI( client = OpenAI(
base_url="http://127.0.0.1:6446/v1", base_url="http://127.0.0.1:6446/v1",
api_key="your-proxy-key" # or "public" if no auth api_key="your-proxy-key"
) )
response = client.chat.completions.create( response = client.chat.completions.create(
@@ -128,35 +139,31 @@ response = client.chat.completions.create(
```bash ```bash
# Build for Linux # Build for Linux
GOOS=linux GOARCH=amd64 go build -o opencode-proxy . GOOS=linux GOARCH=amd64 go build -o free-ide-proxy .
# Copy and run # Copy and run
scp opencode-proxy user@vps:/home/user/ scp free-ide-proxy user@vps:/home/user/
ssh user@vps './opencode-proxy -api-key "secure-key" -host 0.0.0.0' ssh user@vps './free-ide-proxy -api-key "secure-key" -host 0.0.0.0'
# Or use systemd (create /etc/systemd/system/opencode-proxy.service)
``` ```
systemd unit: ### systemd service
```ini ```ini
[Unit] [Unit]
Description=OpenCode Zen Proxy Description=Free IDE Proxy
After=network.target After=network.target
[Service] [Service]
ExecStart=/home/user/opencode-proxy -api-key "${PROXY_KEY}" -host 0.0.0.0 ExecStart=/home/user/free-ide-proxy -api-key "${PROXY_KEY}" -host 0.0.0.0
Restart=always Restart=always
User=user User=user
EnvironmentFile=/etc/opencode-proxy.env EnvironmentFile=/etc/free-ide-proxy.env
[Install] [Install]
WantedBy=multi-user.target WantedBy=multi-user.target
``` ```
## Local SSH tunnel ### Local SSH tunnel
If you don't want to expose the port publicly:
```bash ```bash
ssh -L 6446:127.0.0.1:6446 user@your-vps ssh -L 6446:127.0.0.1:6446 user@your-vps
@@ -164,6 +171,28 @@ ssh -L 6446:127.0.0.1:6446 user@your-vps
Then point your tools at `http://127.0.0.1:6446/v1`. Then point your tools at `http://127.0.0.1:6446/v1`.
## Quirks & Gotchas
- **No tests** exist yet. All manual testing.
- **In-memory only** — session and health state dies on restart. Health map auto-rebuilds via initial scan.
- **Health check logs** show rate-limit activity: `sudo journalctl -u free-ide-proxy | grep health`
- **Unknown models** are forwarded as-is via the default backend (opencode) — catch-all for new models.
- **Reasoning model caveat:** DeepSeek, Nemotron and StepFun need `max_tokens` ≥ 500; with very low values they return empty content. This is model behaviour, not a proxy bug.
## Architecture
```
Client → internal/server/ (HTTP, auth, routing)
→ internal/proxy/ (model resolve, backend router, session mgmt,
forwarding, Anthropic↔OpenAI conversion)
→ Backend → upstream
```
- **Pure stdlib Go 1.22** — no router, no dependencies beyond stdlib.
- **Multi-backend** via `Backend` interface (`internal/proxy/backend.go`). Add a new backend by implementing the interface and registering it in `NewProxy` (`models.go`).
- **Model aliases** live in each backend (`opencode.go`, `kilo.go`).
- **Auto rate-limit jailing** built into `Models()` — models returning 429 are hidden until they recover.
## License ## License
MIT MIT
internal/proxy/anthropic.go
+16 -9
View File
@@ -652,15 +652,19 @@ func (p *Proxy) ForwardAnthropicMessages(w http.ResponseWriter, r *http.Request,
return fmt.Errorf("invalid Anthropic request: %w", err) return fmt.Errorf("invalid Anthropic request: %w", err)
} }
// Resolve model // Route to the backend that owns this model (default = pass-through).
resolved, _ := p.ResolveModel(ar.Model) backend, resolved, known := p.resolveModel(ar.Model)
if !known {
resolved = ar.Model
}
// Keep original client-requested model name for response rewriting. // Keep original client-requested model name for response rewriting.
// Claude Code validates that the response model matches the request model. // Claude Code validates that the response model matches the request model.
clientModel := ar.Model clientModel := ar.Model
// Auto-route to vision model if the request has images and resolved model can't handle them // Auto-route to a vision model only inside the opencode backend (which owns
// a vision-capable free model). Kilo models are forwarded as-is.
hasImages := requestHasImages(&ar) hasImages := requestHasImages(&ar)
if hasImages { if hasImages && backend.Name() == "opencode" {
mi := ModelByID(resolved) mi := ModelByID(resolved)
if mi == nil || !mi.SupportsVision { if mi == nil || !mi.SupportsVision {
fmt.Printf("[anthropic] model=%s lacks vision, auto-routing to mimo-v2.5-free\n", resolved) fmt.Printf("[anthropic] model=%s lacks vision, auto-routing to mimo-v2.5-free\n", resolved)
@@ -668,8 +672,8 @@ func (p *Proxy) ForwardAnthropicMessages(w http.ResponseWriter, r *http.Request,
} }
} }
fmt.Printf("[anthropic] model=%s → %s, max_tokens=%d, stream=%v, messages=%d, tools=%d\n", fmt.Printf("[anthropic] model=%s → %s via %s, max_tokens=%d, stream=%v, messages=%d, tools=%d\n",
clientModel, resolved, ar.MaxTokens, ar.Stream, len(ar.Messages), len(ar.Tools)) clientModel, resolved, backend.Name(), ar.MaxTokens, ar.Stream, len(ar.Messages), len(ar.Tools))
// Convert Anthropic → OpenAI // Convert Anthropic → OpenAI
oaReq := AnthropicToOpenAI(&ar) oaReq := AnthropicToOpenAI(&ar)
@@ -686,13 +690,13 @@ func (p *Proxy) ForwardAnthropicMessages(w http.ResponseWriter, r *http.Request,
requestID := RequestID() requestID := RequestID()
sessionID := p.SessionID("anthropic") sessionID := p.SessionID("anthropic")
// Build upstream request // Build upstream request to the chosen backend
upstreamReq, err := http.NewRequestWithContext(r.Context(), "POST", ZenChatURL(), bytes.NewReader(oaBody)) upstreamReq, err := http.NewRequestWithContext(r.Context(), "POST", backend.ChatURL(), bytes.NewReader(oaBody))
if err != nil { if err != nil {
return fmt.Errorf("failed to create upstream request: %w", err) return fmt.Errorf("failed to create upstream request: %w", err)
} }
for k, v := range UpstreamHeaders(requestID, sessionID) { for k, v := range backend.Headers(requestID, sessionID) {
upstreamReq.Header.Set(k, v) upstreamReq.Header.Set(k, v)
} }
@@ -709,6 +713,9 @@ func (p *Proxy) ForwardAnthropicMessages(w http.ResponseWriter, r *http.Request,
if resp.StatusCode != 200 { if resp.StatusCode != 200 {
bodyBytes, _ := io.ReadAll(resp.Body) bodyBytes, _ := io.ReadAll(resp.Body)
fmt.Printf("[anthropic] upstream error body (%d bytes): %s\n", len(bodyBytes), string(bodyBytes)) fmt.Printf("[anthropic] upstream error body (%d bytes): %s\n", len(bodyBytes), string(bodyBytes))
if resp.StatusCode == http.StatusTooManyRequests {
p.markRateLimited(resolved)
}
w.Header().Set("Content-Type", "application/json") w.Header().Set("Content-Type", "application/json")
w.WriteHeader(resp.StatusCode) w.WriteHeader(resp.StatusCode)
json.NewEncoder(w).Encode(AnthropicError{ json.NewEncoder(w).Encode(AnthropicError{
internal/proxy/backend.go
+27
View File
@@ -0,0 +1,27 @@
package proxy
// Backend represents one upstream free-model provider (opencode zen, kilo, mimo, ...).
//
// Each backend knows its own model catalogue, how to resolve a requested model
// name to the upstream ID, and the URL + headers required to forward a chat
// completion request to it. The Proxy holds an ordered list of backends and
// routes each request to the first one that recognises the model.
type Backend interface {
// Name is a short stable identifier ("opencode", "kilo").
Name() string
// Models returns the free models this backend exposes, for /v1/models listing.
Models() []ModelInfo
// Resolve maps a client-requested model name to the upstream model ID.
// Returns ok=false when this backend does not recognise the model, so the
// router can try the next backend (or fall back to pass-through).
Resolve(model string) (resolved string, ok bool)
// ChatURL is the full upstream chat completions URL.
ChatURL() string
// Headers returns the headers required to authenticate/identify the
// request to the upstream (auth, client-id, request/session tracking, etc.).
Headers(requestID, sessionID string) map[string]string
}
internal/proxy/kilo.go
+91
View File
@@ -0,0 +1,91 @@
package proxy
// KiloGatewayBase is the Kilo Code public gateway base URL.
// Confirmed working without auth for free models (HTTP 200 on both
// GET /api/gateway/models and POST /api/gateway/chat/completions).
const KiloGatewayBase = "https://api.kilo.ai/api/gateway"
// KiloBackend forwards to Kilo Code's free gateway models.
//
// Kilo is an opencode-based CLI whose gateway exposes a handful of free models
// (suffixed ":free") reachable without authentication, in standard OpenAI
// Chat Completions format.
type KiloBackend struct{}
// kiloFreeModels are the free models advertised by the Kilo gateway.
// Sourced from GET /api/gateway/models filtering isFree:true.
var kiloFreeModels = []ModelInfo{
{
ID: "stepfun/step-3.7-flash:free",
Name: "StepFun Step 3.7 Flash (Free)",
OwnedBy: "stepfun",
ContextWindow: 256_000,
MaxOutputTokens: 32_000,
Description: "StepFun. Flash rápido, tareas generales.",
},
{
ID: "poolside/laguna-m.1:free",
Name: "Poolside Laguna M.1 (Free)",
OwnedBy: "poolside",
ContextWindow: 256_000,
MaxOutputTokens: 32_000,
Description: "Poolside. Modelo orientado a código.",
},
{
ID: "nvidia/nemotron-3-ultra-550b-a55b:free",
Name: "Nemotron 3 Ultra 550B (Free)",
OwnedBy: "nvidia",
ContextWindow: 1_000_000,
MaxOutputTokens: 16_384,
Description: "NVIDIA Nemotron 3 Ultra (550B/55B MoE). 1M contexto, razonamiento.",
},
{
ID: "openrouter/free",
Name: "OpenRouter Best Free (Free)",
OwnedBy: "openrouter",
ContextWindow: 256_000,
MaxOutputTokens: 32_000,
Description: "OpenRouter selecciona automáticamente el mejor modelo free disponible.",
},
}
// kiloAliases lets clients use shorter names for Kilo free models.
var kiloAliases = map[string]string{
"stepfun": "stepfun/step-3.7-flash:free",
"stepfun-free": "stepfun/step-3.7-flash:free",
"poolside": "poolside/laguna-m.1:free",
"poolside-free": "poolside/laguna-m.1:free",
"laguna": "poolside/laguna-m.1:free",
"openrouter": "openrouter/free",
}
// Name returns the backend identifier.
func (KiloBackend) Name() string { return "kilo" }
// Models returns the Kilo free models.
func (KiloBackend) Models() []ModelInfo { return kiloFreeModels }
// Resolve maps a requested name to a Kilo free model ID.
func (KiloBackend) Resolve(model string) (string, bool) {
if mapped, ok := kiloAliases[model]; ok {
return mapped, true
}
for _, m := range kiloFreeModels {
if m.ID == model {
return model, true
}
}
return "", false
}
// ChatURL is the Kilo gateway chat completions endpoint.
func (KiloBackend) ChatURL() string { return KiloGatewayBase + "/chat/completions" }
// Headers returns minimal headers for the Kilo gateway (no auth needed for free).
func (KiloBackend) Headers(_, _ string) map[string]string {
return map[string]string{
"Content-Type": "application/json",
"Accept": "text/event-stream, application/json",
"User-Agent": "kilo/7.3.54",
}
}
internal/proxy/models.go
+226 -126
View File
@@ -1,98 +1,16 @@
package proxy package proxy
import ( import (
"bytes"
"crypto/rand" "crypto/rand"
"encoding/hex" "encoding/hex"
"fmt"
"net/http"
"sync" "sync"
"time" "time"
) )
// ZenBaseURL is the upstream OpenCode Zen API base URL. // ModelInfo describes a single model exposed by a backend.
const ZenBaseURL = "https://opencode.ai/zen/v1"
// Free models available via OpenCode Zen (no API key required).
// These are the 5 models currently offered for free.
var zenModels = []ModelInfo{
{
ID: "big-pickle",
Name: "Big Pickle (Free)",
OwnedBy: "opencode-zen",
ContextWindow: 200_000,
MaxOutputTokens: 32_000,
Description: "Permanentemente gratis. Código legacy, refactors, mantenimiento.",
SupportsVision: false,
},
{
ID: "deepseek-v4-flash-free",
Name: "DeepSeek V4 Flash (Free)",
OwnedBy: "deepseek",
ContextWindow: 1_000_000,
MaxOutputTokens: 384_000,
Description: "El todoterreno. 1M contexto, 284B params (13B activos), razonamiento.",
SupportsVision: false,
},
{
ID: "mimo-v2.5-free",
Name: "MiMo V2.5 (Free)",
OwnedBy: "xiaomi",
ContextWindow: 1_000_000,
MaxOutputTokens: 32_000,
Description: "Multimodal de Xiaomi. 1M contexto, imágenes → código. MIT license.",
SupportsVision: true,
},
{
ID: "north-mini-code-free",
Name: "North Mini Code (Free)",
OwnedBy: "cohere",
ContextWindow: 256_000,
MaxOutputTokens: 64_000,
Description: "Cohere. 30B total / 3B activos (MoE). Respuesta rápida, scripts.",
SupportsVision: false,
},
{
ID: "nemotron-3-ultra-free",
Name: "Nemotron 3 Ultra (Free)",
OwnedBy: "nvidia",
ContextWindow: 1_000_000,
MaxOutputTokens: 16_384,
Description: "NVIDIA. 550B params (55B activos). Mamba-2 + MoE híbrido. Razonamiento.",
SupportsVision: false,
},
}
// Model aliases: short name → full model ID.
var modelAliases = map[string]string{
// Short aliases
"pickle": "big-pickle",
"big-pickle": "big-pickle",
"deepseek": "deepseek-v4-flash-free",
"deepseek-v4": "deepseek-v4-flash-free",
"ds": "deepseek-v4-flash-free",
"mimo": "mimo-v2.5-free",
"mimo-v2.5": "mimo-v2.5-free",
"xiaomi": "mimo-v2.5-free",
"north": "north-mini-code-free",
"north-mini": "north-mini-code-free",
"cohere": "north-mini-code-free",
"nemotron": "nemotron-3-ultra-free",
"nemotron-3": "nemotron-3-ultra-free",
"nvidia": "nemotron-3-ultra-free",
// Claude model name aliases (for Claude Code compatibility)
// Claude Code validates model names client-side; set ANTHROPIC_MODEL to a Claude name.
"claude-sonnet-4-6": "deepseek-v4-flash-free",
"claude-sonnet-4-5": "deepseek-v4-flash-free",
"claude-sonnet-4": "deepseek-v4-flash-free",
"claude-opus-4-8": "deepseek-v4-flash-free",
"claude-opus-4-5": "deepseek-v4-flash-free",
"claude-opus-4": "deepseek-v4-flash-free",
"claude-haiku-4-5": "north-mini-code-free",
"claude-haiku-4": "north-mini-code-free",
"claude-3.5-sonnet": "north-mini-code-free",
"claude-3.5-haiku": "big-pickle",
}
// ModelInfo describes a single model.
type ModelInfo struct { type ModelInfo struct {
ID string `json:"id"` ID string `json:"id"`
Name string `json:"name,omitempty"` Name string `json:"name,omitempty"`
@@ -103,11 +21,31 @@ type ModelInfo struct {
SupportsVision bool `json:"supports_vision"` SupportsVision bool `json:"supports_vision"`
} }
// Proxy handles forwarding OpenAI-format requests to OpenCode Zen. // healthCheckInterval is how often caged models get re-probed.
const healthCheckInterval = 30 * time.Minute
// healthPingTimeout caps each health probe. Set high enough for slow reasoning
// models (nvidia/nemotron can take ~80s to first token).
const healthPingTimeout = 90 * time.Second
// modelHealth tracks the rate-limit state of a single model.
type modelHealth struct {
status string // "active" | "rate_limited"
lastCheck time.Time
}
// Proxy routes OpenAI-format requests to the right Backend and manages session IDs.
type Proxy struct { type Proxy struct {
apiKey string apiKey string
sessions map[string]*session // keyed by client-provided API key
backends []Backend
defaultBackend Backend
sessions map[string]*session
mu sync.Mutex mu sync.Mutex
health map[string]*modelHealth
healthMu sync.RWMutex
} }
type session struct { type session struct {
@@ -115,63 +53,74 @@ type session struct {
expiresAt time.Time expiresAt time.Time
} }
// NewProxy creates a new Proxy instance. // NewProxy creates a Proxy with the default set of free-model backends.
//
// Order matters: more specific backends come first so their model IDs win over
// the catch-all opencode aliases. The default backend is the fallback for
// unknown models (pass-through as-is).
func NewProxy(apiKey string) *Proxy { func NewProxy(apiKey string) *Proxy {
return &Proxy{ oc := OpenCodeBackend{}
kl := KiloBackend{}
p := &Proxy{
apiKey: apiKey, apiKey: apiKey,
backends: []Backend{kl, oc},
defaultBackend: oc,
sessions: make(map[string]*session), sessions: make(map[string]*session),
health: make(map[string]*modelHealth),
} }
go p.healthLoop()
return p
} }
// APIKey returns the configured API key (empty = no auth). // APIKey returns the configured proxy API key (empty = no auth).
func (p *Proxy) APIKey() string { return p.apiKey } func (p *Proxy) APIKey() string { return p.apiKey }
// Models returns the list of available free models. // Models returns the aggregated free-model catalogue across all backends,
func (p *Proxy) Models() []ModelInfo { return zenModels } // excluding any model currently caged for rate limiting (HTTP 429).
func (p *Proxy) Models() []ModelInfo {
// ModelByID returns the ModelInfo entry for a given model ID, or nil if not found. var all []ModelInfo
func ModelByID(id string) *ModelInfo { for _, b := range p.backends {
for _, m := range zenModels { for _, m := range b.Models() {
if m.ID == id { if !p.isRateLimited(m.ID) {
return &m all = append(all, m)
} }
} }
return nil }
return all
} }
// ResolveModel maps a requested model name to a Zen model ID. // BackendFor returns the backend that recognises the model, or the default
// Returns the resolved model ID and whether it was found. // backend (pass-through) when no backend claims it.
func (p *Proxy) BackendFor(model string) Backend {
b, _, _ := p.resolveModel(model)
return b
}
// resolveModel walks the backends in order and returns the first match.
// When nothing matches it returns the default backend with the model unchanged
// and ok=false, so callers can forward it as-is (pass-through).
func (p *Proxy) resolveModel(model string) (backend Backend, resolved string, ok bool) {
for _, b := range p.backends {
if r, matched := b.Resolve(model); matched {
return b, r, true
}
}
return p.defaultBackend, model, false
}
// ResolveModel maps a requested model name to its upstream ID (backend-aware).
// Kept for backwards compatibility with existing call sites.
func (p *Proxy) ResolveModel(requested string) (string, bool) { func (p *Proxy) ResolveModel(requested string) (string, bool) {
// Check aliases first (covers both short names and exact IDs) _, r, ok := p.resolveModel(requested)
if mapped, ok := modelAliases[requested]; ok { return r, ok
return mapped, true
}
// Check if it's already a valid model ID (belt and suspenders)
for _, m := range zenModels {
if m.ID == requested {
return requested, true
}
}
// Pass through unknown models (might be newly added upstream)
return requested, false
} }
// ZenChatURL returns the full upstream URL for chat completions. // SessionID returns a session ID for the given user key, rotating every 30 min.
func ZenChatURL() string {
return ZenBaseURL + "/chat/completions"
}
// ZenModelsURL returns the full upstream URL for model listing.
func ZenModelsURL() string {
return ZenBaseURL + "/models"
}
// SessionID returns a session ID for the given user key, rotating every 30 minutes.
func (p *Proxy) SessionID(userKey string) string { func (p *Proxy) SessionID(userKey string) string {
p.mu.Lock() p.mu.Lock()
defer p.mu.Unlock() defer p.mu.Unlock()
if s, ok := p.sessions[userKey]; ok && time.Now().Before(s.expiresAt) { if s, exists := p.sessions[userKey]; exists && time.Now().Before(s.expiresAt) {
return s.id return s.id
} }
@@ -192,3 +141,154 @@ func randomHex(n int) string {
rand.Read(b) rand.Read(b)
return hex.EncodeToString(b) return hex.EncodeToString(b)
} }
// ---------- Model health / rate-limit tracking ----------
// markRateLimited cages a model so it disappears from /v1/models until it recovers.
// Only logs the transition to avoid spam when many 429s arrive in a row.
func (p *Proxy) markRateLimited(modelID string) {
p.healthMu.Lock()
defer p.healthMu.Unlock()
h := p.health[modelID]
if h == nil {
h = &modelHealth{}
p.health[modelID] = h
}
if h.status != "rate_limited" {
fmt.Printf("[health] caging model '%s' (rate limited)\n", modelID)
}
h.status = "rate_limited"
h.lastCheck = time.Now()
}
// markActive restores a caged model so it reappears in /v1/models.
func (p *Proxy) markActive(modelID string) {
p.healthMu.Lock()
defer p.healthMu.Unlock()
h := p.health[modelID]
if h == nil {
h = &modelHealth{}
p.health[modelID] = h
}
h.status = "active"
h.lastCheck = time.Now()
}
// isRateLimited reports whether a model is currently caged for rate limiting.
func (p *Proxy) isRateLimited(modelID string) bool {
p.healthMu.RLock()
defer p.healthMu.RUnlock()
h, ok := p.health[modelID]
return ok && h.status == "rate_limited"
}
// healthLoop runs an initial full scan on startup, then re-probes caged models
// every healthCheckInterval (30 min). Released models reappear in /v1/models.
func (p *Proxy) healthLoop() {
p.scanAllModels()
ticker := time.NewTicker(healthCheckInterval)
defer ticker.Stop()
for range ticker.C {
p.recheckCagedModels()
}
}
// scanAllModels probes every known model once. Models returning non-200 (429)
// are caged; the rest stay active. Used for the startup scan.
func (p *Proxy) scanAllModels() {
var ids []string
for _, b := range p.backends {
for _, m := range b.Models() {
ids = append(ids, m.ID)
}
}
fmt.Printf("[health] initial scan: probing %d models\n", len(ids))
for _, id := range ids {
ok := p.pingModel(id)
p.healthMu.Lock()
h := p.health[id]
if h == nil {
h = &modelHealth{}
p.health[id] = h
}
h.lastCheck = time.Now()
if ok {
h.status = "active"
} else {
h.status = "rate_limited"
fmt.Printf("[health] caging model '%s' (rate limited on initial scan)\n", id)
}
p.healthMu.Unlock()
}
active := 0
p.healthMu.RLock()
for _, h := range p.health {
if h.status == "active" {
active++
}
}
p.healthMu.RUnlock()
fmt.Printf("[health] initial scan complete: %d active, %d caged\n", active, len(ids)-active)
}
// recheckCagedModels re-probes only the models currently caged. A 200 releases
// them; a 429 or timeout keeps them caged for another interval.
func (p *Proxy) recheckCagedModels() {
p.healthMu.RLock()
var caged []string
for id, h := range p.health {
if h.status == "rate_limited" {
caged = append(caged, id)
}
}
p.healthMu.RUnlock()
if len(caged) == 0 {
return
}
fmt.Printf("[health] re-checking %d caged models\n", len(caged))
for _, id := range caged {
ok := p.pingModel(id)
p.healthMu.Lock()
h := p.health[id]
if h == nil {
h = &modelHealth{}
p.health[id] = h
}
h.lastCheck = time.Now()
if ok {
h.status = "active"
fmt.Printf("[health] releasing model '%s' (recovered)\n", id)
} else {
fmt.Printf("[health] model '%s' still rate limited, retry in 30m\n", id)
}
p.healthMu.Unlock()
}
}
// pingModel sends a minimal chat completion to the model's backend and reports
// whether the upstream answered HTTP 200. Timeouts and non-200 count as
// unhealthy so the model stays caged (conservative).
func (p *Proxy) pingModel(modelID string) bool {
backend, resolved, known := p.resolveModel(modelID)
if !known {
resolved = modelID
}
body := fmt.Sprintf(`{"model":%q,"messages":[{"role":"user","content":"OK"}],"max_tokens":5,"temperature":0}`, resolved)
req, err := http.NewRequest("POST", backend.ChatURL(), bytes.NewReader([]byte(body)))
if err != nil {
return false
}
for k, v := range backend.Headers(RequestID(), "ses_health") {
req.Header.Set(k, v)
}
client := &http.Client{Timeout: healthPingTimeout}
resp, err := client.Do(req)
if err != nil {
return false
}
resp.Body.Close()
return resp.StatusCode == 200
}
internal/proxy/opencode.go
+142
View File
@@ -0,0 +1,142 @@
package proxy
// ZenBaseURL is the upstream OpenCode Zen API base URL.
const ZenBaseURL = "https://opencode.ai/zen/v1"
// OpenCodeBackend forwards to OpenCode's free Zen models.
type OpenCodeBackend struct{}
// zenModels are the free models exposed by OpenCode Zen (no API key required).
var zenModels = []ModelInfo{
{
ID: "big-pickle",
Name: "Big Pickle (Free)",
OwnedBy: "opencode-zen",
ContextWindow: 200_000,
MaxOutputTokens: 32_000,
Description: "Permanentemente gratis. Código legacy, refactors, mantenimiento.",
},
{
ID: "deepseek-v4-flash-free",
Name: "DeepSeek V4 Flash (Free)",
OwnedBy: "deepseek",
ContextWindow: 1_000_000,
MaxOutputTokens: 384_000,
Description: "El todoterreno. 1M contexto, 284B params (13B activos), razonamiento.",
},
{
ID: "mimo-v2.5-free",
Name: "MiMo V2.5 (Free)",
OwnedBy: "xiaomi",
ContextWindow: 1_000_000,
MaxOutputTokens: 32_000,
Description: "Multimodal de Xiaomi. 1M contexto, imágenes → código. MIT license.",
SupportsVision: true,
},
{
ID: "north-mini-code-free",
Name: "North Mini Code (Free)",
OwnedBy: "cohere",
ContextWindow: 256_000,
MaxOutputTokens: 64_000,
Description: "Cohere. 30B total / 3B activos (MoE). Respuesta rápida, scripts.",
},
{
ID: "nemotron-3-ultra-free",
Name: "Nemotron 3 Ultra (Free)",
OwnedBy: "nvidia",
ContextWindow: 1_000_000,
MaxOutputTokens: 16_384,
Description: "NVIDIA. 550B params (55B activos). Mamba-2 + MoE híbrido. Razonamiento.",
},
}
// modelAliases maps short/company/Claude names to OpenCode Zen model IDs.
var modelAliases = map[string]string{
// Short aliases
"pickle": "big-pickle",
"big-pickle": "big-pickle",
"deepseek": "deepseek-v4-flash-free",
"deepseek-v4": "deepseek-v4-flash-free",
"ds": "deepseek-v4-flash-free",
"mimo": "mimo-v2.5-free",
"mimo-v2.5": "mimo-v2.5-free",
"xiaomi": "mimo-v2.5-free",
"north": "north-mini-code-free",
"north-mini": "north-mini-code-free",
"cohere": "north-mini-code-free",
"nemotron": "nemotron-3-ultra-free",
"nemotron-3": "nemotron-3-ultra-free",
"nvidia": "nemotron-3-ultra-free",
// Exact IDs
"deepseek-v4-flash-free": "deepseek-v4-flash-free",
"mimo-v2.5-free": "mimo-v2.5-free",
"north-mini-code-free": "north-mini-code-free",
"nemotron-3-ultra-free": "nemotron-3-ultra-free",
// Claude model name aliases (Claude Code validates model names client-side).
"claude-sonnet-4-6": "deepseek-v4-flash-free",
"claude-sonnet-4-5": "deepseek-v4-flash-free",
"claude-sonnet-4": "deepseek-v4-flash-free",
"claude-opus-4-8": "deepseek-v4-flash-free",
"claude-opus-4-5": "deepseek-v4-flash-free",
"claude-opus-4": "deepseek-v4-flash-free",
"claude-haiku-4-5": "north-mini-code-free",
"claude-haiku-4": "north-mini-code-free",
"claude-3.5-sonnet": "north-mini-code-free",
"claude-3.5-haiku": "big-pickle",
}
// Name returns the backend identifier.
func (OpenCodeBackend) Name() string { return "opencode" }
// Models returns the OpenCode Zen free models.
func (OpenCodeBackend) Models() []ModelInfo { return zenModels }
// Resolve maps a requested name to an OpenCode Zen model ID.
func (OpenCodeBackend) Resolve(model string) (string, bool) {
if mapped, ok := modelAliases[model]; ok {
return mapped, true
}
for _, m := range zenModels {
if m.ID == model {
return model, true
}
}
return "", false
}
// ChatURL is the OpenCode Zen chat completions endpoint.
func (OpenCodeBackend) ChatURL() string { return ZenBaseURL + "/chat/completions" }
// Headers builds the headers OpenCode Zen expects (x-opencode-* + public bearer).
func (OpenCodeBackend) Headers(requestID, sessionID string) map[string]string {
return map[string]string{
"Authorization": "Bearer public",
"User-Agent": "opencode/1.15.0 ai-sdk/provider-utils/4.0.23 runtime/bun/1.3.13",
"x-opencode-client": "cli",
"x-opencode-project": "global",
"x-opencode-request": requestID,
"x-opencode-session": sessionID,
"Content-Type": "application/json",
"Accept": "text/event-stream, application/json",
}
}
// ModelByID returns the OpenCode Zen ModelInfo for an ID, or nil if not found.
// Used by the Anthropic path for vision auto-routing.
func ModelByID(id string) *ModelInfo {
for _, m := range zenModels {
if m.ID == id {
return &m
}
}
return nil
}
// ZenChatURL returns the OpenCode Zen chat completions URL.
// Kept for backwards compatibility with existing call sites.
func ZenChatURL() string { return ZenBaseURL + "/chat/completions" }
// ZenModelsURL returns the OpenCode Zen models listing URL.
func ZenModelsURL() string { return ZenBaseURL + "/models" }
internal/proxy/proxy.go
+16 -24
View File
@@ -104,32 +104,20 @@ type OpenAIErrorDetail struct {
// ---------- Proxy forwarding logic ---------- // ---------- Proxy forwarding logic ----------
// UpstreamHeaders builds the required headers for OpenCode Zen. // ForwardChatCompletion routes the OpenAI request to the matching backend and
func UpstreamHeaders(requestID, sessionID string) map[string]string { // streams or collects the response.
return map[string]string{
"Authorization": "Bearer public",
"User-Agent": "opencode/1.15.0 ai-sdk/provider-utils/4.0.23 runtime/bun/1.3.13",
"x-opencode-client": "cli",
"x-opencode-project": "global",
"x-opencode-request": requestID,
"x-opencode-session": sessionID,
"Content-Type": "application/json",
"Accept": "text/event-stream, application/json",
}
}
// ForwardChatCompletion sends the OpenAI request to OpenCode Zen and streams or collects the response.
func (p *Proxy) ForwardChatCompletion(w http.ResponseWriter, r *http.Request, body []byte) error { func (p *Proxy) ForwardChatCompletion(w http.ResponseWriter, r *http.Request, body []byte) error {
var req OpenAIChatRequest var req OpenAIChatRequest
if err := json.Unmarshal(body, &req); err != nil { if err := json.Unmarshal(body, &req); err != nil {
return fmt.Errorf("invalid request body: %w", err) return fmt.Errorf("invalid request body: %w", err)
} }
// Resolve model name // Route to the backend that owns this model (default backend = pass-through).
resolved, ok := p.ResolveModel(req.Model) backend, resolved, known := p.resolveModel(req.Model)
if !ok { if !known {
// Unknown model — log but still forward (might be a new model) fmt.Printf("[proxy] unknown model '%s', forwarding as-is via %s\n", req.Model, backend.Name())
fmt.Printf("[proxy] unknown model '%s', forwarding as-is\n", req.Model) } else {
fmt.Printf("[proxy] model '%s' -> '%s' via %s\n", req.Model, resolved, backend.Name())
} }
// Update the body with resolved model // Update the body with resolved model
@@ -144,7 +132,7 @@ func (p *Proxy) ForwardChatCompletion(w http.ResponseWriter, r *http.Request, bo
// Generate request/session IDs for upstream // Generate request/session IDs for upstream
requestID := RequestID() requestID := RequestID()
// Get session ID from the client's proxy API key (or default) // Session ID keyed by the client's proxy API key (or default)
userKey := "default" userKey := "default"
if p.apiKey != "" { if p.apiKey != "" {
userKey = r.Header.Get("Authorization") userKey = r.Header.Get("Authorization")
@@ -154,13 +142,13 @@ func (p *Proxy) ForwardChatCompletion(w http.ResponseWriter, r *http.Request, bo
} }
sessionID := p.SessionID(userKey) sessionID := p.SessionID(userKey)
// Build upstream request // Build upstream request to the chosen backend
upstreamReq, err := http.NewRequestWithContext(r.Context(), "POST", ZenChatURL(), bytes.NewReader(body)) upstreamReq, err := http.NewRequestWithContext(r.Context(), "POST", backend.ChatURL(), bytes.NewReader(body))
if err != nil { if err != nil {
return fmt.Errorf("failed to create upstream request: %w", err) return fmt.Errorf("failed to create upstream request: %w", err)
} }
for k, v := range UpstreamHeaders(requestID, sessionID) { for k, v := range backend.Headers(requestID, sessionID) {
upstreamReq.Header.Set(k, v) upstreamReq.Header.Set(k, v)
} }
@@ -174,6 +162,10 @@ func (p *Proxy) ForwardChatCompletion(w http.ResponseWriter, r *http.Request, bo
// Handle non-200 responses // Handle non-200 responses
if resp.StatusCode != 200 { if resp.StatusCode != 200 {
fmt.Printf("[proxy] upstream status=%d for model='%s' via %s\n", resp.StatusCode, resolved, backend.Name())
if resp.StatusCode == http.StatusTooManyRequests {
p.markRateLimited(resolved)
}
bodyBytes, _ := io.ReadAll(resp.Body) bodyBytes, _ := io.ReadAll(resp.Body)
w.Header().Set("Content-Type", "application/json") w.Header().Set("Content-Type", "application/json")
w.WriteHeader(resp.StatusCode) w.WriteHeader(resp.StatusCode)